Intelligent highway inspection control system based on multimodal data fusion

Abstract

The present invention discloses an intelligent highway inspection and control system based on multimodal data fusion, comprising a multimodal data acquisition module; a data fusion and preprocessing module; an intelligent inspection and analysis module; an inspection decision and execution module; and a system management and user interaction module. By integrating multiple data sources such as video surveillance data, vehicle traffic data, and meteorological data, real-time monitoring and intelligent analysis of highway traffic are achieved, and abnormal behaviors and violations of vehicles can be identified quickly and accurately, significantly improving inspection efficiency and accuracy. The system overcomes the defects of traditional inspection methods such as reliance on manual labor, low efficiency, insufficient accuracy, and poor real-time performance. It can discover and handle violations in real time, enhance the timeliness and deterrent effect of inspections, improve management efficiency and user experience, and thus effectively ensure the safety and smooth flow of highways, providing traffic management departments with an efficient and reliable intelligent inspection solution.

Description

Technical Field

[0001] The present invention relates to the technical field of highway inspection, and in particular to an intelligent highway inspection control system based on multimodal data fusion. Background Art

[0002] With the continuous expansion of the expressway network and the continued growth of traffic volume, highway management and inspection are facing increasing challenges. Traditional inspection methods rely primarily on manual inspections and monitoring of single data sources. For example, these methods rely on manual inspections, simple sensor monitoring at toll booths, and video surveillance to identify and address violations. While these methods can meet basic inspection needs to a certain extent, their limitations are becoming increasingly apparent as traffic volume and complexity increase.

[0003] Existing technologies suffer from multiple flaws. First, manual inspections are inefficient and susceptible to human error, leading to inconsistent and inaccurate inspection results. Second, monitoring from a single data source fails to provide comprehensive road condition information, making it easy to miss important abnormal behaviors and violations. Furthermore, traditional inspection methods lack real-time performance, preventing them from promptly detecting and addressing violations, resulting in insufficient timeliness and deterrent effect. These issues not only impact highway operational efficiency but also pose a potential threat to traffic safety. Therefore, a more intelligent, efficient, and accurate inspection system is needed to address the shortcomings of existing technologies. Summary of the Invention

[0004] The purpose of this section is to summarize some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract and title of this application to avoid obscuring the purpose of this section, the abstract and the title of the invention, and such simplifications or omissions should not be used to limit the scope of the present invention.

[0005] In view of the above-mentioned problems existing in the existing highway intelligent inspection control system based on multimodal data fusion, the present invention is proposed.

[0006] Therefore, the purpose of the present invention is to provide an intelligent highway inspection and control system based on multimodal data fusion.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions: comprising:

[0008] A multimodal data acquisition module is used to collect various data on the highway, including video surveillance data, vehicle traffic data, and meteorological data;

[0009] The data fusion and preprocessing module is used to perform fusion processing on the collected multimodal data, and the fusion processing includes data cleaning, format conversion, and data association;

[0010] Intelligent inspection and analysis module, used to analyze the integrated data in real time, identify and judge abnormal behavior and violations of vehicles;

[0011] The audit decision and execution module is used to generate audit decisions based on the analysis results of the intelligent audit analysis module and perform audit operations through corresponding execution devices or systems;

[0012] The system management and user interaction module is used to provide system management and user interaction functions. The management functions include device management, user authority management, and data storage management. The user interaction functions include a user interaction interface.

[0013] As a preferred solution of the highway intelligent inspection and control system based on multimodal data fusion of the present invention, wherein: the multimodal data acquisition module includes:

[0014] Video surveillance equipment, used to collect real-time image and video information on highway conditions and vehicle behavior;

[0015] Vehicle traffic data collection equipment, used to record vehicle traffic information, including license plate number, vehicle model, and traffic time;

[0016] Meteorological sensors are used to monitor meteorological conditions along the highway, including temperature, humidity, and visibility.

[0017] As a preferred solution of the highway intelligent inspection and control system based on multimodal data fusion of the present invention, wherein: the data fusion and preprocessing module includes a data cleaning unit for removing noise and redundant information in the collected data;

[0018] A data conversion unit, used to convert data in different formats into a unified format;

[0019] The data association unit is used to associate multi-source data to form a comprehensive data set.

[0020] As a preferred solution of the highway intelligent inspection and control system based on multimodal data fusion described in the present invention, the intelligent inspection and analysis module includes:

[0021] A feature extraction unit is used to extract feature information of vehicle behavior from the fused data;

[0022] A behavior recognition unit, used to identify abnormal behavior of the vehicle based on the extracted feature information;

[0023] A violation determination unit, used to determine whether the abnormal behavior of the vehicle constitutes a violation and to determine the type of violation;

[0024] The violation determination unit includes:

[0025] A first determination subunit is configured to determine whether the abnormal behavior complies with a preset violation characteristic rule based on the characteristic information of the abnormal behavior, wherein the preset violation characteristic rule includes a speeding rule, a wrong-way driving rule, and an illegal lane change rule;

[0026] a second determination subunit, configured to determine a violation degree of the abnormal behavior based on a determination result of the first determination subunit, wherein the violation degree includes a minor violation, a general violation, and a serious violation;

[0027] a third determination subunit, configured to generate a violation determination result according to the determination result of the second determination subunit, wherein the violation determination result includes a violation type and a violation degree;

[0028] When the third determination subunit generates a violation determination result,

[0029] If the second determination subunit determines that the abnormal behavior is a serious violation, it directly generates a violation determination result and transmits the result to the audit decision and execution module;

[0030] If the second determination subunit determines that the abnormal behavior is a general violation or a minor violation, the process returns to the first determination subunit to re-evaluate whether the abnormal behavior meets stricter violation characteristic rules;

[0031] If the first determination subunit determines after re-evaluation that the abnormal behavior does not meet the stricter rule of violation characteristics, the third determination subunit generates a no-violation determination result and ends the determination process;

[0032] If the first determination subunit determines after re-evaluation that the abnormal behavior meets stricter violation characteristic rules, the second determination subunit re-determines the violation degree of the abnormal behavior, and the third determination subunit generates a violation determination result based on the new violation degree and passes the result to the audit decision and execution module.

[0033] As a preferred solution of the highway intelligent inspection control system based on multimodal data fusion described in the present invention, the inspection decision and execution module includes:

[0034] The audit decision-making unit is used to formulate audit measures based on the judgment results of the violation determination unit;

[0035] The execution unit is used to convey the inspection decision to the relevant departments or systems to achieve real-time inspection and processing of illegal vehicles.

[0036] As a preferred solution of the highway intelligent inspection control system based on multimodal data fusion described in the present invention, the system management and user interaction module includes:

[0037] Device management unit, used to monitor and maintain the system's hardware devices;

[0038] User authority management unit, used to provide different operation permissions and functions according to the user's role and authority;

[0039] Data storage management unit, used for system data storage and backup;

[0040] The user interaction interface is used to facilitate users to perform operations such as system configuration, data query and audit result viewing.

[0041] As a preferred solution of the highway intelligent inspection and control system based on multimodal data fusion described in the present invention, the data fusion and preprocessing module also includes a data verification unit for verifying the collected data to ensure the accuracy and integrity of the data.

[0042] As a preferred solution of the highway intelligent inspection and control system based on multimodal data fusion described in the present invention, the intelligent inspection and analysis module also includes a data analysis unit for performing statistical analysis on the fused data and generating an inspection report.

[0043] As a preferred solution of the highway intelligent inspection and control system based on multimodal data fusion described in the present invention, the inspection decision and execution module also includes an inspection result feedback unit, which is used to feed back the inspection results to relevant users or departments for subsequent processing and improvement.

[0044] As a preferred solution of the highway intelligent inspection and control system based on multimodal data fusion described in the present invention, the system management and user interaction module also includes a system monitoring unit for real-time monitoring of the system's operating status and timely detection and processing of system failures.

[0045] The beneficial effects of the present invention are as follows: by integrating multiple data sources such as video surveillance data, vehicle traffic data and meteorological data, and applying data fusion technology and algorithms, real-time monitoring and intelligent analysis of highway traffic is achieved, and abnormal behaviors and violations of vehicles can be identified quickly and accurately, thereby significantly improving the efficiency and accuracy of inspections. The system overcomes the defects of traditional inspection methods such as reliance on manual labor, low efficiency, insufficient accuracy and poor real-time performance. It can not only detect and handle violations in real time, enhance the timeliness and deterrent effect of inspections, but also improve management efficiency and user experience by optimizing system management, thereby effectively ensuring the safety and smooth flow of highways, and providing traffic management departments with an efficient and reliable intelligent inspection solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort. Among them:

[0047] Figure 1 This is a diagram of the computer equipment described in the highway intelligent inspection and control system based on multimodal data fusion of the present invention.

[0048] Figure 2 This is a schematic diagram of video surveillance for the intelligent highway inspection and control system based on multimodal data fusion according to the present invention.

[0049] Figure 3 This is a schematic diagram of traffic incident cause analysis in the highway intelligent inspection and control system based on multimodal data fusion according to the present invention. DETAILED DESCRIPTION

[0050] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0051] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0052] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0053] Furthermore, the present invention is described in detail with reference to schematic diagrams. For ease of illustration, when describing the embodiments of the present invention, cross-sectional views illustrating device structures may be partially enlarged and not to scale. Furthermore, the schematic diagrams are merely illustrative and should not limit the scope of protection of the present invention. Furthermore, in actual production, the three-dimensional dimensions of length, width, and depth should be included.

[0054] Example, see Figure 1-3 As shown in the figure, the intelligent inspection and control system for highways based on multimodal data fusion includes a multimodal data acquisition module for collecting a variety of data on highways. The various data include video surveillance data, vehicle traffic data, and meteorological data. This module collects information such as road conditions, vehicle behavior, and meteorological conditions in real time through video surveillance equipment, vehicle traffic data acquisition equipment, and meteorological sensors deployed at key sections and toll stations, providing comprehensive data support for subsequent analysis and processing. Figure 2 As shown, at the same time, for vehicles entering the jurisdiction, the real-time driving situation monitoring is carried out using the data of the entrance and exit toll stations and gantries. The data of vehicles that have not left the expressway for more than a certain period of time is pushed with early warnings. The data includes key information such as the last recorded gantry position, longitude and latitude, pile number, and dwell time. This effectively improves the timely tracking of vehicle abnormalities and the efficiency of handling accidents and faults. It makes full use of intelligent sensing equipment such as cameras and millimeter-wave radars along the route, and based on intelligent algorithm models such as event AI analysis, it uniformly summarizes and warns traffic events on the road, making it convenient for managers to quickly and accurately grasp road event information.

[0055] The data fusion and preprocessing module is used to fuse the collected multimodal data. This fusion process includes data cleaning, format conversion, and data association. This module cleans, converts, and associates the collected data from different formats and sources to form a comprehensive data set in a unified format, improving data quality and usability. Based on the summary statistics of road event information, it conducts statistical analysis on various data types, and then obtains analysis results such as vehicle type on the road section, event type, accident volume, accident location and season, passenger-to-freight ratio, number of abnormal vehicles, and traffic volume at toll stations. Based on the toll units, toll data, and historical data, it generates traffic flow data for the road network and analyzes it in the form of heat maps.

[0056] The intelligent inspection and analysis module is used to analyze the fused data in real time to identify and judge abnormal vehicle behavior and violations. This module uses advanced data analysis technology and artificial intelligence algorithms to conduct in-depth analysis of the fused data, and can quickly and accurately identify abnormal vehicle behavior and violations such as speeding, wrong-way driving, and illegal lane changes.

[0057] The audit decision and execution module is used to generate audit decisions based on the analysis results of the intelligent audit analysis module and perform audit operations through corresponding execution devices or systems. This module formulates corresponding audit measures based on the analysis results and conducts real-time audits and processing of illegal vehicles through execution devices or systems, thereby improving the efficiency and accuracy of audits;

[0058] The system management and user interaction module is used to provide system management and user interaction functions. The management functions include device management, user authority management, and data storage management. The user interaction functions include the user interaction interface. This module provides comprehensive system management and user interaction functions, making it convenient for users to perform operations such as system configuration, data query, and audit result viewing, thereby improving the system's usability and management efficiency.

[0059] Specifically, the multimodal data acquisition module includes:

[0060] Video surveillance equipment is used to collect real-time images and video information on highway road conditions and vehicle behavior. These devices are deployed at key sections of the highway and toll booths. They can provide real-time visual information on road conditions and vehicle behavior, providing intuitive data support for identifying abnormal behavior.

[0061] Vehicle traffic data collection equipment is used to record vehicle traffic information, including license plate number, vehicle model, and travel time. These devices can accurately record vehicle traffic information and provide basic data for subsequent violation analysis;

[0062] Meteorological sensors are used to monitor meteorological conditions along highways, including temperature, humidity, and visibility. They can monitor meteorological conditions in real time and provide environmental data to the inspection system, helping the system better understand and analyze vehicle behavior.

[0063] Furthermore, the data fusion and preprocessing module includes a data cleaning unit for removing noise and redundant information from the collected data. This unit can identify and remove errors and duplicates in the data to improve the accuracy and reliability of the data;

[0064] Data conversion unit, used to convert data in different formats into a unified format. This unit converts data from different sensors and devices into a unified format to facilitate subsequent fusion and analysis;

[0065] The data association unit is used to associate multi-source data to form a comprehensive data set. This unit associates data from different sources to form a comprehensive data set, providing comprehensive data support for subsequent analysis.

[0066] Specifically, the intelligent audit analysis module includes:

[0067] The feature extraction unit is used to extract characteristic information of vehicle behavior from the fused data. This unit can identify and extract key features of vehicle behavior, providing a basis for subsequent behavior recognition and violation determination;

[0068] The behavior recognition unit is used to identify abnormal vehicle behavior based on extracted feature information. This unit uses advanced data analysis technology and artificial intelligence algorithms to quickly and accurately identify abnormal vehicle behavior, such as speeding, driving against traffic, and illegal lane changes;

[0069] A violation determination unit is used to determine whether the abnormal behavior of the vehicle constitutes a violation and to determine the type of violation. This unit determines the violation of the identified abnormal behavior based on preset rules and standards and determines the specific type of violation;

[0070] The violation determination unit includes:

[0071] The first determination subunit is configured to determine whether the abnormal behavior complies with a preset violation characteristic rule based on characteristic information of the abnormal behavior, wherein the preset violation characteristic rule includes a speeding rule, a wrong-way driving rule, and an illegal lane change rule;

[0072] The second determination subunit is used to determine the violation degree of the abnormal behavior based on the determination result of the first determination subunit, where the violation degree includes minor violation, general violation, and serious violation;

[0073] A third determination subunit is configured to generate a violation determination result according to the determination result of the second determination subunit, where the violation determination result includes a violation type and a violation degree;

[0074] When the third determination subunit generates a violation determination result,

[0075] If the second determination subunit determines that the abnormal behavior is a serious violation, it directly generates a violation determination result and transmits the result to the audit decision and execution module;

[0076] If the second determination subunit determines that the abnormal behavior is a general violation or a minor violation, it returns to the first determination subunit to re-evaluate whether the abnormal behavior meets the stricter violation characteristic rules;

[0077] If the first determination subunit determines after re-evaluation that the abnormal behavior does not meet the stricter rule for the characteristics of illegal behavior, the third determination subunit generates a no-violation determination result and ends the determination process;

[0078] If the first determination subunit determines after re-evaluation that the abnormal behavior meets the stricter violation characteristic rules, the second determination subunit re-determines the violation degree of the abnormal behavior, and the third determination subunit generates a violation determination result based on the new violation degree and transmits the result to the audit decision and execution module. Figure 3 Analysis and determination of the causes of traffic accidents.

[0079] Specifically, the audit decision-making and execution module includes:

[0080] The inspection decision unit is used to formulate inspection measures based on the judgment results of the violation judgment unit. This unit formulates corresponding inspection measures based on the violation judgment results, such as warnings, fines, and point deductions. By monitoring and analyzing the traffic flow status along the route and severe weather, it realizes real-time analysis and auxiliary judgment of the current route traffic flow operation status, and obtains the speed limit value of the road section based on the pre-set traffic control strategy, providing decision support for the implementation of dynamic speed limit, dynamic flow limit, passenger and freight separation and other control strategies for highway traffic flow. Based on the access to real-time meteorological monitoring data, the road section units affected by severe weather conditions are determined, and the traffic condition warning level of each road section unit is determined according to the traffic condition classification standard; combined with real-time data of traffic situation, traffic road condition forecast, and traffic event information, the road network traffic status is analyzed, and a traffic induction model based on the real-time road network traffic status is constructed to induce the operation path distribution of driving vehicles, thereby optimizing the traffic organization and induction of the road network and alleviating traffic congestion;

[0081] During severe weather traffic control,

[0082] ①Graded warning of traffic conditions

[0083] Based on access to real-time meteorological monitoring data, the road sections affected by severe weather conditions are identified, and the traffic condition warning level of each road section is determined according to the traffic condition classification standards.

[0084] ② Decision-making on driving safety control in severe weather

[0085] Driving safety management and control decisions: Based on the variable speed limit and traffic restriction control strategy model in severe weather and the traffic condition classification results, output the recommended digital speed limit and traffic restriction control plan;

[0086] The execution unit is used to convey the inspection decision to the relevant departments or systems to achieve real-time inspection and processing of illegal vehicles. This unit quickly conveys and executes the inspection decision through the interface with the relevant departments or systems, thereby improving the efficiency and accuracy of the inspection.

[0087] Furthermore, the system management and user interaction module includes:

[0088] The device management unit is used to monitor and maintain the system's hardware devices. This unit can monitor the status of hardware devices in real time, detect and handle device failures in a timely manner, and ensure the stable operation of the system;

[0089] The user rights management unit is used to provide different operating rights and functions according to the user's role and rights. This unit provides corresponding operating rights and functions according to the user's role and rights to ensure the security and confidentiality of the system;

[0090] The data storage management unit is used for system data storage and backup. This unit is responsible for the system's data storage and backup work to ensure the security and integrity of the data;

[0091] The user interface is used to facilitate users to perform operations such as system configuration, data query and audit result viewing. This interface provides an intuitive operation interface to facilitate users to perform operations such as system configuration, data query and audit result viewing;

[0092] Based on traffic flow change trends, visibility or weather change trends, traffic accidents, and construction progress, information boards are automatically released to remind drivers to choose the appropriate lane and speed.

[0093] Furthermore, the data fusion and preprocessing module also includes a data verification unit, which is used to verify the collected data to ensure the accuracy and completeness of the data. It can verify the collected data to ensure the accuracy and completeness of the data, provide a reliable data basis for subsequent analysis and processing, and integrate various real-time event information obtained from channels such as manual reporting, traffic flow monitoring, event monitoring, and meteorological monitoring.

[0094] Furthermore, the intelligent inspection and analysis module also includes a data analysis unit for performing statistical analysis on the fused data and generating an inspection report. This unit can perform statistical analysis on the fused data and generate detailed inspection reports to provide decision support for the traffic management department.

[0095] In particular, the audit decision and execution module also includes an audit result feedback unit, which is used to feed back the audit results to relevant users or departments for subsequent processing and improvement. This unit can promptly feed back the audit results to relevant users or departments for subsequent processing and improvement.

[0096] It should be noted that the system management and user interaction module also includes a system monitoring unit, which is used to monitor the system's operating status in real time, promptly detect and handle system failures, and ensure the stable operation of the system.

[0097] Operational Process: In a highway intelligent inspection and control system based on multimodal data fusion, the operational process first involves the deployment of hardware equipment, including the installation of video surveillance equipment, vehicle traffic data acquisition equipment, and meteorological sensors at key sections and toll booths. Data servers and storage devices are deployed to support data storage and processing, and system management and user interaction modules are configured. Subsequently, the software system is deployed, covering the multimodal data acquisition module, data fusion and preprocessing module, intelligent inspection and analysis module, and inspection decision-making and execution module. Parameters for each module are configured to ensure the normal operation of the system. During the data acquisition phase, video surveillance equipment collects real-time image and video information on road conditions and vehicle behavior. Vehicle traffic data acquisition equipment records vehicle traffic information, such as license plate number, vehicle model, and travel time. Meteorological sensors monitor weather conditions along the route, including temperature, humidity, and visibility. Entering the data fusion and preprocessing phase, the data cleaning unit removes noise and redundant information from the collected data, improving its accuracy and reliability. The data conversion unit converts data of different formats into a unified format to facilitate subsequent fusion and analysis. The data correlation unit integrates data from multiple sources to form a comprehensive dataset, providing comprehensive support for subsequent analysis. The intelligent inspection and analysis module then performs real-time analysis on the fused data. The feature extraction unit extracts characteristic information about vehicle behavior from the data. The behavior recognition unit identifies abnormal vehicle behavior based on these characteristics. The violation determination unit determines whether the abnormal behavior constitutes a violation and determines the type of violation based on pre-set rules. The inspection decision and execution module formulates inspection measures based on the analysis results and conducts real-time inspections and actions on violating vehicles through execution devices or systems. The system management and user interaction module provides device management, user permission management, data storage management, and a user interface to facilitate user operations such as system configuration, data query, and review of inspection results. The entire system achieves real-time monitoring and intelligent inspection of highway traffic through the collection, fusion and preprocessing of multimodal data, intelligent analysis, and inspection decision-making and execution, significantly improving inspection efficiency and accuracy and effectively ensuring the safety and smooth flow of highways. If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art or the portion of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present invention.The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code, such as here. Figure 1 Indicates.

[0098] It is important to note that the configuration and arrangement of the present application as shown in various exemplary embodiments are illustrative only. Although only a few embodiments are described in detail in this disclosure, those reading this disclosure should readily understand that many modifications are possible (e.g., variations in the size, scale, structure, shape, and proportions of various components, as well as parameter values ​​(e.g., temperature, pressure, etc.), mounting arrangements, use of materials, color, orientation, etc.) without materially departing from the novel teachings and advantages of the subject matter described in this application. For example, an element shown as integrally formed may be composed of multiple parts or elements, the position of an element may be inverted or otherwise altered, and the nature, number, or position of discrete elements may be modified or changed. Other substitutions, modifications, changes, and omissions may be made in the design, operating conditions, and arrangement of the exemplary embodiments without departing from the scope of the present invention. Therefore, the present invention is not limited to a particular embodiment, but extends to a variety of modifications that still fall within the scope of the appended claims.

[0099] Additionally, in order to provide a concise description of exemplary embodiments, all features of an actual embodiment may not be described (ie, those features that are not relevant to the best mode presently contemplated for carrying out the invention or those that are not relevant to implementing the invention).

[0100] It will be appreciated that in the development of any actual embodiment, as in any engineering or design project, numerous implementation-specific decisions may be made. Such a development effort may be complex and time-consuming, but will, for those of ordinary skill having the benefit of this disclosure, be a routine undertaking of design, fabrication, and production without undue experimentation.

[0101] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. The intelligent highway inspection and control system based on multimodal data fusion is characterized by: include, Multimodal data acquisition module; Data fusion and preprocessing module; Intelligent audit analysis module; Audit decision-making and execution module; System management and user interaction module; The intelligent audit analysis module includes: Feature extraction unit, extracting feature information of vehicle behavior; A behavior recognition unit identifies abnormal vehicle behavior based on the extracted feature information; A violation determination unit determines whether the vehicle's abnormal behavior constitutes a violation and determines the type of violation; The violation determination unit includes: A first determination subunit determines whether the abnormal behavior conforms to a preset rule of characteristic of illegal behavior based on characteristic information of the abnormal behavior; a second determination subunit, determining a violation degree of the abnormal behavior according to a determination result of the first determination subunit; a third determination subunit, generating a violation determination result according to the determination result of the second determination subunit; When the third determination subunit generates a violation determination result, If the second determination subunit determines that the abnormal behavior is a serious violation, it generates a violation determination result and transmits the result to the audit decision and execution module; If the second determination subunit determines that the abnormal behavior is a general violation or a minor violation, the process returns to the first determination subunit for re-evaluation; If the first determination subunit determines after re-evaluation that the abnormal behavior does not meet the stricter rule of violation characteristics, the third determination subunit generates a no-violation determination result and ends the process; If the first determination subunit determines after re-evaluation that the abnormal behavior meets stricter violation characteristic rules, the second determination subunit re-determines the violation degree of the abnormal behavior, and the third determination subunit generates a violation determination result based on the new violation degree and passes the result to the audit decision and execution module.

2. The intelligent highway inspection and control system based on multimodal data fusion according to claim 1, characterized in that: The audit decision and execution module includes: The audit decision-making unit is used to formulate audit measures based on the judgment results of the violation determination unit; The execution unit is used to convey the inspection decision to the relevant departments or systems to achieve real-time inspection and processing of illegal vehicles.

3. The intelligent highway inspection and control system based on multimodal data fusion according to claim 1 or 2, characterized in that: The system management and user interaction module includes: Device management unit, used to monitor and maintain the system's hardware devices; User authority management unit, used to provide different operation permissions and functions according to the user's role and authority; Data storage management unit, used for system data storage and backup; The user interaction interface is used to facilitate users to configure the system, query data and view audit results.

4. The intelligent highway inspection and control system based on multimodal data fusion according to claim 3 is characterized by: The data fusion and preprocessing module also includes a data verification unit for verifying the collected data to ensure the accuracy and integrity of the data.

5. The intelligent highway inspection and control system based on multimodal data fusion according to claim 4 is characterized in that: The intelligent audit analysis module also includes a data analysis unit for performing statistical analysis on the fused data and generating an audit report.

6. The intelligent highway inspection and control system based on multimodal data fusion according to claim 5 is characterized in that: The audit decision and execution module also includes an audit result feedback unit, which is used to feed back the audit results to relevant users or departments for subsequent processing and improvement.

7. The intelligent highway inspection and control system based on multimodal data fusion according to claim 6, characterized in that: The system management and user interaction module also includes a system monitoring unit for real-time monitoring of the system's operating status and timely detection and processing of system failures.

8. The intelligent highway inspection and control system based on multimodal data fusion according to claim 1 is characterized in that: The multimodal data acquisition module includes: Video surveillance equipment, used to collect real-time image and video information on highway conditions and vehicle behavior; Vehicle traffic data collection equipment, used to record vehicle traffic information, including license plate number, vehicle model, and traffic time; Meteorological sensors for monitoring meteorological conditions along the highway, including temperature, humidity, and visibility; The data fusion and preprocessing module includes a data cleaning unit for removing noise and redundant information from the collected data; A data conversion unit, used to convert data in different formats into a unified format; The data association unit is used to associate multi-source data to form a comprehensive data set.

Images