Multi-modal data-based drive test sensing system

Abstract

The invention provides a road test sensing system based on multi-modal data, which belongs to the technical field of road test sensing and comprises a multi-source sensing module, a dynamic fusion processing module, a cooperative control module and a data security module. The multi-source sensing module collects road environment data through vision, millimeter wave radar and laser radar. The dynamic fusion processing module is used for realizing time synchronization, space alignment and adaptive weighted fusion of multi-source data and generating a structured sensing result; the cooperative control module packages the result into a standard vehicle-road cooperative message and distributes the standard vehicle-road cooperative message to the vehicle end and the cloud end, and bidirectional control is supported; and the data security module carries out real-time fuzzy processing and encrypted transmission on the sensitive information. The system improves the sensing reliability, the data security and the cooperative control efficiency in a complex environment, and is suitable for an intelligent traffic scene.

Description

Technical Field

[0001] This invention relates to the field of roadside sensing technology, and in particular to a roadside sensing system based on multimodal data. Background Technology

[0002] Current roadside perception systems mostly rely on single sensors, resulting in a significant decrease in perception capability under adverse weather or low-light conditions. Furthermore, they lack multimodal data fusion and dynamic weight adjustment mechanisms. In addition, existing systems have shortcomings in data security, with sensitive information easily leaked and transmission lacking strong encryption. Inconsistent vehicle-to-infrastructure (V2I) message formats lead to poor system compatibility and interoperability. There is an urgent need for a roadside perception system that can adapt to complex environments and possesses intelligent fusion and security protection capabilities. Therefore, this paper proposes a roadside perception system based on multimodal data to address the aforementioned problems. Summary of the Invention

[0003] Therefore, the purpose of this invention is to provide a road test perception system based on multimodal data to at least solve the above problems.

[0004] The technical solution adopted in this invention is as follows: A road test perception system based on multimodal data includes: A multi-source sensing module, comprising multiple sensors deployed at key locations along the road, for collecting raw sensing data of the road environment from different physical dimensions; The dynamic fusion processing module is communicatively connected to the multi-source sensing module and is used to receive and process the raw sensing data. The processing includes time synchronization and spatial alignment of the multi-source data, adaptive allocation of fusion weights for different sensor data according to real-time environmental information, and generation of structured sensing results containing the target object's location, speed, type and road status through a multi-level fusion strategy. The collaborative control module is communicatively connected to the dynamic fusion processing module. It is used to encapsulate the structured perception results into standardized vehicle-road cooperative messages and distribute them to the vehicle and the cloud. At the same time, it receives and executes global optimization control commands issued by the cloud. The data security module interacts with the multi-source sensing module and the collaborative control module respectively, and is used to perform real-time blurring of sensitive information in the raw data during the data input stage and to encrypt the transmitted data during the data output stage.

[0005] Furthermore, the multi-source sensing module includes a visual sensor, a millimeter-wave radar, and a lidar; the visual sensor adopts a multi-lens configuration, including a wide-angle lens and a telephoto lens, and integrates automatic heating defogging and optical anti-fogging components; the millimeter-wave radar is a 4D imaging millimeter-wave radar, used to provide target height information and micro-Doppler features; the lidar is a solid-state area array lidar, used to acquire high-precision point cloud data.

[0006] Furthermore, the fusion weight allocation strategy of the dynamic fusion processing module specifically includes: Real-time monitoring of environmental parameters, including light intensity, visibility, and precipitation; when light intensity is below a set threshold or visibility decreases due to fog, haze, rain, or snow, the weight of visual sensor data is automatically reduced, and the weight of millimeter-wave radar and lidar data is increased accordingly. In clear weather and under sufficient light conditions, the weight of visual sensor data is increased to utilize its rich texture information to achieve high-precision target classification.

[0007] Furthermore, the multi-level fusion strategy of the dynamic fusion processing module specifically includes: Data-level fusion correlates and clusters point cloud data and detection boxes from LiDAR and millimeter-wave radar to generate a preliminary target list; Feature-level fusion extracts depth features from visual sensor images, concatenates them with the feature vectors of radar targets, and inputs them into a classification neural network for target type identification. Decision-level fusion assesses the confidence level of the fusion results at each level. When decision results from different sensor sources conflict, weighted voting is performed based on the assigned fusion weights to generate the final decision.

[0008] Furthermore, the system also includes an online self-calibration module. The online self-calibration module uses fixed markers deployed in the road scene as a reference to periodically and automatically calculate the changes in the extrinsic parameter matrix between different sensors. When the changes exceed the preset tolerance, the calibration process is triggered to automatically update the spatial alignment parameters in order to maintain the spatial consistency of multimodal sensing data.

[0009] Furthermore, the standardized vehicle-road cooperative messages encapsulated by the cooperative control module conform to the third-party V2X application layer standard message set, including Roadside Unit Traffic Event Message (RSI), Roadside Unit Traffic Participant Message (RSM), and Map Message (MAP); the module communicates directly with the on-board OBU via the PC5 interface.

[0010] Furthermore, the collaborative control module executes the global optimization control commands issued by the cloud in the following ways: receiving the signal timing optimization scheme generated by the cloud traffic control platform based on regional multi-node perception data, and directly controlling the local traffic signal controller according to the scheme to realize the dynamic adjustment of the traffic light cycle; at the same time, pushing the suggested vehicle speed and lane guidance information in the optimization scheme to the connected vehicles through V2I communication.

[0011] Furthermore, the real-time blurring processing of the data security module adopts differential privacy technology based on semantic recognition. The specific process includes: locating the face region and license plate region in the image and point cloud data through a visual recognition algorithm, and performing irreversible pixel perturbation and feature confusion processing on the region, so that the information entropy of the processed data is reduced by no less than 40% and cannot be restored.

[0012] Furthermore, the encrypted transmission of the data security module adopts an asymmetric encryption mechanism based on the national cryptographic SM9 algorithm to authenticate the integrity and source of the transmitted data, ensuring that the data transmission complies with the Level 3 requirements of the network security level protection system.

[0013] Compared with the prior art, the beneficial effects of the present invention are: 1. By using multimodal sensor collaboration and dynamic weight allocation, the sensing accuracy and robustness under adverse weather and complex lighting conditions are significantly improved; 2. Employ a multi-level fusion strategy to achieve end-to-end optimization from data to decision-making, thereby enhancing target identification and tracking capabilities; 3. Supports standardized V2X message encapsulation and direct communication, improving system compatibility and real-time performance; 4. It integrates a data security module to achieve real-time obfuscation and national cryptographic encryption transmission of sensitive information, meeting the requirements of network security level three; 5. It has online self-calibration and cloud-based collaborative control capabilities, supporting traffic signal optimization and vehicle guidance to improve road traffic efficiency and safety. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only preferred embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a schematic diagram of the overall structure of a road test perception system based on multimodal data proposed in an embodiment of the present invention. Detailed Implementation

[0016] The principles and features of the present invention are described below with reference to the accompanying drawings. The listed embodiments are only used to explain the present invention and are not intended to limit the scope of the present invention.

[0017] Reference Figure 1 This invention provides a road test perception system based on multimodal data, characterized in that it includes: A multi-source sensing module, comprising multiple sensors deployed at key locations along the road, for collecting raw sensing data of the road environment from different physical dimensions; The dynamic fusion processing module is communicatively connected to the multi-source sensing module and is used to receive and process the raw sensing data. The processing includes time synchronization and spatial alignment of the multi-source data, adaptive allocation of fusion weights for different sensor data according to real-time environmental information, and generation of structured sensing results containing the target object's location, speed, type and road status through a multi-level fusion strategy. The collaborative control module is communicatively connected to the dynamic fusion processing module. It is used to encapsulate the structured perception results into standardized vehicle-road cooperative messages and distribute them to the vehicle and the cloud. At the same time, it receives and executes global optimization control commands issued by the cloud. The data security module interacts with the multi-source sensing module and the collaborative control module respectively, and is used to perform real-time blurring of sensitive information in the raw data during the data input stage and to encrypt the transmitted data during the data output stage.

[0018] For example, the multi-source perception module deploys multiple sensors at key locations on the road. These sensors collect raw data from different physical dimensions, such as acquiring image information through visual devices and capturing motion trajectories through radar devices. All sensors are directly connected to the dynamic fusion processing module, which is responsible for performing two key processing steps on the raw data: The first step is technical calibration, which ensures that the data collected by different sensors are aligned on the time axis through time synchronization and eliminates coordinate deviations caused by differences in sensor installation positions through spatial alignment; the second step is intelligent weighted fusion, where the system dynamically adjusts the credibility weight of each sensor's data according to real-time environmental parameters (such as light intensity and weather conditions). For example, in hazy weather, the weight of visual sensors is reduced and the weight of radar sensors is increased. Finally, a structured perception result containing target location, speed, type, and road condition is generated through a multi-level fusion strategy.

[0019] The structured perception results are transmitted to the collaborative control module, which encapsulates them into a message format that conforms to the vehicle-road cooperative standard and sends them to both the vehicle terminal and the cloud platform through direct communication. The collaborative control module also has two-way interaction capabilities, which can send global optimization instructions from the cloud (such as traffic signal adjustment schemes) to local devices for execution, and can also upload feedback data from the vehicle terminal to the cloud to form a closed-loop control.

[0020] To ensure data security throughout its entire lifecycle, the system has a data security module: during the data acquisition phase, real-time fuzzing technology is used to irreversibly perturb sensitive information (such as faces and license plates) in the original data, ensuring that privacy information cannot be restored; during the data transmission phase, national cryptographic algorithms are used to encrypt the encapsulated messages, and an authentication mechanism is used to ensure the legitimacy of the data source.

[0021] This system achieves full-process coverage from environmental perception and data processing to secure transmission through modular design. Its dynamic weight allocation and multi-level fusion strategy significantly improve the perception reliability in complex road scenarios, while the vehicle-road-cloud collaborative architecture provides real-time and accurate data support for intelligent traffic management.

[0022] The multi-source sensing module includes a visual sensor, a millimeter-wave radar, and a lidar; the visual sensor adopts a multi-lens configuration, including a wide-angle lens and a telephoto lens, and integrates automatic heating defogging and optical anti-fogging components; the millimeter-wave radar is a 4D imaging millimeter-wave radar, used to provide target height information and micro-Doppler features; the lidar is a solid-state area array lidar, used to acquire high-precision point cloud data.

[0023] For example, the multi-source sensing module employs three heterogeneous sensors working collaboratively: a visual sensor (camera), millimeter-wave radar, and lidar. The visual sensor can utilize a wide-angle lens and a telephoto lens design. The wide-angle lens can be used for monitoring large-scale scenes, while the telephoto lens can be used for capturing details at long distances. Automatic heating defogging can eliminate condensation on the lens surface through an electric heating element, and the optical anti-fogging component can employ a special coating technology to reduce fog adhesion. Millimeter-wave radar employs 4D imaging millimeter-wave radar technology, adding target height information and micro-Doppler features to the traditional three-dimensional coordinate system (X / Y / Z). Target height information accurately senses the vertical spatial position of an object, while micro-Doppler features analyze subtle motion states through frequency changes. LiDAR can utilize a solid-state array architecture, differentiating itself from traditional mechanical rotating designs and offering technological advantages: no moving mechanical parts, thus improving equipment reliability. The multi-source sensing module achieves multi-modal data acquisition from visible to invisible light through the complementary physical dimensions of three sensors (optical imaging, radio wave detection, and laser point cloud), providing raw data support for subsequent dynamic fusion processing modules. The visual sensor acquires texture information, millimeter-wave radar excels in penetrating fog, rain, and snow, and lidar provides centimeter-level three-dimensional structural data. The collaborative work of these three sensors can meet the sensing needs in complex road environments.

[0024] The fusion weight allocation strategy of the dynamic fusion processing module specifically includes: Real-time monitoring of environmental parameters, including light intensity, visibility, and precipitation; when light intensity is below a set threshold or visibility decreases due to fog, haze, rain, or snow, the weight of visual sensor data is automatically reduced, and the weight of millimeter-wave radar and lidar data is increased accordingly. In clear weather and under sufficient light conditions, the weight of visual sensor data is increased to utilize its rich texture information to achieve high-precision target classification.

[0025] For example, when external environmental parameters change, the weight allocation mechanism will be dynamically adjusted. In low light or severe weather scenarios: if the light intensity is lower than the preset threshold (such as at night or in a tunnel), or if the visibility is significantly reduced due to fog, haze, rain or snow, the system will automatically reduce the weight of the visual sensor data. At this time, the weights of millimeter-wave radar (which provides target height and micro-Doppler features) and lidar (which provides high-precision point cloud data) will be increased accordingly to compensate for the performance degradation of the visual sensor in low light or occluded environments. Sunny, high-light scenarios: Under sunny and well-lit conditions during the day, the weight of the visual sensor (which acquires rich texture information through a multi-lens group) will be increased. At this time, the system will prioritize the use of visual data to achieve more accurate target classification (such as distinguishing between pedestrians, vehicles and non-motorized vehicles), while combining radar data to supplement spatial positioning information.

[0026] The fusion weight allocation strategy ensures the robustness of multi-source data fusion through real-time environmental perception and dynamic weight allocation, enabling the system to maintain high-precision perception capabilities under different weather and lighting conditions.

[0027] The multi-level fusion strategy of the dynamic fusion processing module specifically includes: Data-level fusion correlates and clusters point cloud data and detection boxes from LiDAR and millimeter-wave radar to generate a preliminary target list; Feature-level fusion extracts depth features from visual sensor images, concatenates them with the feature vectors of radar targets, and inputs them into a classification neural network for target type identification. Decision-level fusion assesses the confidence level of the fusion results at each level. When decision results from different sensor sources conflict, weighted voting is performed based on the assigned fusion weights to generate the final decision.

[0028] For example, data-level fusion performs correlation processing on the raw perception data of lidar and millimeter-wave radar. The high-precision point cloud data acquired by lidar and the target detection boxes generated by millimeter-wave radar are matched through spatial alignment technology. Different dimensions of data of the same physical target (such as point cloud position and radar reflection intensity) are clustered to form a preliminary target list containing basic target attributes (such as size and velocity). This stage focuses on the alignment and correlation of raw data to provide a unified data base for subsequent processing. Feature-level fusion, based on the data-level fusion results, extracts depth features (such as target texture, color, and shape) from visual sensor images and concatenates them with the spatial feature vectors of radar targets (such as point cloud distribution and micro-Doppler features). The concatenated composite features are then input into a pre-trained classification neural network, which uses a deep learning model to achieve accurate identification of target types (such as distinguishing between pedestrians, vehicles, and non-motorized vehicles). This stage leverages the complementary high semantic information of visual data and the high-precision spatial information of radar data to improve classification accuracy. The decision-level fusion performs a confidence assessment on the results of the first two fusion stages. When there are conflicts in the decision results of different sensor sources (e.g., visual recognition identifies a vehicle while radar identifies a pedestrian), a weighted vote is performed based on dynamically allocated fusion weights. The weight allocation strategy is determined by real-time environmental parameters (such as illumination and visibility). For example, under low illumination conditions, the visual weight is reduced and the radar weight is increased. Finally, a unified structured perception result is generated through a weighted average or voting mechanism to ensure the reliability and environmental adaptability of the output decision.

[0029] The multi-level fusion strategy achieves end-to-end optimization from raw data to final decision through layer-by-layer abstraction and information complementarity, effectively solving the perception limitations of a single sensor in complex road environments.

[0030] The system also includes an online self-calibration module. The online self-calibration module uses fixed markers deployed in the road scene as a reference to periodically and automatically calculate the changes in the extrinsic parameter matrix between different sensors. When the changes exceed the preset tolerance, the calibration process is triggered to automatically update the spatial alignment parameters in order to maintain the spatial consistency of multimodal sensing data.

[0031] For example, the online self-calibration module continuously ensures the spatial alignment accuracy of multimodal sensor data through an automated parameter adjustment mechanism. The online self-calibration module presets fixed landmarks in the road scene as reference points, such as specific signs or road markings. The system initiates the self-calibration process at fixed intervals. By comparing the spatial coordinate measurements of the same reference point by different sensors (such as vision sensors, LiDAR, and millimeter-wave radar), the deviation of the extrinsic parameter matrix of each sensor is calculated. When the change in the extrinsic parameter matrix exceeds the preset tolerance threshold, the system automatically triggers the calibration program, generates new spatial alignment parameters, and updates them to the dynamic fusion processing module. This ensures the continuous consistency of multi-source sensing data in the spatial dimension. The start conditions for the calibration process include changes in environmental factors (such as temperature fluctuations causing sensor installation displacement) or parameter drift caused by equipment aging. Through this closed-loop correction mechanism, the system can dynamically maintain millimeter-level spatial alignment accuracy without manual intervention, providing a reliable data foundation for multi-level fusion strategies.

[0032] The standardized vehicle-road cooperative messages encapsulated by the cooperative control module conform to the third-party V2X application layer standard message set, including Roadside Unit Traffic Event Message (RSI), Roadside Unit Traffic Participant Message (RSM), and Map Message (MAP); the module communicates directly with the on-board unit (OBU) via the PC5 interface.

[0033] For example, the collaborative control module encapsulates the structured perception results into vehicle-road cooperative messages conforming to the third-party V2X (Vehicle-to-Everything) application layer standard message set. This includes three core message types: Roadside Unit Traffic Event Messages (RSI), used to transmit information about sudden events on the road (such as traffic accidents, congestion, construction, etc.); Roadside Unit Traffic Participant Messages (RSM), used to describe the dynamic status of traffic participants (such as vehicle position, speed, and direction of travel); and Map Messages (MAP), used to provide static or semi-static road information such as road topology, traffic light timing, and speed limits. Adherence to third-party standards ensures the universality and interoperability of the message format, enabling this system to seamlessly interface with V2X devices or platforms from different manufacturers.

[0034] The collaborative control module also establishes direct communication with the on-board unit (OBU) via the PC5 interface. The PC5 interface is a device-to-device (D2D) communication technology that supports direct data transmission between the roadside unit and the on-board unit without the need for relaying through cellular network base stations. This direct connection method features low latency, high reliability, and low power consumption, which can meet the stringent real-time requirements of vehicle-road cooperation (such as collision warning and traffic light status synchronization). Through the PC5 interface, the roadside perception system can quickly push information such as traffic events and road conditions to connected vehicles, while receiving vehicle status or driving intentions from on-board equipment, forming a highly efficient two-way information interaction between vehicles and the road.

[0035] The collaborative control module enables efficient collaboration between the roadside perception system and vehicle-side equipment, providing crucial support for the real-time response and global optimization of the intelligent transportation system.

[0036] The collaborative control module executes the global optimization control commands issued by the cloud in the following ways: receiving the signal timing optimization scheme generated by the cloud traffic control platform based on regional multi-node perception data, and directly controlling the local traffic signal controller according to the scheme to realize the dynamic adjustment of the traffic light cycle; at the same time, pushing the suggested vehicle speed and lane guidance information in the optimization scheme to the connected vehicles through V2I communication.

[0037] For example, the specific way the collaborative control module executes the cloud-based global optimization control commands embodies the collaborative working mechanism between the road test system and the cloud platform. By receiving the optimization scheme generated by the cloud, it directly applies it to the local traffic control equipment and synchronizes key information to connected vehicles, thereby achieving dynamic optimization of regional traffic flow. For traffic signal control, the cloud-based traffic control platform generates a signal timing optimization scheme based on real-time perception data gathered from multiple road test perception nodes (such as this system and other adjacent road test units) within the region, using a global algorithm. After receiving the scheme, the collaborative control module can directly control the local traffic signal controller without manual intervention, dynamically adjusting the cycle length of the traffic lights (such as extending or shortening the green light time in a certain direction) to adapt to real-time traffic flow changes. For vehicle guidance, the suggested vehicle speed (such as speed limit adjustment) and lane guidance information (such as variable lane indication) included in the cloud-based optimization scheme are directly pushed to the on-board terminal of the connected vehicle through the collaborative control module via V2I (vehicle-to-infrastructure) communication, helping the driver or autonomous driving system to plan driving strategies in advance and reduce the risk of congestion or conflict.

[0038] The real-time blurring processing of the data security module adopts differential privacy technology based on semantic recognition. The specific process includes: locating the face region and license plate region in the image and point cloud data through visual recognition algorithm, and performing irreversible pixel perturbation and feature confusion processing on the region, so that the information entropy of the processed data is reduced by no less than 40% and cannot be restored.

[0039] For example, real-time blurring technology protects the security of sensitive personal information in road scenes through differential privacy. It automatically locates sensitive areas in images and point cloud data through visual recognition algorithms, focusing on facial features (such as facial contours and the position of facial features) and license plate markings (including character areas and number structures). During the location process, the system combines semantic information (such as "human body area" and "vehicle markings") for accurate identification, ensuring that the processing scope covers all potential privacy leakage points. After localization, the system employs irreversible pixel perturbation technology to process sensitive areas. Specific methods include: locally blurring the face region by randomly replacing pixels and obfuscating features to destroy the original biometric recognizability; and using a character structure obfuscation algorithm for the license plate region, randomly perturbing the character strokes while preserving the overall outline of the license plate, making the original number unrecoverable. The processed data must meet the technical requirement of a minimum 40% reduction in information entropy, ensuring that even reverse calculations cannot restore the original sensitive information.

[0040] By combining semantic recognition with differential privacy, the thoroughness of data anonymization is ensured, while avoiding the overall data quality degradation caused by traditional obfuscation methods. The processed data can still retain macroscopic structural information of the road scene (such as vehicle position and road alignment), meeting the needs of subsequent perception fusion and collaborative control, while achieving strict protection of personal privacy.

[0041] The data security module employs an asymmetric encryption mechanism based on the national cryptographic algorithm SM9 to authenticate the integrity and source of transmitted data, ensuring that data transmission complies with the Level 3 requirements of the network security level protection system.

[0042] For example, the data security module's encrypted transmission mechanism employs an asymmetric encryption system based on the national standard SM9 algorithm. The SM9 algorithm, based on identifier-based asymmetric encryption, does not rely on traditional digital certificate systems, making it particularly suitable for resource-constrained IoT device scenarios. Its encryption process uses the receiver's public key to encrypt the transmitted data, and only the legitimate receiver holding the corresponding private key can decrypt it, thus eliminating the possibility of unauthorized access. To meet the Level 3 requirements of network security protection, the system uses a hardware security module (HSM) to store the private key, preventing it from being exposed in plaintext to the general computing environment. Regarding encryption strength, the SM9 algorithm's key length reaches 256 bits or more, meeting the Level 3 requirement of "using a high-strength encryption algorithm." This encrypted transmission mechanism ensures data confidentiality while achieving dual verification of data integrity and source reliability, providing a high-security communication guarantee that complies with network security standards for the interaction between the road test perception system and the vehicle and cloud.

[0043] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A road test sensing system based on multimodal data, characterized in that, include: A multi-source sensing module, comprising multiple sensors deployed at key locations along the road, for collecting raw sensing data of the road environment from different physical dimensions; The dynamic fusion processing module is communicatively connected to the multi-source sensing module and is used to receive and process the raw sensing data. The processing includes time synchronization and spatial alignment of the multi-source data, adaptive allocation of fusion weights for different sensor data according to real-time environmental information, and generation of structured sensing results containing the target object's location, speed, type and road status through a multi-level fusion strategy. The collaborative control module is communicatively connected to the dynamic fusion processing module. It is used to encapsulate the structured perception results into standardized vehicle-road cooperative messages and distribute them to the vehicle and the cloud. At the same time, it receives and executes global optimization control commands issued by the cloud. The data security module interacts with the multi-source sensing module and the collaborative control module respectively, and is used to perform real-time blurring of sensitive information in the raw data during the data input stage and to encrypt the transmitted data during the data output stage.

2. The system according to claim 1, characterized in that, The multi-source sensing module includes a visual sensor, a millimeter-wave radar, and a lidar; the visual sensor adopts a multi-lens configuration, including a wide-angle lens and a telephoto lens, and integrates automatic heating defogging and optical anti-fogging components; the millimeter-wave radar is a 4D imaging millimeter-wave radar, used to provide target height information and micro-Doppler features; the lidar is a solid-state area array lidar, used to acquire high-precision point cloud data.

3. The system according to claim 2, characterized in that, The fusion weight allocation strategy of the dynamic fusion processing module specifically includes: Real-time monitoring of environmental parameters, including light intensity, visibility, and precipitation; when light intensity is below a set threshold or visibility decreases due to fog, haze, rain, or snow, the weight of visual sensor data is automatically reduced, and the weight of millimeter-wave radar and lidar data is increased accordingly. In clear weather and under sufficient light conditions, the weight of visual sensor data is increased to utilize its rich texture information to achieve high-precision target classification.

4. The system according to claim 1, characterized in that, The multi-level fusion strategy of the dynamic fusion processing module specifically includes: Data-level fusion correlates and clusters point cloud data and detection boxes from LiDAR and millimeter-wave radar to generate a preliminary target list; Feature-level fusion extracts depth features from visual sensor images, concatenates them with the feature vectors of radar targets, and inputs them into a classification neural network for target type identification. Decision-level fusion assesses the confidence level of the fusion results at each level. When decision results from different sensor sources conflict, weighted voting is performed based on the assigned fusion weights to generate the final decision.

5. The system according to claim 1, characterized in that, The system also includes an online self-calibration module. The online self-calibration module uses fixed markers deployed in the road scene as a reference to periodically and automatically calculate the changes in the extrinsic parameter matrix between different sensors. When the changes exceed the preset tolerance, the calibration process is triggered to automatically update the spatial alignment parameters in order to maintain the spatial consistency of multimodal sensing data.

6. The system according to claim 1, characterized in that, The standardized vehicle-road cooperative messages encapsulated by the cooperative control module conform to the third-party V2X application layer standard message set, including Roadside Unit Traffic Event Message (RSI), Roadside Unit Traffic Participant Message (RSM), and Map Message (MAP); the module communicates directly with the on-board unit (OBU) via the PC5 interface.

7. The system according to claim 1, characterized in that, The collaborative control module executes the global optimization control commands issued by the cloud in the following ways: receiving the signal timing optimization scheme generated by the cloud traffic control platform based on regional multi-node perception data, and directly controlling the local traffic signal controller according to the scheme to realize the dynamic adjustment of the traffic light cycle; at the same time, pushing the suggested vehicle speed and lane guidance information in the optimization scheme to the connected vehicles through V2I communication.

8. The system according to claim 1, characterized in that, The real-time blurring processing of the data security module adopts differential privacy technology based on semantic recognition. The specific process includes: locating the face region and license plate region in the image and point cloud data through visual recognition algorithm, and performing irreversible pixel perturbation and feature confusion processing on the region, so that the information entropy of the processed data is reduced by no less than 40% and cannot be restored.

9. The system according to claim 1, characterized in that, The data security module employs an asymmetric encryption mechanism based on the national cryptographic algorithm SM9 to authenticate the integrity and source of transmitted data, ensuring that data transmission complies with the Level 3 requirements of the network security level protection system.

Images