A multi-view radar and visual fusion perception data monitoring method and system

Abstract

The application discloses a kind of multi-purpose radar vision fusion perception data monitoring method and system, it is related to traffic road condition supervision technical field, comprising: S1, determine the information of to-be-measured intersection, the intersection information includes: intersection name, intersection import quantity, intersection latitude and longitude, each import and export vehicle direction and quantity, equipment information installed in intersection;According to intersection information, determine the radar original data corresponding to the vehicle information of current intersection, radar video original data, multi-purpose video data, electric police video original data;S2, the radar original data and radar video original data obtained are initially processed, obtain the first radar vision processing data after processing, determine target vehicle and the vehicle ID, time stamp of target vehicle, and extract the first feature of target vehicle about speed, distance, direction;It can be realized in complex scene to the accurate positioning of vehicle.

Description

Technical Field

[0001] This invention relates to the field of traffic condition monitoring technology, and in particular to a monitoring method and system for multi-view radar-visual fusion sensing data. Background Technology

[0002] A multi-view radar-visual integrated machine is a traffic sensor that combines multiple cameras, millimeter-wave radar, and a high-performance processor into one unit, enabling the fusion calculation of radar and video data. This machine can simultaneously output the positions of multiple targets and road events, typically by fusing the recognition results of radar and video data. In the application of holographic intersection digital twins, radar and video data often need to be fused multiple times to reconstruct the traffic operation status of the entire intersection. However, data loss is inevitable during the fusion process, and due to the numerous fusion stages, problem localization is challenging. This paper presents a monitoring method and system for multi-view radar-visual fusion data, aiming to accurately locate the problem and provide optimization directions for developers to optimize the fusion algorithm. Summary of the Invention

[0003] This application provides a monitoring method and system for multi-view radar-visual fusion perception data, which solves the problem in the prior art that vehicles are easily lost in complex areas, and achieves accurate vehicle tracking and positioning.

[0004] This application provides a method for monitoring multi-view radar-visual fusion sensing data, including:

[0005] S1. Determine the information of the intersection to be tested. The intersection information includes: intersection name, number of intersection entrances, intersection latitude and longitude, direction and number of vehicles at each entrance and exit, and equipment information installed at the intersection. Based on the intersection information, determine the original radar data, original radar video data, multi-view video data, and original electronic police video data corresponding to the vehicle information at the current intersection.

[0006] S2. The acquired raw radar data and raw radar video data are processed initially to obtain the first processed radar video data. The target vehicle and its vehicle ID and timestamp are determined, and the first features of the target vehicle regarding speed, distance and direction are extracted.

[0007] S3, when the position of the target vehicle changes, the target vehicle is tracked using multi-view video data and first radar vision processing data to obtain second radar vision processing data, and the vehicle type and speed of the target vehicle are identified when tracking the target vehicle, and the second feature corresponding to the vehicle behavior information is obtained.

[0008] S4. The first radar video processing data, the second radar video processing data, and the original electronic police video data are fused and processed to associate the target vehicle. Based on the first feature and the second feature, the fused target vehicle is determined.

[0009] Step S2 also includes the following implementation methods:

[0010] S21. Detect the scene using radar and determine the coordinates of the target vehicle at the current intersection relative to the radar coordinate system;

[0011] S22. Capture the scene using a camera to obtain an image of the target vehicle, and determine the coordinates of the target vehicle relative to the image coordinate system;

[0012] S23. The obtained coordinate values ​​of the radar coordinate system are fused with the coordinate values ​​of the image coordinate system to determine the position and angle of the target vehicle.

[0013] S24. Based on the changes in the target vehicle's position at different timestamps, determine the target vehicle's speed and distance relative to the intersection and surrounding vehicles, and predict the target vehicle's direction based on the current intersection information and the target vehicle's distance within the intersection.

[0014] Step S3 also includes the following implementation methods:

[0015] S31, obtain the sampling rate and field of view corresponding to the multi-view camera, and determine that the vehicle in the multi-view camera and the target vehicle in the original radar video data are the same;

[0016] S32, acquire the visual features of the target vehicle, the visual features including the outline, color, and texture of the target vehicle;

[0017] S33, the first radar vision processing data is divided into different regions, and the shape, size and speed features of the target vehicle in each region are extracted as the first radar vision features;

[0018] S34, based on the first radar-visual feature and visual feature, perform feature matching on the target vehicle, and identify the first radar-visual feature and visual feature that are highly similar under different times and ambient light conditions, and output them as the second feature.

[0019] The implementation methods for fusing the first radar video processing data, the second radar video processing data, and the original electronic police video data in step S4 include:

[0020] S41, retrieve the corresponding license plate information from the original data of the electronic police system;

[0021] S42, bind the license plate information with the identified target vehicle, and determine whether the movement trajectory of the target vehicle after binding is the same as the movement trajectory of the license plate information;

[0022] S43, add the feature corresponding to the license plate information to the first feature and the second feature, as the final feature of the target vehicle.

[0023] A monitoring system for multi-view radar-visual fusion sensing data includes: an information acquisition module for acquiring intersection information and the corresponding raw radar data, raw radar video data, multi-view video data, and raw electronic police video data;

[0024] The first processing module is used to perform initial processing on the original radar data and original radar video data to obtain the first feature corresponding to the target vehicle, and output the data after initial processing as the first radar video processing data.

[0025] The second processing module is used to track the target vehicle using multi-view video data and first radar-view processed data, and to determine the second feature corresponding to the target vehicle.

[0026] The final processing module is used to fuse the first radar video processing data, the second radar video processing data, and the original electronic police video data to determine the target vehicle for tracking after fusion.

[0027] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0028] By integrating raw radar data, raw radar video data, multi-view video data, and raw electronic police video data, this combination of multimodal data enables the monitoring system to maintain high performance under various weather and lighting conditions.

[0029] Through initial and secondary processing, the system can accurately track target vehicles and extract key features such as speed, distance, and direction, while also identifying vehicle type and behavior information.

[0030] The system uses multi-step verification and feature matching to ensure accurate identification of target vehicles under different times and ambient lighting conditions, thus improving the system's robustness.

[0031] By linking license plate information to the identified target vehicle, the accuracy of tracking is further enhanced, and the information can be used for subsequent data analysis and traffic management. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating steps S1-S4 of a multi-view radar vision fusion sensing data monitoring method according to the present invention.

[0033] Figure 2 This is a flowchart illustrating steps S21-S24 of a multi-view radar vision fusion sensing data monitoring method according to the present invention.

[0034] Figure 3 This is a flowchart illustrating steps S31-S34 of a multi-view radar vision fusion sensing data monitoring method according to the present invention.

[0035] Figure 4 This is a flowchart illustrating steps S41-S43 of a multi-view radar vision fusion sensing data monitoring method according to the present invention.

[0036] Figure 5 This is a system framework diagram of a multi-view radar-visual fusion sensing data monitoring system according to the present invention. Detailed Implementation

[0037] To facilitate understanding of the present invention, a more complete description of this application will be given below with reference to the accompanying drawings, which illustrate preferred embodiments of the invention. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to enable a more thorough and complete understanding of the disclosure of the present invention.

[0038] It should be noted that the terms "vertical," "horizontal," "up," "down," "left," "right," and similar expressions used in this article are for illustrative purposes only and do not represent the only possible implementation.

[0039] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains; the terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention; the term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0040] If a multi-view radar detector outputs the identified location result without further evaluation, and the recognition performance is poor, it may output illogical location results. This can lead to stuttering or jumps when the user restores the location. For example, in a scenario where the multi-view radar detector primarily identifies vehicles, changes in vehicle speed can cause it to lose track of the target during tracking, resulting in inaccurate identification. However, due to the numerous fusion steps involved, locating the problem is challenging.

[0041] To address the challenge of locating the problem, this invention processes the currently monitored data through five components: intersection information configuration, data detection location, monitoring task configuration, scene threshold configuration, data monitoring, and problem location.

[0042] 1. Obtain intersection information configuration; intersection information configuration includes: configuring intersection name, number of intersection entrances, intersection latitude and longitude, direction and number of vehicles at each entrance and exit, and information on equipment installed at the intersection.

[0043] 2. Determine the data detection location.

[0044] In the application of holographic intersections, the data flow of the multi-view radar all-in-one machine's sensing data goes through five stages: sensing device, edge computing unit, device data access, big data platform, and holographic intersection platform. This application mainly involves data monitoring in three stages: sensing device, edge computing unit, and device data access.

[0045] The intelligent sensing equipment processes radar data, video data, fuses near and far video data, and identifies vehicle information such as license plates. The edge computing unit fuses radar and video data from a single intersection entrance, as well as radar and video data from the entire intersection. The device data access stage handles data access and image format conversion. The system embeds data points before and after each data ID change location, acquiring the vehicle ID, timestamp, original vehicle ID, license plate number, and vehicle type information at each embedding location. The vehicle ID, timestamp, and original vehicle ID at the embedding location are mandatory.

[0046] 3. Monitoring task configuration.

[0047] The monitoring task configuration mainly involves configuring the data monitoring location and data extraction time. The data monitoring location configuration function is used to select the monitoring location for this data monitoring task, and the data extraction time configuration function configures the data extraction duration and data extraction start time.

[0048] 4. Scene threshold configuration.

[0049] In practical applications, the installation location of the multi-view radar-guided integrated camera often needs to be determined based on factors such as intersection channelization and pole conditions. Different installation locations will directly affect the data acquisition accuracy and fusion effect. Scene threshold configuration mainly completes the configuration of the installation scene for the multi-view radar-guided integrated camera.

[0050] 5. Data monitoring and problem localization.

[0051] Data monitoring primarily involves monitoring the data quality at each data acquisition point, supporting monitoring of the entire intersection and data from each approach direction. It also compiles statistics on the amount of data monitored at each data acquisition point and details of vehicle traffic data. Through comparative analysis of the data from each data acquisition point, problems can be identified.

[0052] Example 1

[0053] like Figure 1As shown, the monitoring method for multi-view radar-visual fusion sensing data of this application includes:

[0054] S1. Determine the information of the intersection to be tested. The intersection information includes: intersection name, number of intersection entrances, intersection latitude and longitude, direction and number of vehicles at each entrance and exit, and equipment information installed at the intersection. Based on the intersection information, determine the original radar data, original radar video data, multi-view video data, and original electronic police video data corresponding to the vehicle information at the current intersection.

[0055] S2. The acquired raw radar data and raw radar video data are processed initially to obtain the first processed radar video data. The target vehicle and its vehicle ID and timestamp are determined, and the first features of the target vehicle regarding speed, distance and direction are extracted.

[0056] S3, when the position of the target vehicle changes, the target vehicle is tracked using multi-view video data and first radar vision processing data to obtain second radar vision processing data, and the vehicle type and speed of the target vehicle are identified when tracking the target vehicle, and the second feature corresponding to the vehicle behavior information is obtained.

[0057] S4. The first radar video processing data, the second radar video processing data, and the original electronic police video data are fused and processed to associate the target vehicle. Based on the first feature and the second feature, the fused target vehicle is determined.

[0058] The aforementioned raw radar data is used to receive echo signals sent by nearby vehicles using radar, including information such as the time, amplitude, phase, and frequency of the echo signals; and to convert this received information into a visual radar image. Each point on the radar image is regarded as an identified target vehicle and assigned a location-related vehicle ID and timestamp.

[0059] Raw radar video data is obtained by recording equipment to identify the type and speed of the target vehicle in the radar image. Multi-view video data is obtained by using multiple cameras or multiple cameras to associate the target vehicle in multiple images and identify how the target vehicle adjusts its direction and speed in multiple images. This allows for the identification of the target vehicle's driving intention, which facilitates timely warnings at intersections and reduces traffic accidents.

[0060] Raw video data from electronic traffic enforcement systems refers to unprocessed video data captured by these systems (usually installed at traffic intersections). This video data is primarily used for traffic monitoring, violation detection, and recording to provide real-time images of road traffic and for subsequent evidence collection.

[0061] First, the system uses raw radar data and raw radar video data to identify the vehicles that appear. It combines the high-precision ranging and speed measurement capabilities of radar with the intuitive visual information of video data, thereby providing more comprehensive and accurate target detection and identification results.

[0062] Example 2

[0063] To improve the processing effect of raw radar data, during the initial processing of raw radar data and raw radar video data, such as... Figure 2 As shown, step S2 further includes:

[0064] S21. Detect the scene using radar and determine the coordinates of the target vehicle at the current intersection relative to the radar coordinate system;

[0065] S22. Capture the scene using a camera to obtain an image of the target vehicle, and determine the coordinates of the target vehicle relative to the image coordinate system;

[0066] S23. The obtained coordinate values ​​of the radar coordinate system are fused with the coordinate values ​​of the image coordinate system to determine the position and angle of the target vehicle.

[0067] When fusing radar coordinates and image coordinates, it also includes determining the timestamps and intersection information corresponding to the radar and camera, and determining that the data containing the target vehicle are in the same space and time.

[0068] The images of the target vehicles are acquired, and the center points of each target vehicle and the distance between the center points and the vehicle edges are obtained based on the image positions. The angles of the vehicle edges in the image coordinate system are used to obtain the direction of the target vehicle. Based on the current intersection information and the relative position of the target vehicle in the intersection, the predicted direction of the target vehicle is determined.

[0069] S24. Based on the changes in the target vehicle's position at different timestamps, determine the target vehicle's speed and distance relative to the intersection and surrounding vehicles, and predict the target vehicle's direction based on the current intersection information and the target vehicle's distance within the intersection.

[0070] By fusing radar and radar video data, the position and angle of the target vehicle at the intersection can be accurately determined, overcoming the errors and limitations that may exist with a single sensor, and the processed data at this time is the first radar video processing data for initial processing.

[0071] By analyzing the changes in the target vehicle's position at different timestamps, the vehicle's real-time speed can be calculated. Simultaneously, by measuring the distances between the target vehicle and intersections and surrounding vehicles, the relative positions of the vehicles and potential risks can be assessed.

[0072] Preferably, when the change in the position of the target vehicle at different timestamps is obtained, the movement trajectory of the target vehicle is generated, the point with the largest slope on the movement trajectory is selected, and the direction corresponding to the point with the largest slope is used as the predicted direction of the target vehicle.

[0073] By combining current intersection information with the distance of the target vehicle within the intersection, the future direction of the target vehicle can be predicted. This is crucial for traffic management and autonomous driving systems, helping to plan routes in advance and avoid potential collisions.

[0074] Example 3

[0075] If the current image acquisition uses multi-view cameras, the data acquired by the multi-view cameras is combined with radar video data to determine how the position and relative situation of the target vehicle being tracked changes under normal image capture conditions.

[0076] Specifically, such as Figure 3 As shown, step S3 also includes the following implementation methods:

[0077] S31, acquire the sampling rate and field of view corresponding to the multi-view camera, and determine that the vehicle in the multi-view camera is the same as the target vehicle in the original radar video data.

[0078] S32, acquire the visual features of the target vehicle, including the outline, color, and texture of the target vehicle.

[0079] In this step, contour extraction uses edge detection algorithms (such as Canny edge detection) to identify the vehicle's contours, color features are extracted through color space transformation (such as RGB to HSV) and color histogram statistics, and texture features are extracted using methods such as gray-level co-occurrence matrix and local binary mode (LBP).

[0080] S33, the first radar vision processing data is divided into different regions, and the shape, size and speed features of the target vehicle in each region are extracted as the first radar vision features.

[0081] In this step, the radar point cloud is segmented into different targets using a clustering algorithm (such as DBSCAN), and features such as the shape, size, and speed of the targets are extracted. After obtaining the first radar-visual features, the change values ​​of the features of the target vehicle when it is moving, as well as the movement trajectory of the target vehicle, are determined to facilitate the alignment of the target vehicle with the position of the radar and video.

[0082] S34, based on the first radar-visual feature and visual feature, perform feature matching on the target vehicle, and identify the first radar-visual feature and visual feature that remain highly similar under different times and ambient light conditions, and output them as the second feature to verify the accuracy of the matched target vehicle.

[0083] Since this uses multi-view video data collected by multiple cameras, and each camera has a different coordinate system, the collected data is prone to showing the same or similar targets. Each different perspective contains some detailed information. Therefore, when comparing the data, it is necessary to determine the current target and its details in order to control the collected data.

[0084] Step S34 for target vehicle matching also includes: comparing the shape and size of the target vehicle in the first radar visual features with the contour features in the visual features to obtain the vehicle to be identified;

[0085] Based on the speed feature in the first radar visual feature and the movement trajectory of the vehicle to be identified in the visual features, it is determined whether the vehicle to be identified is the target vehicle.

[0086] Preferably, in the vehicle tracking scenario, since the vehicle's speed and direction of movement will change, the contour texture extracted at different times will also change. In order to identify the possible changes in features when the vehicle is moving, determining whether the vehicle to be identified is the target vehicle also includes determining the similarity between the time series corresponding to the first radar visual feature and the visual feature.

[0087] The methods for achieving similarity between the first visual feature and the corresponding time series of visual features include:

[0088] Extract a first feature sequence that changes over time from visual features, and extract a second feature sequence that changes over time from the first radar features;

[0089] Calculate the distance between all point pairs between the first feature sequence and the second feature sequence, and generate a distance matrix;

[0090] Find a path through the distance matrix that minimizes the sum of distances between all pairs of points on the path, and output it as the best alignment path. Based on the output best alignment path, obtain the similarity between the time series corresponding to the first visual feature and the visual feature.

[0091] Preferably, in one specific embodiment of this application, when determining whether the vehicle to be identified is the target vehicle, the method further includes determining scene thresholds under different light sources, such that the feature values ​​of the colors in the extracted first feature sequence and second feature sequence are both greater than the scene thresholds.

[0092] The method for obtaining the extracted scene threshold is as follows:

[0093] Set an initial threshold T_init.

[0094] The monitored light intensity is denoted as L. A mapping function f(L) is used to adjust the threshold, and the formula is expressed as:

[0095] T_adj = T_init + k*f(L), where k is an adjustment coefficient used to control the degree of influence of illumination on the threshold, and T_adj is the scene threshold of the current environment.

[0096] The scene threshold is set based on color, shape, size, or other effective features to track targets. However, as the ambient light source changes unintentionally, such as from morning to night, the originally set threshold may no longer be applicable. A threshold taken in the morning may not be able to accurately track the target at night because the target's color characteristics will change under different lighting conditions. The scene threshold is set to reduce errors caused by the influence of light when tracking target vehicles.

[0097] When the similarity value between the time series corresponding to the first radar visual feature and the visual feature is greater than the scene threshold, the current vehicle to be detected is the target vehicle, and the data corresponding to the output first radar visual feature and the visual feature is used as the second radar visual processing data.

[0098] Example 4

[0099] In this embodiment, the original video data captured by the electronic police system is obtained, and the license plate information of the current target vehicle is identified from the data of the electronic police system. The obtained license plate information is then bound to the target vehicle identified in the first and second radar vision processing data, thereby obtaining more comprehensive information about the target vehicle when it is moving.

[0100] Specifically, such as Figure 4 As shown, the implementation method for fusing the first radar video processing data, the second radar video processing data, and the original electronic police video data in step S4 includes:

[0101] S41, retrieve the corresponding license plate information from the original data of the electronic police system;

[0102] S42, bind the license plate information with the identified target vehicle, and determine whether the movement trajectory of the target vehicle after binding is the same as the movement trajectory of the license plate information;

[0103] S43, add the feature corresponding to the license plate information to the first feature and the second feature, as the final feature of the target vehicle.

[0104] This step involves adding features related to license plate information to the already identified and verified target vehicle features. This enriches the feature set of the target vehicle, improving the accuracy and stability of vehicle identification. Simultaneously, these integrated features can also be used for more complex scenario analyses, such as traffic congestion prediction and abnormal behavior detection.

[0105] The target vehicle is tracked based on the final features obtained to determine its direction of movement and trajectory at different timestamps.

[0106] Preferably, to prevent the loss of target vehicles during the fusion process, step S4 further includes the following when processing the target vehicles:

[0107] The vehicle ID of the target vehicle is determined. When the vehicle ID disappears, the currently acquired image is filled in once, and data points are embedded. If the vehicle ID is still not visible after the first filling, a second filling is performed. If the vehicle ID disappears after the second filling, it is deleted, and the image is acquired again.

[0108] Once the vehicle ID is collected in the above steps, the movement status of the corresponding target vehicle is determined based on the vehicle ID, and the movement trajectory of the target vehicle is displayed.

[0109] In this step, the first filling involves filling one frame of the image and identifying the filled image to determine whether the current vehicle exists in the identified image. The second filling is done in the same way as the first filling, with the purpose of determining whether the target vehicle has been tracked in the adjacent frames of the image, so as to accurately locate the identified vehicle information and facilitate subsequent tracking based on the corresponding timestamp.

[0110] A monitoring system for multi-view radar vision fusion sensing data, such as Figure 5 As shown, it includes:

[0111] The information acquisition module is used to acquire intersection information and the corresponding raw radar data, raw radar video data, multi-view video data, and raw electronic police video data.

[0112] The first processing module is used to perform initial processing on the original radar data and original radar video data to obtain the first feature corresponding to the target vehicle, and output the data after initial processing as the first radar video processing data.

[0113] The second processing module is used to track the target vehicle using multi-view video data and first radar-view processed data, and to determine the second feature corresponding to the target vehicle.

[0114] The final processing module is used to fuse the first radar video processing data, the second radar video processing data, and the original electronic police video data to determine the target vehicle for tracking after fusion.

[0115] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A monitoring method for multi-view radar vision fusion sensing data, characterized in that, include: S1. Determine the information of the intersection to be tested. The intersection information includes: intersection name, number of intersection entrances, intersection latitude and longitude, direction and number of vehicles at each entrance and exit, and equipment information installed at the intersection. Based on the intersection information, determine the original radar data, original radar video data, multi-view video data, and original electronic police video data corresponding to the vehicle information at the current intersection. S2. The acquired raw radar data and raw radar video data are processed initially to obtain the first processed radar video data. The target vehicle and its vehicle ID and timestamp are determined, and the first features of the target vehicle regarding speed, distance and direction are extracted. S3, when the position of the target vehicle changes, the target vehicle is tracked using multi-view video data and first radar-processed data to obtain second radar-processed data, and the vehicle type and speed of the target vehicle are identified during tracking, obtaining a second feature corresponding to the vehicle behavior information; step S3 further includes: S31, obtain the sampling rate and field of view corresponding to the multi-view camera, and determine that the vehicle in the multi-view camera and the target vehicle in the original radar video data are the same; S32, acquire the visual features of the target vehicle, the visual features including the outline, color, and texture of the target vehicle; S33, the first radar vision processing data is divided into different regions, and the shape, size and speed features of the target vehicle in each region are extracted as the first radar vision features; S34, based on the first radar visual feature and the visual feature, perform feature matching on the target vehicle, and identify the first radar visual feature and the visual feature that are highly similar under different times and ambient light conditions, and output them as the second feature; S4. The first radar video processing data, the second radar video processing data, and the original electronic police video data are fused and processed to associate the target vehicle. Based on the first feature and the second feature, the fused target vehicle is determined.

2. The monitoring method for multi-view radar vision fusion sensing data as described in claim 1, characterized in that, Step S2 further includes the following implementation methods: S21. Detect the scene using radar and determine the coordinates of the target vehicle at the current intersection relative to the radar coordinate system; S22. Capture the scene using a camera to obtain an image of the target vehicle, and determine the coordinates of the target vehicle relative to the image coordinate system; S23. The obtained coordinate values ​​of the radar coordinate system are fused with the coordinate values ​​of the image coordinate system to determine the position and angle of the target vehicle. S24. Based on the changes in the target vehicle's position at different timestamps, determine the target vehicle's speed and distance relative to the intersection and surrounding vehicles, and predict the target vehicle's direction based on the current intersection information and the target vehicle's distance within the intersection.

3. The monitoring method for multi-view radar vision fusion sensing data as described in claim 1, characterized in that, The target vehicle matching in step S34 further includes: comparing the shape and size of the target vehicle in the first radar visual features with the contour features in the visual features to obtain the vehicle to be identified; Based on the speed feature in the first radar visual feature and the movement trajectory of the vehicle to be identified in the visual features, it is determined whether the vehicle to be identified is the target vehicle.

4. The monitoring method for multi-view radar vision fusion sensing data as described in claim 3, characterized in that, The method of determining whether the vehicle to be identified is the target vehicle also includes determining the similarity between the time series corresponding to the first radar visual feature and the visual feature; The methods for achieving similarity between the first visual feature and the corresponding time series of visual features include: Extract a first feature sequence that changes over time from visual features, and extract a second feature sequence that changes over time from the first radar features; Calculate the distance between all point pairs between the first feature sequence and the second feature sequence, and generate a distance matrix; Find a path through the distance matrix that minimizes the sum of distances between all pairs of points on the path, and output it as the best alignment path. Based on the output best alignment path, obtain the similarity between the time series corresponding to the first visual feature and the visual feature.

5. The monitoring method for multi-view radar vision fusion sensing data as described in claim 3, characterized in that, When determining whether a vehicle to be identified is the target vehicle, the process also includes determining scene thresholds under different light sources, such that the feature values ​​of the colors in the extracted first feature sequence and second feature sequence are both greater than the scene threshold.

6. The monitoring method for multi-view radar vision fusion sensing data as described in claim 1, characterized in that, The implementation method for fusing the first radar video processing data, the second radar video processing data, and the original electronic police video data in step S4 includes: S41, retrieve the corresponding license plate information from the original data of the electronic police system; S42, bind the license plate information with the identified target vehicle, and determine whether the movement trajectory of the target vehicle after binding is the same as the movement trajectory of the license plate information; S43, add the feature corresponding to the license plate information to the first feature and the second feature, as the final feature of the target vehicle.

7. The monitoring method for multi-view radar vision fusion sensing data as described in claim 1, characterized in that, Step S4, when processing the target vehicle, also includes: The vehicle ID of the target vehicle is determined. When the vehicle ID disappears, the currently acquired image is filled in once, and data points are embedded. If the vehicle ID is still not visible after the first filling, a second filling is performed. If the vehicle ID disappears after the second filling, it is deleted, and the image is acquired again. Once the vehicle ID is collected in the above steps, the movement status of the corresponding target vehicle is determined based on the vehicle ID, and the movement trajectory of the target vehicle is displayed.

8. The monitoring method for multi-view radar vision fusion sensing data as described in claim 5, characterized in that, The method for obtaining the scene threshold is as follows: Set an initial threshold T_init; The monitored light intensity is denoted as L. A mapping function f(L) is used to adjust the threshold, and the formula is expressed as: T_adj = T_init + k * f(L), where k is an adjustment coefficient used to control the degree of influence of illumination on the threshold, and T_adj is the scene threshold of the current environment.

9. A monitoring system for multi-view radar-visual fusion sensing data, using the monitoring method for multi-view radar-visual fusion sensing data as described in claim 1, characterized in that, include: The information acquisition module is used to acquire intersection information and the corresponding raw radar data, raw radar video data, multi-view video data, and raw electronic police video data. The first processing module is used to perform initial processing on the original radar data and original radar video data to obtain the first feature corresponding to the target vehicle, and output the data after initial processing as the first radar video processing data. The second processing module is used to track the target vehicle using multi-view video data and first radar-view processed data, and determine the second feature corresponding to the target vehicle. The final processing module is used to fuse the first radar video processing data, the second radar video processing data, and the original electronic police video data to determine the target vehicle for tracking after fusion.

Images