Road area perception data trusted fusion method for vehicle-road cooperative automatic driving

Abstract

The application discloses a kind of road area perception data credible fusion methods for vehicle-road cooperation automatic driving, comprising: obtaining road side radar perception trajectory dataset and road side video perception trajectory dataset;The road side video camera is initially calibrated, and the video perception trajectory coordinates are initially converted from pixel coordinate system to world coordinate system;Matching between radar and video perception trajectory data is realized, and coordinate conversion parameters are solved;Solve coordinate correction parameters and correct the world coordinates of video perception trajectory data, fuse the corrected video perception trajectory data with radar perception trajectory data;Periodically check radar video fusion accuracy, when radar video trajectory error exceeds threshold, recalculate coordinate correction parameters and correct video perception trajectory data.The application can realize the space-time synchronization of radar video coordinates automatically through road side radar and road side video perception trajectory data, and realize high-precision tracking of perception traffic target through radar video perception data fusion.

Description

Technical Field

[0001] This invention relates to the field of traffic perception data fusion, and specifically to a reliable fusion method for road domain perception data for vehicle-road cooperative autonomous driving. Background Technology

[0002] With the continuous development of 5G and autonomous driving technologies, roadside perception units have become a crucial support for building intelligent connected vehicle-road cooperative systems. Roadside intelligent sensors (such as high-definition cameras, millimeter-wave radar, and lidar) can promptly acquire the location, speed, and other motion trajectory information of various road traffic participants, enabling intelligent collaboration and cooperation between vehicles and between vehicles and road infrastructure. This achieves the goals of optimizing system resource utilization, improving road traffic safety, and alleviating traffic congestion. Specifically, video cameras can extract image features of multiple target vehicles, and camera calibration technology and target detection algorithms can also be used to obtain the location data of multiple target vehicles. Millimeter-wave radar can detect the location data of multiple target vehicles, and data fusion algorithms can obtain higher-precision location data. Integrating radar and video perception data to achieve higher-precision target detection has become an important direction in the current construction of roadside perception systems.

[0003] The prerequisite for radar-visual fusion is ensuring that the radar and visual sensing data are located in the same coordinate system and that the system time is synchronized. This requires spatiotemporal calibration of the roadside millimeter-wave radar and video cameras. Traditional spatiotemporal calibration methods typically require customized calibration targets and manual calibration experiments to collect data and calibrate spatiotemporal parameters. Such methods require creating relatively ideal experimental conditions (e.g., closed roads) to ensure that the customized targets are reliably identified by the sensors, and also require significant manual fieldwork for data collection. Furthermore, these methods are generally only suitable for commissioning after the radar and video equipment are installed synchronously. For later-installed equipment, the data collection for calibration is difficult due to road traffic constraints. Additionally, after completing spatiotemporal calibration using these methods, slight shifts in sensor installation positions due to environmental factors can cause spatiotemporal calibration parameters to become invalid, thus affecting the accuracy of radar-visual fusion.

[0004] How to automatically calibrate the spatiotemporal parameters of radar vision using inherent scene information and device perception data, and automatically determine the accuracy of radar vision fusion based on historical perception data and make timely corrections to the spatiotemporal calibration parameters so that the relevant spatiotemporal calibration and data fusion methods can be widely applied to newly built or later-added radar vision perception facilities is an important issue in the construction of radar vision perception systems.

[0005] Therefore, to solve the above problems, a reliable fusion method for road domain perception data for vehicle-road cooperative autonomous driving is needed. This method can automatically achieve spatiotemporal synchronization of radar-view coordinates through roadside radar and roadside video perception trajectory data, and achieve high-precision tracking of perceived traffic targets through radar-view perception data fusion. Summary of the Invention

[0006] In view of this, the purpose of this invention is to overcome the defects in the prior art and provide a reliable fusion method for road domain perception data for vehicle-road cooperative autonomous driving. This method can automatically achieve spatiotemporal synchronization of radar-view coordinates through roadside radar and roadside video perception trajectory data, and achieve high-precision tracking of perceived traffic targets through radar-view perception data fusion.

[0007] The reliable fusion method for road domain perception data for vehicle-road cooperative autonomous driving of the present invention includes the following steps:

[0008] S1. Use roadside radar and roadside video cameras to collect data synchronously, and obtain a structured roadside radar perception trajectory dataset and a roadside video perception trajectory dataset.

[0009] S2. Use the lane line corner point coordinates to perform preliminary calibration of the roadside video camera, and initially transform the video perception trajectory coordinates from the pixel coordinate system to the world coordinate system;

[0010] S3. Use vehicle trajectory similarity to match radar and video sensing trajectory data, and solve for the initial values ​​of coordinate transformation parameters based on the trajectory matching relationship;

[0011] S4. Solve for coordinate correction parameters and correct the world coordinates of the video sensing trajectory data, then fuse the corrected video sensing trajectory data with the radar sensing trajectory data;

[0012] S5. Periodically verify the accuracy of the video-sensing trajectory using the stored video-sensing trajectory data. When the video-sensing trajectory error exceeds the threshold, recalculate the coordinate correction parameters using the process in step S4 and correct the video-sensing trajectory data.

[0013] Furthermore, step S1 specifically includes:

[0014] S11. Use roadside radar to collect perception data, identify vehicles with the same ID, and sort their position coordinates by time to form trajectories, thus creating a roadside radar perception trajectory dataset. Wherein, the trajectory of vehicle j is denoted as Let be the positioning coordinates of vehicle j at time a, located in the world coordinate system;

[0015] S12. Collect perception data using roadside video cameras, perform target detection and tracking based on the video data to determine the IDs of each vehicle, and use the coordinates of the midpoint of the bottom edge of each target detection box as the vehicle position coordinates, sort them by time to form a trajectory, thus constructing a roadside video perception trajectory dataset. Wherein, the trajectory of vehicle j is denoted as Let be the positioning coordinates of vehicle j at time b, which are located in the pixel coordinate system.

[0016] Furthermore, step S2 specifically includes:

[0017] S21. Select lane corner points in the roadside perception image to construct a vertical grid, select no less than four lane corner points as calibration points, take the point closest to the radar as the calibration origin, and estimate the approximate world coordinates of each calibration point based on the prior information of the relative position of the lane lines in space.

[0018] S22. Based on the estimated world coordinates and pixel coordinates of the selected calibration points, the optimal homography matrix is ​​solved using the least squares method. The formula for calculating the homography matrix H is as follows:

[0019]

[0020] Where H is the homography matrix, Z is the scaling factor, u and v are pixel coordinates, and X and Y are the corresponding world coordinates;

[0021] S23. Using the homography matrix obtained from the solution, a preliminary transformation from the pixel coordinate system to the world coordinate system is achieved, resulting in a preliminary transformed roadside video perception trajectory dataset. Wherein, the trajectory of vehicle j is denoted as Let b be the position coordinates of vehicle j when it is initially converted to the world coordinate system.

[0022] Furthermore, step S3 specifically includes:

[0023] S31. Using Gaussian process regression, the roadside radar sensing trajectory dataset R... r and the initially converted roadside video perception trajectory dataset Interpolate the various sensing trajectories in the data to unify the sampling frequency of each trajectory;

[0024] S32. Dataset R of roadside radar sensing trajectory r The trajectories in Calculate its relationship with the initially converted roadside video sensing trajectory dataset. Each trajectory DTW distance between

[0025] S33. Construct the cost matrix C based on the calculated DTW distance. DTW And according to C DTW Solving for the correspondence between radar-sensing trajectories and video-sensing trajectories, the cost matrix C... DTW The format is as follows:

[0026]

[0027] S34. Based on the correspondence and matching relationship between the radar sensing trajectory and the video sensing trajectory determined in S33, calculate the average distance between the corresponding matching trajectory points. and the corresponding initial values ​​of coordinate transformation parameters

[0028] S35. Based on the coordinate transformation parameters obtained from the solution. Reassess the roadside video perception trajectory dataset Perform coordinate transformation on each trajectory, and repeat steps S32-S34 until the average distance between two consecutive calculations is obtained. The difference is less than γ d .

[0029] Furthermore, the DTW distance

[0030]

[0031] Where d(·) is the distance between the two coordinate points; These are the coordinates of the first point in the corresponding trajectory. These are the trajectory segments remaining after removing the first point from the corresponding trajectory.

[0032] Furthermore, the average distance shown for:

[0033]

[0034]

[0035] Where mean(·) represents the average value of the trajectory coordinates. Roadside video perception trajectory dataset for initial conversion Trajectory in The coordinates at time i, For the roadside radar sensing trajectory dataset R r The trajectory that corresponds to the video trajectory In i+t d The coordinates of time; θ * The initial value of the rotation angle is Δx. * Indicates the initial value of the translation in the x-direction, Δy* This represents the initial value of the translation in the y-direction. Let W represent the initial time delay; W represents the matrix to be decomposed. Singular value decomposition of W yields the left orthogonal matrix U, the diagonal matrix Λ, and the right orthogonal matrix V.

[0036] Furthermore, step S4 specifically includes:

[0037] S41. Construct 5 parameters (ΔX, ΔY, θ, K) x ,K y Coordinate correction model for the initially transformed roadside video perception trajectory dataset Each trajectory The world coordinates are corrected accordingly; the corresponding coordinate correction formula is as follows:

[0038]

[0039] Among them, K x K y θ is the scaling factor; θ is the plane angle; (x w_j (i),y w_j (i)) is Mid-track The world coordinates obtained from the initial transformation; (x cw_j (i),y cw_j (i) is for the trajectory The corrected world coordinates; ΔX represents translation in the x-direction, and ΔY represents translation in the y-direction;

[0040] S42. Construct an objective function based on the average distance between each point on the video sensing trajectory after coordinate correction and the corresponding matching point on the radar sensing trajectory, using the (Δx) determined in step S35. * ,Δy * ,θ * The coordinate transformation parameters are used as initial values ​​for the optimization parameters. The optimized coordinate correction parameters (ΔX, ΔY, θ, K) are then solved. x ,K y );

[0041] S43. Based on the optimized coordinate correction parameters (ΔX, ΔY, θ, K) x ,K y ), for the initially converted roadside video perception trajectory dataset The world coordinates of each trajectory are corrected to obtain the corrected roadside video perception trajectory dataset.

[0042] S44. The corrected video-aware trajectory dataset Each trajectory in the dataset is compared with the radar sensing trajectory dataset. The corresponding matched trajectories are fused to obtain the Rave-Vision fused trajectory dataset. Among them, the trajectory of vehicle j Let the radar-visual fusion positioning coordinates of vehicle j at time p be the coordinates of the vehicle j. The corresponding coordinate fusion formula is as follows:

[0043]

[0044] in, These are the position coordinates of the corrected video sensing trajectory. λ represents the position coordinates of the corresponding matched radar sensing trajectory, and λ is the coordinate fusion weight.

[0045] Furthermore, the objective function is:

[0046]

[0047] Among them, (x cw_j (i),y cw_j (i)) is for World coordinates after correction of each trajectory; (x r_j (i),y r_j (i) represents the position coordinates of the corresponding matching trajectory in the roadside radar perception trajectory dataset Rr, n represents the number of frames within the common detection time, and k represents the total number of vehicles.

[0048] Furthermore, step S5 specifically includes:

[0049] S51. Store the corrected video-aware trajectory dataset With radar sensing trajectory dataset Calculate the average distance D between corresponding matching trajectories in the two trajectory datasets every c days. c (R c,w ,R r ), determine whether it exceeds the threshold γ d ;

[0050] S52. When the average distance D c (R c,w ,R r The threshold γ was not exceeded. d At this time, existing calibration results can continue to be used for radar-visual data fusion; when the average distance D c (R c,w ,R r Exceeding the threshold γ d Then, the coordinates of the video perception trajectory are corrected again according to the process in step S4.

[0051] Furthermore, the average distance D c (R c,w ,Rr The calculation formula is as follows:

[0052]

[0053] Among them, (x cw_j (i),y cw_j (i) represents the corrected video-aware trajectory dataset R. c,w Mid-track The coordinates; (x r_j (i),y r_j (i) represents the radar sensing trajectory dataset R. r Corresponding matching trajectory The coordinates are defined by mean(·), where mean(·) represents the average value, n represents the number of frames within the common detection time, and k represents the total number of vehicles.

[0054] The beneficial effects of this invention are as follows: The reliable fusion method for road domain perception data for vehicle-road cooperative autonomous driving disclosed in this invention utilizes the motion similarity between lane line information in the scene and the corresponding matching trajectories in the radar-visual perception trajectory dataset to automatically achieve corresponding coordinate matching. The solution of the corresponding coordinate transformation parameters does not require additional calibration equipment and field work, which greatly simplifies the workload of radar-visual coordinate spatiotemporal registration. It can be applied to the perception data fusion of newly built or renovated radar and video equipment. Through the framework of automatic monitoring and correction of radar-visual fusion data, the quality of radar-visual fusion can be judged periodically and the coordinate correction parameters can be recalculated based on historical perception data, ensuring that radar-visual fusion always maintains a good perception effect. Attached Figure Description

[0055] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0056] Figure 1 This is a schematic diagram of the fusion method of the present invention;

[0057] Figure 2 A schematic diagram showing the installation locations of roadside millimeter-wave radar and roadside video cameras;

[0058] Figure 3 A schematic diagram for selecting lane line corner points and estimating their coordinates;

[0059] Figure 4(a) is a schematic diagram of roadside video perception trajectory data in pixel coordinate system;

[0060] Figure 4(b) is a schematic diagram of the roadside video sensing trajectory data initially converted to the world coordinate system;

[0061] Figure 5 This is a schematic diagram showing the result after correcting the world coordinates of the roadside video sensing trajectory data. Detailed Implementation

[0062] The present invention will be further described below with reference to the accompanying drawings, as shown in the figures:

[0063] This embodiment discloses a reliable fusion method for road domain perception data for vehicle-road cooperative autonomous driving, including the following steps:

[0064] S1. The roadside millimeter-wave radar and the roadside video camera simultaneously collect data to obtain a structured roadside radar perception trajectory dataset and a roadside video perception trajectory dataset.

[0065] S2. Use the lane line corner point coordinates to perform preliminary calibration of the roadside video camera, and initially transform the video perception trajectory coordinates from the pixel coordinate system to the world coordinate system.

[0066] S3. Use vehicle trajectory similarity to match the radar and video perception trajectory data, and solve for the initial values ​​of coordinate transformation parameters based on the trajectory matching relationship;

[0067] S4. Use the Gauss-Newton optimization algorithm to solve the coordinate correction parameters and correct the world coordinates of the video sensing trajectory data. Then, fuse the corrected video sensing trajectory data with the radar sensing trajectory data.

[0068] S5. Periodically verify the accuracy of the video-sensing trajectory using the stored video-sensing trajectory data. When the video-sensing trajectory error exceeds the threshold, recalculate the coordinate correction parameters using the process in step S4 and correct the video-sensing trajectory data.

[0069] This embodiment focuses on roadside millimeter-wave radar sensors and roadside video sensors installed along the Donghai Bridge. The roadside millimeter-wave radar has a sampling frequency of 10Hz, and the roadside video camera has a sampling frequency of 25Hz. Both types of sensors have the same sensing area and can collect vehicle target-level trajectory data within their sensing range. The corresponding installation locations are as follows: Figure 2 As shown. This embodiment applies the above technical solution, and its main process is as follows:

[0070] Step S1: The roadside millimeter-wave radar and roadside video camera simultaneously collect data to obtain a structured roadside radar perception trajectory dataset and a roadside video perception trajectory dataset, specifically including:

[0071] Step S11: Use roadside millimeter-wave radar to collect perception data, identify the movement information of vehicles with the same ID, and sort their position coordinates by time to form trajectories, thus constructing a roadside radar perception trajectory dataset. Among them, the trajectory of vehicle j Let be the positioning coordinates of vehicle j at time a, which are located in the world coordinate system.

[0072] Step S12: Use roadside video cameras to collect perception data. Based on the acquired video data, use YOLO and DeepSORT algorithms to detect and track vehicle targets, identifying vehicles with the same ID. Then, use the midpoint coordinates of the bottom edge of each target detection box as the vehicle position coordinates and sort them by time to form a trajectory, thus creating a roadside video perception trajectory dataset. Among them, the trajectory of vehicle j The coordinates of vehicle j at time b are located in the pixel coordinate system.

[0073] Step S2: Use the lane line corner point coordinates to perform preliminary calibration of the roadside video camera, and initially transform the video perception trajectory coordinates from the pixel coordinate system to the world coordinate system. This specifically includes:

[0074] Step S21: Select lane corner points in the roadside sensing image to construct a vertical grid, and select four points close to the millimeter-wave radar. Select one of these points as the calibration origin, and refer to lane line standards to obtain prior information on the relative positions of lane lines in space (the longitudinal length of Chinese lane lines is 6m along the lane direction, the longitudinal distance between lane lines is 9m, and the lane width is within the range of 3-4m). Then, estimate the approximate world coordinates of the other three points. The selection of corresponding calibration points and coordinate estimation are as follows: Figure 3 As shown.

[0075] Step S22: Based on the estimated world coordinates and pixel coordinates of the selected calibration points, solve for the optimal homography matrix using the least squares method. The formula for calculating the corresponding homography matrix H is as follows:

[0076]

[0077] Where H is the homography matrix, Z is the scaling factor, u and v are pixel coordinates, and X and Y are the corresponding world coordinates.

[0078] Step S23: Using the homography matrix obtained from the solution, perform a preliminary transformation from the pixel coordinate system to the world coordinate system to obtain the preliminary transformed roadside video perception trajectory dataset. Wherein, the trajectory of vehicle j is denoted as Figure 4 shows the initial coordinates of vehicle j at time b, after its coordinates have been transformed to the world coordinate system. The effect of the initial coordinate transformation of the video-sensing trajectory is shown in Figure 4.

[0079] Step S3: Matching radar and video-sensing trajectory data using vehicle trajectory similarity, and solving for initial values ​​of coordinate transformation parameters based on the trajectory matching relationship. Specifically, this includes:

[0080] Step S31: Use Gaussian process regression to process the roadside radar sensing trajectory dataset R.r and the initially converted roadside video perception trajectory dataset Interpolation is performed on each sensing trajectory in the data. The sampling frequency of each trajectory data is unified to 25Hz, based on the sampling frequency of the roadside video sensing data.

[0081] S32, Dataset R of roadside radar sensing trajectory r The trajectories in Calculate its relationship with the initially converted roadside video sensing trajectory dataset. Each trajectory DTW distance between

[0082] The calculation formula is as follows:

[0083]

[0084] Where d(·) is the distance between the two coordinate points, which can be Euclidean distance; These are the coordinates of the first point in the trajectory. This is the trajectory segment remaining after removing the first point.

[0085] Step S33: Construct the cost matrix C based on the calculated DTW distance. DTW And according to C DTW The Hungarian algorithm is used to solve the correspondence and matching relationship between radar sensing trajectory and video sensing trajectory.

[0086] The corresponding DTW distance cost matrix is ​​in the following form:

[0087]

[0088] Step S34: Based on the correspondence between the radar sensing trajectory and the video sensing trajectory determined in step S33, the average distance between the corresponding matching trajectory points is calculated using the singular value decomposition algorithm. and the corresponding coordinate transformation parameters

[0089] The corresponding calculation formula is as follows:

[0090]

[0091] Where mean(·) represents the average value of the trajectory coordinates. Roadside video perception trajectory dataset for initial conversion Trajectory in The coordinates at time i, For the roadside radar sensing trajectory dataset R r The trajectory that corresponds to the video trajectory In i+td The coordinates of time, θ * The initial value of the rotation angle is Δx. * Indicates the initial value of the translation in the x-direction, Δy * This represents the initial value of the translation in the y-direction. Let W represent the initial time delay; W represents the matrix to be decomposed. Singular value decomposition of W yields the left orthogonal matrix U, the diagonal matrix Λ, and the right orthogonal matrix V.

[0092] Step S35: Based on the solved coordinate transformation parameters Reassess the roadside video perception trajectory dataset Perform coordinate transformation on each trajectory, and repeat steps S32-S34 until the average distance between two consecutive calculations is obtained. The difference is less than 0.2.

[0093] Step S4: Solve for the coordinate correction parameters using the Gauss-Newton optimization algorithm and correct the world coordinates of the video sensing trajectory data. Then, fuse the corrected video sensing trajectory data with the radar sensing trajectory data. This specifically includes:

[0094] Step S41: Construct 5 parameters (ΔX, ΔY, θ, K) x ,K y Coordinate correction model for the initially transformed roadside video perception trajectory dataset Each trajectory The world coordinates are corrected using the following formula:

[0095]

[0096] Among them, K x K y θ is the scaling factor; θ is the plane angle; (x w_j (i),y w_j (i)) is Mid-track The world coordinates obtained from the initial transformation; (x cw_j (i),y cw_j (i) is for the trajectory Corrected world coordinates; (ΔX,ΔY,θ,K) x ,K y ) represents the coordinate correction parameter, ΔX represents the translation in the x-direction, and ΔY represents the translation in the y-direction.

[0097] Step S42: Construct the objective function based on the average distance between each point on the video sensing trajectory after coordinate correction and the corresponding matching point on the radar sensing trajectory, using the (Δx) determined in step S35. * ,Δy * ,θ *The coordinate transformation parameters are used as initial values ​​for the optimization parameters. The coordinate correction parameters (ΔX, ΔY, θ, K) are solved using the Gauss-Newton optimization algorithm. x ,K y The corresponding optimization objective function is as follows:

[0098]

[0099] Among them, (x cw_j (i),y cw_j (i)) is for World coordinates after correction of each trajectory; (x r_j (i),y r_j (i) represents the position coordinates of the corresponding matching trajectory in the roadside radar perception trajectory dataset Rr, n represents the number of frames within the common detection time, and k represents the total number of vehicles.

[0100] Step S43: Based on the optimized coordinate correction parameters (ΔX, ΔY, θ, K) x ,K y ), for the initially converted roadside video perception trajectory dataset The world coordinates of each trajectory are corrected to obtain the corrected roadside video perception trajectory dataset. The corresponding coordinate correction effect is as follows: Figure 5 As shown.

[0101] Step S44: The corrected video-aware trajectory dataset Each trajectory in the dataset is compared with the radar sensing trajectory dataset. The corresponding matched trajectories are fused to obtain the Rave-Vision fused trajectory dataset.

[0102] Among them, the trajectory of vehicle j Let the radar-visual fusion positioning coordinates of vehicle j at time p be the coordinates of the vehicle j. The corresponding coordinate fusion formula is as follows:

[0103]

[0104] in, These are the position coordinates of the corrected video sensing trajectory. λ represents the position coordinates of the corresponding matched radar sensing trajectory, and λ is the coordinate fusion weight.

[0105] Step S5: Periodically verify the accuracy of the video-visual fusion using the stored video-visual perception trajectory data. When the video-visual trajectory error exceeds the threshold, recalculate the coordinate correction parameters using the process in step S4 and correct the video-visual perception trajectory data. Specifically, this includes:

[0106] Step S51: Store the corrected video-aware trajectory dataset. With radar sensing trajectory dataset Calculate the average distance D between corresponding matching trajectories in the two trajectory datasets every 7 days. c (R c,w ,R r Determine the average distance D. c (R c,w ,R r Does it exceed 1m? Average distance D c (R c,w ,R r The calculation formula for ) is as follows:

[0107]

[0108] Among them, (x cw_j (i),y cw_j (i) represents the corrected video-aware trajectory dataset R. c,w Mid-track The coordinates; (x r_j (i),y r_j (i) represents the radar sensing trajectory dataset R. r Corresponding matching trajectory The coordinates are defined by mean(·), where mean(·) represents the average value, n represents the number of frames within the common detection time, and k represents the total number of vehicles.

[0109] Step S52, when the average distance D c (R c,w ,R r When the distance is less than 1m, the existing calibration results can continue to be used for radar-visual data fusion; when the average distance D c (R c,w ,R r When the distance is greater than 1m, the coordinates of the video perception trajectory are corrected again according to the process in step S4, wherein the initial value of the optimization parameter in step S43 is taken according to the parameter determined in the previous calibration.

[0110] This invention utilizes the motion similarity between lane line information in the scene and the corresponding matching trajectory in the radar-visual perception trajectory dataset to automatically achieve corresponding coordinate matching and transformation parameter solving. It periodically judges the quality of radar-visual fusion and recalculates coordinate correction parameters based on historical perception data, which greatly simplifies the workload of radar-visual coordinate spatiotemporal registration and data fusion, and ensures that the radar-visual fusion system always maintains a good perception effect.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for road area perception data trust fusion for vehicle-road cooperative automatic driving, characterized in that: include: S1. Use roadside radar and roadside video cameras to collect data synchronously, and obtain a structured roadside radar perception trajectory dataset and a roadside video perception trajectory dataset. S2. Use the lane line corner point coordinates to perform preliminary calibration of the roadside video camera, and initially transform the video perception trajectory coordinates from the pixel coordinate system to the world coordinate system; S3. Use vehicle trajectory similarity to match radar and video sensing trajectory data, and solve for the initial values ​​of coordinate transformation parameters based on the trajectory matching relationship; S4. Solve for coordinate correction parameters and correct the world coordinates of the video sensing trajectory data, then fuse the corrected video sensing trajectory data with the radar sensing trajectory data; Step S4 specifically includes: S41. Construct a 5-parameter (x, y, z, s, t) coordinate correction model to correct the world coordinates of each trajectory in the preliminary converted roadside video perception trajectory dataset; the corresponding coordinate correction formula is as follows: ) coordinate correction model, to correct the world coordinates of each trajectory in the preliminary converted roadside video perception trajectory dataset the world coordinates of each trajectory in the preliminary converted roadside video perception trajectory dataset; the corresponding coordinate correction formula is as follows:​ ； in, , It is a scaling factor; It is a plane angle; for Mid-track The world coordinates obtained from the initial transformation; To track Corrected world coordinates; Indicates translation in the x-direction. Indicates translation in the y-direction; S42. Construct an objective function based on the average distance between each point on the video sensing trajectory after coordinate correction and the corresponding matching point on the radar sensing trajectory, using the objective function determined in step S35. The coordinate transformation parameters are used as initial values ​​for the optimization parameters, and the optimized coordinate correction parameters are then solved. ); S43. Based on the optimized coordinate correction parameters ( ), for the initially converted roadside video perception trajectory dataset The world coordinates of each trajectory are corrected to obtain the corrected roadside video perception trajectory dataset. ; S44. The corrected video-aware trajectory dataset Each trajectory in the dataset is compared with the radar sensing trajectory dataset. The corresponding matched trajectories are fused to obtain the Rave-Vision fused trajectory dataset. Among them, the trajectory of vehicle j , Let the radar-visual fusion positioning coordinates of vehicle j at time p be the coordinates of the vehicle j. The corresponding coordinate fusion formula is as follows: ； in, These are the position coordinates of the corrected video sensing trajectory. These are the position coordinates of the corresponding matched radar sensing trajectory. For coordinate fusion weights; S5. Periodically verify the accuracy of the video-sensing trajectory using the stored video-sensing trajectory data. When the video-sensing trajectory error exceeds the threshold, recalculate the coordinate correction parameters using the process in step S4 and correct the video-sensing trajectory data.

2. The reliable fusion method for road domain perception data for vehicle-road cooperative autonomous driving according to claim 1, characterized in that: Step S1 specifically includes: S11. Use roadside radar to collect perception data, identify vehicles with the same ID, and sort their position coordinates by time to form trajectories, thus creating a roadside radar perception trajectory dataset. Wherein, the trajectory of vehicle j is denoted as , Let be the positioning coordinates of vehicle j at time a, located in the world coordinate system; S12. Collect perception data using roadside video cameras, perform target detection and tracking based on the video data to determine the IDs of each vehicle, and use the coordinates of the midpoint of the bottom edge of each target detection box as the vehicle position coordinates, sort them by time to form a trajectory, thus constructing a roadside video perception trajectory dataset. Wherein, the trajectory of vehicle j is denoted as , Let be the positioning coordinates of vehicle j at time b, which are located in the pixel coordinate system.

3. The reliable fusion method for road domain perception data for vehicle-road cooperative autonomous driving according to claim 1, characterized in that: Step S2 specifically includes: S21. Select lane corner points in the roadside perception image to construct a vertical grid, select no less than four lane corner points as calibration points, take the point closest to the radar as the calibration origin, and estimate the approximate world coordinates of each calibration point based on the prior information of the relative position of the lane lines in space. S22. Based on the estimated world coordinates and pixel coordinates of the selected calibration points, the optimal homography matrix is ​​solved using the least squares method. The formula for calculating the homography matrix H is as follows: Where H is the homography matrix, Z is the scaling factor, u and v are pixel coordinates, and X and Y are the corresponding world coordinates; S23. Using the homography matrix obtained from the solution, a preliminary transformation from the pixel coordinate system to the world coordinate system is achieved, resulting in a preliminary transformed roadside video perception trajectory dataset. Wherein, the trajectory of vehicle j is denoted as , Let b be the position coordinates of vehicle j when it is initially converted to the world coordinate system.

4. The reliable fusion method for road domain perception data for vehicle-road cooperative autonomous driving according to claim 1, characterized in that: Step S3 specifically includes: S31. Using Gaussian process regression on the roadside radar sensing trajectory dataset. and the initially converted roadside video perception trajectory dataset Interpolate the various sensing trajectories in the data to unify the sampling frequency of each trajectory; S32. Dataset of roadside radar sensing trajectory The trajectories in Calculate its relationship with the initially converted roadside video perception trajectory dataset. Each trajectory DTW distance between ; S33. Construct the cost matrix based on the calculated DTW distance. and based on Solve for the correspondence and matching relationship between the radar-sensing trajectory and the video-sensing trajectory, the cost matrix. The format is as follows: ； S34. Based on the correspondence and matching relationship between the radar sensing trajectory and the video sensing trajectory determined in S33, calculate the average distance between the corresponding matching trajectory points. and the corresponding initial values ​​of coordinate transformation parameters ; S35. Based on the coordinate transformation parameters obtained from the solution. Reassess the roadside video perception trajectory dataset Perform coordinate transformation on each trajectory, and repeat steps S32-S34 until the average distance between two consecutive calculations is obtained. The difference is less than .

5. The reliable fusion method for road domain perception data for vehicle-road cooperative autonomous driving according to claim 4, characterized in that: The DTW distance : in, This represents the distance between two coordinate points; These are the coordinates of the first point in the corresponding trajectory. These are the trajectory segments remaining after removing the first point from the corresponding trajectory.

6. The reliable fusion method for road domain perception data for vehicle-road cooperative autonomous driving according to claim 4, characterized in that: The average distance shown for: ； in, This indicates that the average value of the trajectory coordinates is taken. Roadside video perception trajectory dataset for initial conversion Trajectory in The coordinates at time i, For roadside radar sensing trajectory dataset The trajectory that corresponds to the video trajectory exist The coordinates of time; This represents the initial value of the rotation angle. This represents the initial value of the translation in the x-direction. This represents the initial value of the translation in the y-direction. Indicates the initial value of the delay; Represents the matrix to be decomposed, for Singular value decomposition yields a left orthogonal matrix. diagonal matrix and right orthogonal matrix .

7. The reliable fusion method for road domain perception data for vehicle-road cooperative autonomous driving according to claim 1, characterized in that: The objective function is: ； in, To World coordinates after correction of each trajectory; For roadside radar sensing trajectory dataset The coordinates of the corresponding matching trajectory are given, where n represents the number of frames within the common detection time and k represents the total number of vehicles.

8. The reliable fusion method for road domain perception data for vehicle-road cooperative autonomous driving according to claim 1, characterized in that: Step S5 specifically includes: S51. Store the corrected video-aware trajectory dataset With radar sensing trajectory dataset Calculate the average distance between corresponding matching trajectories in the two trajectory datasets every c days. Determine whether it exceeds the threshold. ; S52. When the average distance Not exceeding the threshold At this time, existing calibration results can continue to be used for radar-visual data fusion; when the average distance Exceeding the threshold Then, the coordinates of the video perception trajectory are corrected again according to the process in step S4.

9. The reliable fusion method for road domain perception data for vehicle-road cooperative autonomous driving according to claim 8, characterized in that: The average distance The calculation formula is as follows: in, For the corrected video-aware trajectory dataset Mid-track The coordinates; For radar sensing trajectory dataset Corresponding matching trajectory coordinates This indicates that the average value is taken, where n represents the number of frames within the common detection time, and k represents the total number of vehicles.

Images