A smart traffic data fusion method based on vehicle-road cooperative sensors

Abstract

The present invention discloses a smart traffic data fusion method based on vehicle-road cooperative sensors. The present invention relates to the field of data fusion technology and solves the problem that if relevant monomer features are not calibrated, it is easy to cause the accuracy and precision of the relevant data images after fusion to be unable to be guaranteed in the later data fusion process. The present invention identifies the data performance of relevant monomers based on the relevant data collected by specific sensors, determines the data priority, and then in the subsequent data fusion process, preferentially selects the monomer with the most consistent data performance as the standard monomer, and performs relevant data fusion. After the relevant fusion of the image is completed, it is identified whether there is a relevant monomer to be adjusted, and the monomer to be adjusted is adjusted to complete the relevant fusion of the monomer to be adjusted, thereby completing the overall fusion effect of the data collected by the relevant sensors and ensuring the overall accuracy of the fused image.

Description

Technical Field

[0001] The present invention relates to the technical field of data fusion, and specifically to a smart traffic data fusion method based on vehicle-road cooperative sensors. Background Art

[0002] Vehicle-road cooperative sensor technology not only fills the blind spots of single-vehicle intelligence and improves the consistency and accuracy of information, but also plays an important role in complex road environments, providing support for ensuring traffic safety and improving traffic efficiency. Vehicle-road cooperative sensors include lidar, millimeter-wave radar and video cameras.

[0003] The application with publication number CN113442923B discloses an automatic driving optimization method and system that integrates vehicle perception and intelligent traffic data, including: setting the lane where the user's target vehicle is located as the target lane; obtaining the navigation route of the target vehicle; extracting the traffic light position between the vehicle's real-time position and the destination position from the navigation route; calculating the distance between the real-time position and its next traffic light position; when the distance is less than or equal to a first preset distance, obtaining the candidate lane in which the target vehicle is allowed to travel between the real-time position and the next traffic light position; obtaining other vehicles that are located in front of the target vehicle and on the candidate lane; obtaining the turning status of other vehicles, and pre-calculating the number of vehicles on each candidate lane after the other vehicles complete the turn; obtaining a candidate lane with the least number of vehicles, and setting the candidate lane as a transfer lane; controlling the target vehicle to enter the transfer lane for driving, which can help the user enter the lane with the least traffic in advance.

[0004] During the relevant fusion process of the intelligent traffic data of the vehicle cooperative sensors, the acquired relevant data are generally directly fused. However, in this fusion method, the relevant individual features are not calibrated during the actual processing, which can easily lead to the failure to ensure the accuracy and precision of the fused data images in the later data fusion process, and the display effect of the fused data is not good. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a smart traffic data fusion method based on vehicle-road cooperative sensors, which solves the problem that if the relevant individual features are not calibrated, it will easily lead to the problem that the accuracy and precision of the relevant data images after fusion cannot be guaranteed in the subsequent data fusion process.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a smart traffic data fusion method based on vehicle-road cooperative sensors, comprising the following steps:

[0007] S1) Acquire relevant data from the vehicle-road cooperative sensor and confirm the data priority based on the acquired relevant data, where the relevant data includes scene data and radar data. The specific method is as follows:

[0008] S11, identifying the relevant distance L1 of the farthest single feature in the scene data, and then identifying the relevant distance L2 of the farthest single feature identified in the corresponding radar data;

[0009] S12, if L1>L2, calibrate the scene data as priority data; if L1<L2, calibrate the radar data as priority data; if L1=L2, do not perform relevant calibration of the corresponding priority data;

[0010] S2), based on the scene data and radar data acquired at the current moment, similar monomer features existing in the identified related data are analyzed for feature data of similar monomer features to identify whether the scene data and radar data can be relatedly fused, specifically in the following manner:

[0011] S21, confirm the single features in the scene data and radar data in turn, identify the distance between the single feature and the vehicle body, and calibrate the distance between different single features confirmed in the scene data as J i , where i represents different monomer features, and the relevant distances of different monomer features confirmed in the radar data are calibrated as X i ;

[0012] S22, identify the distance difference C of the same type of monomer features i , and C i =|J i -X i |, and then the confirmed distance difference C i Perform variance processing to determine the unknown variance F;

[0013] S23, identifying whether the pending variance F satisfies: F≤Y1, where Y1 is a preset value. If so, it means that the distance between the scene data and the radar data is not much different, and relevant fusion can be performed; if the pending variance does not satisfy: F≤Y1, a data error signal is directly generated and relevant display is performed;

[0014] S3) Based on the similar monomer features confirmed in the scene data and radar data, the optimal monomer feature is confirmed, and the initial image fusion is performed. During the fusion, based on the priority of the data, different fusion processing methods are used to completely fuse the scene data and radar data. The specific methods are as follows:

[0015] S31, based on the scene data and the characteristics of the same monomer in the radar data, and from the determined groups of distance difference values ​​C i In the example, the distance difference Ci The monomer characteristic corresponding to the minimum value is calibrated as the standard monomer;

[0016] S32, identifying the external contour of the standard monomer from the scene pictures of the two sets of data, and identifying the feature points associated with the external contour, wherein the feature point is the intersection of two related lines, and the angle between the intersection and the two related lines satisfies: angle ≤ 90°, and the direction of the angle points to the middle position of the external contour, and confirming the external feature points of the two standard monomers one by one. After confirmation, based on the external feature points, the two standard monomers are relatedly fused. During the fusion process, the two related scene pictures are driven to be uniformly fused to confirm the initial fusion picture;

[0017] S33, identifying the overlap of other similar monomer features in the initial fusion image, confirming the center point of the corresponding similar monomer feature, and confirming the straight-line distance Lx between the two center points. If Lx≥Y2, the corresponding similar monomer feature is calibrated as the monomer to be adjusted. Otherwise, no calibration is performed, wherein Y2 is a preset value;

[0018] S34, based on the determined monomer to be adjusted, calibrate the monomer to be adjusted in the scene data as the scene monomer to be adjusted, and calibrate the monomer to be adjusted in the radar data as the radar monomer to be adjusted:

[0019] If the scene data is priority data, the center point of the radar unit to be adjusted is moved to the center point of the scene unit to be adjusted to complete the data fusion of the two units to be adjusted;

[0020] If the radar data is priority data, the center point of the scene to-be-adjusted unit is moved to the center point of the radar to-be-adjusted unit to complete the data fusion of the two to-be-adjusted units;

[0021] If the relevant calibration of the priority data is not performed, the scene to-be-adjusted monomer is taken as the priority monomer, and the center point of the radar to-be-adjusted monomer is moved to the center point of the scene to-be-adjusted monomer to complete the data fusion of the two to-be-adjusted monomers.

[0022] Preferably, in step S1, the vehicle cooperative sensor includes a camera sensor and a radar sensor, wherein the camera sensor acquires scene data around the vehicle, and the radar sensor acquires radar data around the vehicle.

[0023] Preferably, it also includes:

[0024] S4), based on the relevant data of the completed fusion, for the monomer features that appear in sequence later, the fusion data of the similar monomer features that have been fused is used as a benchmark, the monomer features are initially adjusted, and then it is identified whether there is a monomer to be adjusted, and then the monomer to be adjusted is fused; the specific sub-steps are:

[0025] S41, based on the monomer features that appear subsequently, identify the most recent similar monomer features that have been fused for this type of monomer, and based on the movement data corresponding to the similar monomer features, initially fuse the similar monomer features that appear subsequently to confirm the fusion scene;

[0026] S42, using the same method as step S3 to identify whether there are monomers to be adjusted in the fusion scenes that appear in sequence, and perform relevant fusion on the monomers to be adjusted, complete the relevant calibration process of the subsequent fusion scenes, and fuse the features of the monomers that appear one by one.

[0027] The present invention provides a smart traffic data fusion method based on vehicle-road cooperative sensors. Compared with the prior art, it has the following beneficial effects:

[0028] The present invention identifies the data performance of related monomers based on the related data collected by specific sensors, determines the data priority, and then in the subsequent data fusion process, preferentially selects the monomer with the most consistent data performance as the standard monomer, and performs related data fusion. After the relevant fusion of the picture is completed, it is identified whether there is a related monomer to be adjusted, and the monomer to be adjusted is adjusted to complete the relevant fusion of the monomer to be adjusted, thereby completing the overall fusion effect of the data collected by the related sensors and ensuring the overall accuracy of the fused picture;

[0029] For the related images generated subsequently, the subsequent real-time monomers are fused and calibrated in real time to ensure that the corresponding smart traffic data can be quickly integrated to achieve better data fusion processing effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic diagram of the process of the present invention;

[0031] Figure 2 This is a schematic diagram of the evaluation of priority data of the present invention. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] Example 1

[0034] See also Figure 1 as well as Figure 2 The present application provides a method for intelligent traffic data fusion based on vehicle-road cooperative sensors, comprising the following steps:

[0035] S1) Acquire relevant data of the vehicle-road cooperative sensor, and confirm data priority based on the relevant data acquired, wherein the vehicle cooperative sensor includes a camera sensor and a radar sensor, wherein the camera sensor acquires scene data around the vehicle, and the radar sensor acquires radar data around the vehicle. Specifically, the radar data and the scene data are two different data, wherein the radar data can clearly acquire the distance data of the relevant object, but the overall outline and color of the relevant object are not clear, and the scene data can acquire the overall outline and color of the relevant object, but the relevant distance data is not accurate, so the relevant data of the corresponding sensors need to be fused to obtain the most complete scene data, wherein the specific method of determining the data priority is:

[0036] S11, identify the relevant distance L1 of the farthest single feature in the scene data, and then identify the relevant distance L2 of the farthest single feature identified in the corresponding radar data. Specifically, in the scene data, the farthest distance of the farthest single feature can be directly obtained based on the corresponding preset ratio. The single feature can be determined based on the relevant surface features and numerical features. The single feature may include vehicles on the road or external related buildings, etc. Since the method of determining single features is relatively popular in the prior art, it will not be described in detail here: Use algorithms such as Harris, SIFT, SURF, FAST, BRIEF and ORB to detect key points in the image, such as edges, corners, straight lines and other local features. These algorithms can automatically detect areas with specific geometric shapes or grayscale distributions from images; for example, the Harris algorithm is suitable for motion image tracking, while the FAST algorithm focuses on improving the speed of feature point detection, etc.;

[0037] S12, if L1>L2, the scene data is calibrated as priority data; if L1<L2, the radar data is calibrated as priority data; if L1=L2, no relevant calibration of the corresponding priority data is performed. This is mainly aimed at the relevant environment caused by different weather conditions. Different weather environments will not have relevant effects on radars, but will have effects on camera sensors. If the camera image is not enough to cover the range of the radar sensor, then the priority of radar data must be higher than that of camera data. If it is on a sunny day, the range of the camera data must be larger than that of the radar data, so the priority of the camera data must be higher than that of the radar data. In order to determine the relevant priorities of different data, it is to follow the specified fusion rules in the subsequent data fusion process to ensure the corresponding data fusion effect;

[0038] S2), based on the scene data and radar data acquired at the current moment, similar monomer features existing in the identified related data are analyzed for feature data of similar monomer features to identify whether the scene data and radar data can be relatedly fused. Specifically, in order to identify whether the related data meets the corresponding fusion conditions, it is necessary to evaluate the numerical differences of the corresponding monomer features, and based on the corresponding specific numerical difference degrees, evaluate the final fusion process;

[0039] The specific method of identification is as follows:

[0040] S21, confirm the single features in the scene data and radar data in turn, identify the distance between the single feature and the vehicle body, and calibrate the distance between different single features confirmed in the scene data as J i , where i represents different monomer features, and the relevant distances of different monomer features confirmed in the radar data are calibrated as X i ;

[0041] S22, identify the distance difference C of the same type of monomer features i , and C i =|J i -X i |, and then the confirmed distance difference C i Perform variance processing to determine the undetermined variance F, wherein the specific method of variance processing is: propose i = 1, 2, ..., n, and several groups of distance difference values ​​C i The mean value of is Jz, using Confirm its pending variance F;

[0042] S23, identifying whether the undetermined variance F satisfies: F≤Y1, where Y1 is a preset value, and its specific value is determined by the operator based on experience. If it satisfies, it means that the distance between the scene data and the radar data is not much different, and relevant fusion can be performed. If it does not satisfy, a data error signal is directly generated and relevant display is performed, indicating that a certain sensor has a data accuracy problem, and human intervention and relevant adjustments are required;

[0043] S3), based on the similar monomer features confirmed in the scene data and radar data, confirm the optimal monomer feature, and perform initial fusion of the picture. During the fusion, based on the priority of the data, different fusion processing methods are used to completely fuse the scene data and radar data. Specifically, during the data fusion process, generally based on the corresponding features, the relevant feature points of the features are determined, and based on the corresponding feature points, the relevant similar features can be fused. During the fusion, the scene in the scene data and the scene in the radar data are driven to be initially fused, and then identify whether there is a correlation deviation in other monomer features after the fusion. For the related monomers with correlation deviations, the center points are replaced to complete the full fusion of the entire picture data;

[0044] The specific method of performing image fusion is as follows:

[0045] S31, based on the scene data and the characteristics of the same monomer in the radar data, and from the determined groups of distance difference values ​​C i In the example, the distance difference C i The monomer characteristic corresponding to the minimum value is calibrated as the standard monomer;

[0046] S32, identifying the outer contour of the standard monomer from the scene pictures of the two sets of data, and identifying the feature points associated with the outer contour, wherein the feature point is the intersection of two related lines, and the angle between the intersection and the two related lines satisfies: angle ≤ 90°, and the direction of the angle points to the middle position of the outer contour (because there are two groups of angles of the intersection generated by the related lines, one group is the external angle and the other group is the internal angle, so in order to make a related judgment, it is necessary to select the internal angle, that is, the related angle pointing to the middle position of the outer contour), confirm the external feature points of the two standard monomers one by one, and after the confirmation is completed, the two standard monomers are relatedly fused based on the external feature points. During the fusion process, the two related scene pictures are driven to be uniformly fused to confirm the initial fusion picture;

[0047] S33, identifying the overlap of other similar monomer features in the initial fusion image, confirming the center point of the corresponding similar monomer feature, and confirming the straight-line distance Lx between the two center points (if there are two groups of similar monomer features, then there are two center points, and there is a related straight-line distance), if Lx≥Y2, then the corresponding similar monomer feature is calibrated as the monomer to be adjusted, otherwise, no calibration is performed, wherein Y2 is a preset value, and its specific value is determined by the operator based on experience;

[0048] S34, based on the determined monomer to be adjusted, calibrate the monomer to be adjusted in the scene data as the scene monomer to be adjusted, and calibrate the monomer to be adjusted in the radar data as the radar monomer to be adjusted:

[0049] If the scene data is priority data, the center point of the radar unit to be adjusted is moved to the center point of the scene unit to be adjusted to complete the data fusion of the two units to be adjusted;

[0050] If the radar data is priority data, the center point of the scene to-be-adjusted unit is moved to the center point of the radar to-be-adjusted unit to complete the data fusion of the two to-be-adjusted units;

[0051] If the relevant calibration of the priority data is not performed, the scene to-be-adjusted monomer is taken as the priority monomer, and the center point of the radar to-be-adjusted monomer is moved to the center point of the scene to-be-adjusted monomer to complete the data fusion of the two to-be-adjusted monomers.

[0052] Specifically, according to the confirmed monomers to be adjusted, in order to ensure a more standardized fusion of the corresponding scene data, it is necessary to determine which monomer's picture data is more complete and clear. For rainy and foggy weather, the radar data is definitely better than the picture data, so the relevant data in the radar data shall prevail. For clearer weather, the monomer outline in the picture data is clearer than the radar data, so the priority of its scene data is definitely higher than the radar data. After relevant fusion, its radar data can better display the distance-related parameters of the corresponding monomer, and its picture data can better display the overall outline shape of the corresponding monomer, thereby ensuring the overall accuracy after the fusion of the picture.

[0053] Example 2

[0054] In the specific implementation process, compared with the above embodiment, this embodiment further includes:

[0055] S4), based on the relevant data of the completed fusion, for the monomer features that appear in sequence later, the fusion data of the similar monomer features that have been fused is used as a benchmark, the monomer features are initially adjusted, and then it is identified whether there is a monomer to be adjusted, and then the monomer to be adjusted is related to the fusion, wherein the specific sub-steps of the processing are:

[0056] S41, based on the monomer features that appear subsequently, identify the nearest similar monomer features that have been fused, and based on the movement data of the corresponding similar monomer features, initially fuse the similar monomer features that appear subsequently, and confirm the fusion scene. Specifically, the corresponding movement data has a movement direction and a movement distance, so based on this kind of movement data, the relatively close monomer features that appear subsequently are fused one by one to determine the relevant fusion scene;

[0057] S42, using the same method as step S3 to identify whether there are monomers to be adjusted in the fusion scenes that appear in sequence, and perform relevant fusion on the monomers to be adjusted, complete the relevant calibration process of the subsequent fusion scenes, and fuse the features of the monomers that appear one by one.

[0058] Specifically, for the relevant data generated by different sensors, the data are fused based on the corresponding data standards to ensure the relevant optimization of the corresponding scene data, so that the corresponding scene data can achieve better optimization processing effects and achieve better data fusion processing effects.

[0059] Example 3

[0060] The specific implementation process of this embodiment includes the entire implementation process of the above two groups of embodiments.

[0061] Some of the data in the above formulas are dimensionless and numerically calculated. Meanwhile, the contents not described in detail in this specification belong to the prior art known to those skilled in the art.

[0062] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A smart traffic data fusion method based on vehicle-road cooperative sensors, characterized in that: The following steps are involved: S1) Acquire relevant data from the vehicle-road cooperative sensor and confirm the data priority based on the acquired relevant data, where the relevant data includes scene data and radar data. The specific method is as follows: S11, identifying the relevant distance L1 of the farthest single feature in the scene data, and then identifying the relevant distance L2 of the farthest single feature identified in the corresponding radar data; S12, if L1>L2, calibrate the scene data as priority data; if L1<L2, calibrate the radar data as priority data; if L1=L2, do not perform relevant calibration of the corresponding priority data; S2), based on the scene data and radar data acquired at the current moment, similar monomer features existing in the identified related data are analyzed for feature data of similar monomer features to identify whether the scene data and radar data can be relatedly fused, specifically in the following manner: S21, confirm the single features in the scene data and radar data in turn, identify the distance between the single feature and the vehicle body, and calibrate the distance between different single features confirmed in the scene data as J i , where i represents different monomer features, and the relevant distances of different monomer features confirmed in the radar data are calibrated as X i ; S22, identify the distance difference C of the same type of monomer features i , and C i =|J i -X i |, and then the confirmed distance difference C i Perform variance processing to determine the unknown variance F; S23, identifying whether the undetermined variance F satisfies: F≤Y1, where Y1 is a preset value. If so, it means that the distance between the scene data and the radar data is not much different, and relevant fusion can be performed; S3) Based on the similar monomer features confirmed in the scene data and radar data, the optimal monomer feature is confirmed, and the initial image fusion is performed. During the fusion, based on the priority of the data, different fusion processing methods are used to completely fuse the scene data and radar data. The specific methods are as follows: S31, based on the scene data and the characteristics of the same monomer in the radar data, and from the determined groups of distance difference values ​​C i In the example, the distance difference C i The monomer characteristic corresponding to the minimum value is calibrated as the standard monomer; S32, identifying the external contour of the standard monomer from the scene pictures of the two sets of data, and identifying the feature points associated with the external contour, wherein the feature point is the intersection of two related lines, and the angle between the intersection and the two related lines satisfies: angle ≤ 90°, and the direction of the angle points to the middle position of the external contour, and confirming the external feature points of the two standard monomers one by one. After confirmation, based on the external feature points, the two standard monomers are relatedly fused. During the fusion process, the two related scene pictures are driven to be uniformly fused to confirm the initial fusion picture; S33, identifying the overlap of other similar monomer features in the initial fusion image, confirming the center point of the corresponding similar monomer feature, and confirming the straight-line distance Lx between the two center points. If Lx≥Y2, the corresponding similar monomer feature is calibrated as the monomer to be adjusted. Otherwise, no calibration is performed, wherein Y2 is a preset value; S34, based on the determined monomer to be adjusted, calibrate the monomer to be adjusted in the scene data as the scene monomer to be adjusted, and calibrate the monomer to be adjusted in the radar data as the radar monomer to be adjusted: If the scene data is priority data, the center point of the radar unit to be adjusted is moved to the center point of the scene unit to be adjusted to complete the data fusion of the two units to be adjusted; If the radar data is priority data, the center point of the scene to-be-adjusted unit is moved to the center point of the radar to-be-adjusted unit to complete the data fusion of the two to-be-adjusted units; If the relevant calibration of the priority data is not performed, the scene to-be-adjusted monomer is taken as the priority monomer, and the center point of the radar to-be-adjusted monomer is moved to the center point of the scene to-be-adjusted monomer to complete the data fusion of the two to-be-adjusted monomers.

2. According to claim 1, a smart traffic data fusion method based on vehicle-road cooperative sensors is characterized in that: In the step S1, the vehicle cooperative sensor includes a camera sensor and a radar sensor, wherein the camera sensor acquires scene data around the vehicle, and the radar sensor acquires radar data around the vehicle.

3. The intelligent traffic data fusion method based on vehicle-road cooperative sensor according to claim 1 is characterized in that: In the step S23, if the undetermined variance does not satisfy: F≤Y1, a data error signal is directly generated and related display is performed.

4. The intelligent traffic data fusion method based on vehicle-road cooperative sensor according to claim 1 is characterized in that: Also includes: S4) Based on the relevant data of the completed fusion, for the monomer features that appear in sequence later, the fusion data of the similar monomer features that have been fused are used as a benchmark to perform initial adjustment on the monomer features, then identify whether there are monomers to be adjusted, and then perform relevant fusion on the monomers to be adjusted.

5. The intelligent traffic data fusion method based on vehicle-road cooperative sensor according to claim 4 is characterized in that: In step S4, the specific sub-steps for processing are: S41, based on the monomer features that appear subsequently, identify the most recent similar monomer features that have been fused for this type of monomer, and based on the movement data corresponding to the similar monomer features, initially fuse the similar monomer features that appear subsequently to confirm the fusion scene; S42, using the same method as step S3 to identify whether there are monomers to be adjusted in the fusion scenes that appear in sequence, and perform relevant fusion on the monomers to be adjusted, complete the relevant calibration process of the subsequent fusion scenes, and fuse the features of the monomers that appear one by one.

Images