A method and device for identifying vehicles cutting into the side lane based on multi-sensor fusion

Abstract

This invention proposes a method and device for identifying vehicles cutting into adjacent lanes based on multi-sensor fusion. Utilizing a Kalman filter fusion model, it associates the target and outputs three types of lateral position information: lateral position information detected by millimeter-wave radar, lateral position information detected by a camera, and lateral position information calculated by the Kalman filter fusion model. It then determines whether a vehicle is cutting in or calculates the probability of cutting in. This invention and its technical solution can accurately predict cutting-in scenarios in advance, enabling ADAS (Advanced Driver Assistance Systems) functions to control the vehicle ahead of time, reducing driver intervention frequency and improving comfort.

Description

Technical Field

[0001] This invention belongs to the field of intelligent driving technology for automobiles, and in particular relates to a method and device for identifying vehicles cutting into the adjacent lane based on multi-sensor fusion. Background Technology

[0002] In ADAS driver assistance systems, when a vehicle in the adjacent lane cuts in close proximity to the vehicle, the system often fails to switch to a dangerous target in time, thus affecting the system's effectiveness.

[0003] In existing technologies, there are several possible solutions for detecting vehicles cutting into adjacent lanes. A single radar solution can detect the lateral distance of the target and determine whether the target is approaching the vehicle laterally. A single camera solution can determine this based on the relative position of the vehicle and the lane lines. A solution combining radar and camera data can determine whether the target intends to enter the lane by combining lane line and target lateral position information.

[0004] However, all of the above solutions have shortcomings. The single radar solution cannot obtain lane line information, thus it cannot accurately determine the distance between the vehicle in the adjacent lane and the lane line of the vehicle itself; the single camera solution has a large image processing computation load, insufficient real-time performance, and only starts to judge when the vehicle crosses the line, which is too late; the radar and camera fusion solution has data synchronization problems, and the lateral position information output by the sensor may be disturbed, affecting the entry judgment. Summary of the Invention

[0005] This invention proposes a method and device for identifying vehicles cutting into adjacent lanes based on multi-sensor fusion, which can accurately identify and predict cutting-in scenarios.

[0006] To achieve the above objectives, the technical solution of the present invention is implemented as follows:

[0007] A method for identifying vehicles cutting into adjacent lanes based on multi-sensor fusion includes:

[0008] S1. Obtain the output information from the vehicle-mounted camera and millimeter-wave radar installed on the vehicle;

[0009] S2. Based on the output information obtained in step S1, synchronize the target information and lane line information in time.

[0010] S3. Using the Kalman filter fusion model, the target is correlated and three types of lateral position information of the target are output, including the lateral position information detected by the millimeter-wave radar, the lateral position information detected by the camera, and the lateral position information calculated by the Kalman filter fusion model.

[0011] S4. Compare the lateral position information of this detection cycle with the lateral position information of the previous detection cycle to determine whether this detection cycle is an entry point; if not, increment the error count by 1, increment the consecutive error count by 1, and increment the total count by 1.

[0012] S5. When the number of errors, the number of consecutive errors, and the total number of times meet the preset threshold, all are reset to zero, and the counting and judgment are restarted; otherwise, the entry probability is determined based on the number of errors, the number of consecutive errors, and the total number of times.

[0013] S6. Record the three types of lateral position information for this detection cycle, for comparison in the next detection cycle.

[0014] Furthermore, the time synchronization method described in step S2 includes: obtaining the timestamps in the target information and lane line information; using the timestamp of the lane line information as the reference time; and using the timestamp of the target information and the position and speed information to calculate the position information of the target at the reference time, thereby achieving time synchronization.

[0015] Furthermore, the method for associating targets using the Kalman filter fusion model described in step S3 includes:

[0016] Using the target information output by the Kalman filter fusion model as the reference target information, the target information output by the radar and camera is compared with the reference target information using Euclidean distance to obtain the correlation distance matrix between the reference target and the target output by the radar and camera. Then, the nearest target is associated by traversing the matrix.

[0017] Furthermore, the method for determining the entry probability based on the number of errors and the total number of attempts in step S5 includes:

[0018] Based on the correlation data between the number of errors, consecutive errors, and total errors and the entry probability, and using the correlation data and the number of errors and total errors in this detection cycle, the entry probability is obtained.

[0019] Furthermore, the preset threshold in step S5 includes: consecutive error count > 5 or error count > 2 / 3 * total count.

[0020] In another aspect, the present invention also proposes a vehicle lane-changing detection device based on multi-sensor fusion, comprising:

[0021] The acquisition module is used to acquire the output information of the vehicle-mounted camera and millimeter-wave radar installed on the vehicle;

[0022] The synchronization module synchronizes the target information and lane line information in time based on the output information obtained by the acquisition module.

[0023] The output module uses a Kalman filter fusion model to associate targets and output three types of lateral position information of the targets, including lateral position information detected by millimeter-wave radar, lateral position information detected by camera, and lateral position information calculated by Kalman filter fusion model.

[0024] The comparison module compares the lateral position information of the current detection cycle with the lateral position information of the previous detection cycle to determine whether the current detection cycle is an entry point; if not, the error count is incremented by 1, the consecutive error count is incremented by 1, and the total count is incremented by 1.

[0025] The cut-in probability module resets all values ​​to zero and restarts counting and judgment when the number of errors, the number of consecutive errors, and the total number of times meet the preset thresholds; otherwise, it determines the cut-in probability based on the number of errors, the number of consecutive errors, and the total number of times.

[0026] The recording module records three types of lateral position information for this detection cycle, which will be used for comparison in the next detection cycle.

[0027] Furthermore, the synchronization module includes: acquiring the timestamps from the target information and lane line information; using the timestamp of the lane line information as the reference time; and using the timestamp, position, and speed information of the target information to calculate the target's position information at the reference time, thereby achieving time synchronization.

[0028] Furthermore, the output module includes:

[0029] Using the target information output by the Kalman filter fusion model as the reference target information, the target information output by the radar and camera is compared with the reference target information using Euclidean distance to obtain the correlation distance matrix between the reference target and the target output by the radar and camera. Then, the nearest target is associated by traversing the matrix.

[0030] Furthermore, the entry probability module includes:

[0031] Based on the correlation data between the number of errors, consecutive errors, and total errors and the entry probability, and using the correlation data and the number of errors and total errors in this detection cycle, the entry probability is obtained.

[0032] Furthermore, the cut-in probability module also includes a preset threshold unit: the threshold is set to error count > 5 or error count > 2 / 3 * total count.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] The present invention and its technical solution can accurately predict the entry scenario in advance, so that ADAS assisted driving functions can be controlled in advance, reducing the frequency of driver takeover and improving comfort. Attached Figure Description

[0035] Figure 1 This is a flowchart illustrating an embodiment of the present invention. Detailed Implementation

[0036] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0037] To make the purpose and features of this invention patent clearer and easier to understand, the specific embodiments of this invention patent will be further described below with reference to the accompanying drawings. It should be noted that the drawings are all in a very simplified form and use non-precise ratios, and are only used to facilitate and clearly illustrate the purpose of the embodiments of this invention patent.

[0038] The design concept of this invention is to use multiple sensors and a fusion model to identify and judge the vehicle's entry by using the lateral position information of the target vehicle detected by each of the multiple sensors and the fusion model.

[0039] The invention first involves installing a vehicle-mounted camera and a millimeter radar on the vehicle. The most effective installation locations are: the vehicle-mounted camera is installed on the windshield of the vehicle, and the millimeter radar is installed on the front bumper.

[0040] This invention is mainly based on three types of lateral position information: lateral position information detected by millimeter-wave radar, lateral position information detected by camera, and lateral position information calculated by Kalman filter fusion model.

[0041] Millimeter-wave radar detects the target vehicle's distance, speed, and heading angle. Based on the distance and heading angle, the lateral position can be calculated using the formula: Lateral position = Distance * sin(heading angle);

[0042] The camera can be a smart camera, which can detect lane lines and target vehicles, and output the lateral position information of the target vehicle and the vehicle itself.

[0043] The Kalman filter fusion model is mainly based on the ingress probability p; the calculated lateral position = p * lateral position detected by the millimeter-wave radar + (1-p) * lateral position detected by the camera. Then, the p value is updated according to a certain method, and the calculation for the next cycle is performed.

[0044] In this invention, an in-vehicle camera and a millimeter-wave radar are installed on the vehicle, and their output information is obtained through a CAN bus.

[0045] like Figure 1 The diagram shown illustrates the specific usage process:

[0046] After initialization, the three types of lateral position information of the target vehicle are set to zero for the first time;

[0047] Entering the next cycle, which becomes the current detection cycle, the output of the vehicle-mounted camera and millimeter radar wave are synchronized. The synchronization method is as follows: first, the timestamps in the target vehicle information and lane line information are obtained. Using the timestamp of the lane line information as the reference time, the position information of the target vehicle at the reference time is calculated using the timestamp of the target vehicle information and information such as position and speed, thereby achieving time synchronization.

[0048] This invention acquires three types of lateral position information of the target vehicle during the current detection period. This includes associating targets and outputting three lateral position information parameters. The target association method involves using the target information output by the Kalman filter fusion model as the benchmark information for comparison. The target information output by the radar and camera is compared with the benchmark information using Euclidean distance to obtain the association distance matrix between the benchmark target and the targets output by the radar and camera. Then, this matrix is ​​traversed to associate the nearest neighboring targets. The target information includes longitudinal position information (longitudinal distance), lateral position information (lateral distance), longitudinal velocity, and lateral velocity. This invention uses lateral position information for determining the entry point.

[0049] The system performs a judgment, comparing the lateral position information of the current detection cycle with that of the previous detection cycle to determine whether the current cycle represents an intrusion. If the absolute value of the target's lateral distance in the current detection cycle is less than or equal to that in the previous detection cycle (this is the case where the target is not detected), the system will determine whether the target is intruding. Figure 1 If condition 1) is met, then the current detection period is considered a cut-in, and the total number of times (i.e. Figure 1 The number of judgments in the test is incremented by 1, and the number of consecutive errors is set to zero; otherwise, the number of errors is incremented by 1, the total number of judgments is incremented by 1, and the number of consecutive errors is incremented by 1.

[0050] When the number of errors, the number of consecutive errors, and the total number of errors meet the conditions ( Figure 1 Condition 2): If the number of consecutive errors is greater than 5 or the number of errors is greater than 2 / 3 * the total number of times, then the number of errors, the number of consecutive errors, and the total number of times are all cleared to zero, and the counting and judgment are restarted. This is because the previous data cannot clearly determine whether it is a cut-in and will interfere with the judgment of the subsequent cycle, so it must be cleared to zero.

[0051] If condition 2 is not met, the entry probability is determined based on the number of errors, the number of consecutive errors, and the total number of attempts. The method for determining the entry probability based on the number of errors and the total number of attempts can be by looking up a table. The table is obtained by training and statistically analyzing the corresponding test data based on road test data and the method of this invention. The basic contents of the table include the number of errors, the number of consecutive errors, the total number of attempts, and the entry probability.

[0052] Record the three types of lateral position information for this period, for comparison in the next period.

[0053] By adopting the above method and after testing and verification, it is possible to accurately predict the entry scenario at least 0.5 seconds in advance, so that ADAS assisted driving function can be controlled in advance, reducing the frequency of driver takeover and improving comfort.

[0054] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for identifying vehicles cutting into adjacent lanes based on multi-sensor fusion, characterized in that, include: S1. Obtain the output information from the vehicle-mounted camera and millimeter-wave radar installed on the vehicle; S2. Based on the output information obtained in step S1, synchronize the target information and lane line information in time. S3. Using the Kalman filter fusion model, the target is correlated and three types of lateral position information of the target are output, including the lateral position information detected by the millimeter-wave radar, the lateral position information detected by the camera, and the lateral position information calculated by the Kalman filter fusion model. S4. Compare the lateral position information of this detection cycle with the lateral position information of the previous detection cycle to determine whether this detection cycle is an entry point; if not, increment the error count by 1, increment the consecutive error count by 1, and increment the total count by 1. S5. When the number of errors, the number of consecutive errors, and the total number of errors meet the preset thresholds, all are reset to zero, and the counting and judgment are restarted. Otherwise, the probability of entry is determined based on the number of errors, the number of consecutive errors, and the total number of attempts. S6. Record the three types of lateral position information for this detection cycle, for comparison in the next detection cycle.

2. The method for identifying vehicles cutting into adjacent lanes based on multi-sensor fusion according to claim 1, characterized in that, The time synchronization method described in step S2 includes: obtaining the timestamps from the target information and lane line information; using the timestamp of the lane line information as the reference time; and using the timestamp, position, and speed information of the target information to calculate the target's position information at the reference time, thereby achieving time synchronization.

3. The method for identifying vehicles cutting into adjacent lanes based on multi-sensor fusion according to claim 1, characterized in that, The method for associating targets using the Kalman filter fusion model described in step S3 includes: Using the target information output by the Kalman filter fusion model as the reference target information, the target information output by the radar and camera is compared with the reference target information using Euclidean distance to obtain the correlation distance matrix between the reference target and the target output by the radar and camera. Then, the nearest target is associated by traversing the matrix.

4. The method for identifying vehicles cutting into adjacent lanes based on multi-sensor fusion according to claim 1, characterized in that, The method for determining the entry probability based on the number of errors and the total number of attempts in step S5 includes: Based on the correlation data between the number of errors, consecutive errors, and total errors and the entry probability, and using the correlation data and the number of errors and total errors in this detection cycle, the entry probability is obtained.

5. The method for identifying vehicles cutting into adjacent lanes based on multi-sensor fusion according to claim 1, characterized in that, The preset thresholds mentioned in step S5 include: consecutive error count > 5 or error count > 2 / 3 * total count.

6. A vehicle lane cutting-in recognition device based on multi-sensor fusion, characterized in that, include: The acquisition module is used to acquire the output information of the vehicle-mounted camera and millimeter-wave radar installed on the vehicle; The synchronization module synchronizes the target information and lane line information in time based on the output information obtained by the acquisition module. The output module uses a Kalman filter fusion model to associate targets and output three types of lateral position information of the targets, including lateral position information detected by millimeter-wave radar, lateral position information detected by camera, and lateral position information calculated by Kalman filter fusion model. The comparison module compares the lateral position information of the current detection cycle with the lateral position information of the previous detection cycle to determine whether the current detection cycle is an entry point; if not, the error count is incremented by 1, the consecutive error count is incremented by 1, and the total count is incremented by 1. When the probability module is activated, if the number of errors, the number of consecutive errors, and the total number of errors meet the preset thresholds, all values ​​are reset to zero, and the counting and judgment process restarts. Otherwise, the probability of entry is determined based on the number of errors, the number of consecutive errors, and the total number of attempts. The recording module records three types of lateral position information for this detection cycle, which will be used for comparison in the next detection cycle.

7. The lane-changing vehicle cut-in recognition device based on multi-sensor fusion according to claim 6, characterized in that, The synchronization module includes: acquiring timestamps from target information and lane line information; using the timestamp of the lane line information as a reference time; and using the timestamp, position, and speed information of the target information to calculate the target's position information at the reference time, thereby achieving time synchronization.

8. The lane-changing vehicle cut-in recognition device based on multi-sensor fusion according to claim 6, characterized in that, The output module includes: Using the target information output by the Kalman filter fusion model as the reference target information, the target information output by the radar and camera is compared with the reference target information using Euclidean distance to obtain the correlation distance matrix between the reference target and the target output by the radar and camera. Then, the nearest target is associated by traversing the matrix.

9. The lane-changing vehicle cut-in recognition device based on multi-sensor fusion according to claim 6, characterized in that, The cut-in probability module includes: Based on the correlation data between the number of errors, consecutive errors, and total errors and the entry probability, and using the correlation data and the number of errors and total errors in this detection cycle, the entry probability is obtained.

10. The lane-changing vehicle cut-in recognition device based on multi-sensor fusion according to claim 6, characterized in that, The cut-in probability module also includes a preset threshold unit: the threshold is set to either error count > 5 or error count > 2 / 3 * total count.

Images