Blind zone collision early warning method fusing vehicle-mounted camera and V2X

Abstract

The invention relates to the technical field of intelligent network connection automobiles, in particular to a blind area collision early warning method fusing a vehicle-mounted camera and V2X. The blind area collision early warning method comprises the steps that target matching is carried out based on a vehicle target list and a V2X target list; adjusting the confidence of the V2X target data; comparing matched target data in the V2X target list with matched target data in the own vehicle target list, and judging whether a target possibly conflicting with the own vehicle exists in the blind area or not; self-vehicle trajectory estimation and target trajectory estimation are carried out; predicting the motion intention of the target; and track comparison is carried out by combining the self-vehicle track, the target track and the target motion intention so as to evaluate the collision risk. Through fusion of V2X and a vehicle-mounted camera, blind area collision early warning is realized, misjudgment caused by ghosting, stains and the like of a vehicle vision sensor or missed judgment caused by related algorithms is avoided, misjudgment and missed judgment caused by blind areas of radar and the like and insufficient sensing distance are avoided, and the blind area collision early warning method is high in accuracy and high in reliability. And the influence on function performance caused by packet loss and packet damage due to poor communication quality when only V2X communication is carried out is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent connected vehicles, and in particular to a blind spot collision warning method integrating a vehicle-mounted camera and V2X. Background Art

[0002] At present, V2X vehicle-road collaboration is a technological trend in the development of automobile intelligence, and blind spot collision warning has always been a focus of automobile research and development. The mainstream blind spot collision warning technology in the current market that relies only on single-vehicle intelligence or only on V2X functions has some performance defects. The method based on the monocular camera cannot generate reliable visual depth information, and the image of the monocular camera often cannot effectively judge the distance. Such visual perception results cannot detect the exact position of the blind spot target. The calculation method based on the single-vehicle intelligence is limited by the vehicle's computing performance and algorithm capabilities, and the detection effect is often unsatisfactory, and the probability of missed or wrong detection of targets is high. At the same time, such algorithms have poor recognition capabilities for the target's motion trajectory, making it difficult to provide accurate collision warnings. The method based on the vehicle's sensor may have occlusion blind spots, or a small field of view, or an imperfect visual algorithm. At the same time, such methods have limited perception distances and cannot effectively obtain a large amount of reliable information about the vehicle's surroundings. Collision warning based on the V2X function is also unreliable due to the inability to guarantee V2I coverage on the road, the inability to ensure that every vehicle in the scene can be connected, and problems such as network connection quality.

[0003] In view of the above problems, the inventors of the present invention finally obtained the present invention after a long period of research and practice. Summary of the invention

[0004] In order to solve at least one aspect of the above technical problems, the present invention provides a blind spot collision warning method that integrates a vehicle-mounted camera and V2X, which can effectively detect and verify targets in the vehicle's blind spot, detect the relative distance and its motion trajectory, calculate the motion trajectory through the motion states of the vehicle and the target, compare the two motion trajectories and predict the target's intention, and realize collision warning between the vehicle and the target.

[0005] The technical solution adopted by the present invention is:

[0006] A blind spot collision warning method integrating a vehicle-mounted camera and V2X is provided, comprising the following steps:

[0007] Step 1: Collect blind spot video data in real time through the camera on the vehicle;

[0008] Step 2: Perform online target recognition based on the blind spot video data to form a self-vehicle target list;

[0009] Step 3: The vehicle receives standardized V2X basic safety messages through the V2X module to form a V2X target list;

[0010] Step 4: Perform target matching based on the ego vehicle target list and the V2X target list; adjust the confidence of the V2X target data;

[0011] Step 5: Compare the target data in the V2X target list and the target list of the ego vehicle to determine whether there is a target in the blind spot that may conflict with the ego vehicle; if there is a target that may conflict with the ego vehicle, execute step 6;

[0012] Step 6: Estimating the trajectory of the vehicle and the target.

[0013] Step 7: Predict the target's movement intention;

[0014] Step 8: Compare the ego vehicle trajectory, target trajectory, and target motion intention to assess collision risk.

[0015] Furthermore, in step 1, the number of cameras on the vehicle is greater than or equal to 3, and the video streams of multiple cameras are time-aligned through data timestamps.

[0016] Furthermore, in step 2, the self-vehicle target list includes the position, size, type, relative speed information, timestamp information and self-vehicle posture information of the target.

[0017] Furthermore, in step 3, the V2X target list includes the positioning, posture, dynamic information, timestamp information and confidence information of the target.

[0018] Furthermore, in step 4, the adjusting the confidence of the V2X target data specifically includes:

[0019] A time threshold T is set based on the frequency and time of receiving V2X messages, and the time difference Δt of two source images from two lists of the target to be matched is obtained. A dynamic coefficient k1 related to the current speed of the vehicle is set, and a confidence adjustment coefficient k2 is set. k2 should be a function of (Δt, k1, T). The dynamic adjustment of the confidence is achieved through k2.

[0020] Furthermore, in step 5, the specific method for determining whether there is a target in the blind spot that may conflict with the vehicle is as follows:

[0021] ① If there is no target in the V2X target list and the vehicle target list, it is determined that there is no target with which there may be a conflict, the frame ends, and the next frame is judged;

[0022] ② If there is a target in one of the V2X target list and the ego vehicle target list or both of them, first convert the identified target location into a location consistent with the ego vehicle reference system based on the ego vehicle posture in the ego vehicle target list, set a positioning difference threshold, and form a fused target list based on the positioning difference threshold and the target location, size, and type; preset a confidence difference threshold for the two target lists, and select the target data to be used based on the fused target list.

[0023] Furthermore, in step 5, the target data to be adopted is selected based on the fused target list, specifically including:

[0024] A. When the target data perceived by the ego vehicle is consistent with the target data perceived by V2X, the target data is used;

[0025] B. When V2X senses the presence of a target at a location where the ego vehicle senses the absence of a target;

[0026] Query the ego vehicle target list to find out whether there is an obstruction between the ego vehicle position and the target position. If there is an obstruction, use the target data from the V2X target list; if there is no obstruction, use the target data from the V2X target list and reduce the confidence of the data;

[0027] C. When the target state perceived by the ego vehicle is inconsistent with the target state perceived by V2X, the target data in the list with high confidence is used;

[0028] D. When V2X senses that there is no target at a location where the ego vehicle senses that there is a target; use the target data from the ego vehicle's target list and reduce the confidence of the data.

[0029] Furthermore, in step 6,

[0030] Ego vehicle trajectory estimation: using a filtering algorithm based on a dynamic model, based on the ego vehicle position information obtained by the ego vehicle system;

[0031] Target trajectory estimation: Use the target's historical trajectory and real-time pose data, combined with the target data confidence, and fit through a filtering algorithm.

[0032] Furthermore, in step 7, the prediction of the target's movement intention includes predicting the target's movement direction by combining the road surface information and the target information.

[0033] Furthermore, in step 8, if there is a risk of collision, a warning is given on the human-computer interaction interface; if there is no risk of collision, the driver is only reminded to a minimum that there is a target in the blind spot.

[0034] Compared with the prior art, the beneficial effects of the present invention are:

[0035] The present invention is based on the fusion of V2X functions and car cameras to perform blind spot collision warnings, effectively improving the problems of missed detection and false detection that only rely on vehicle-mounted sensors. The trajectory can be predicted based on the current state of the target and the environment it is in. Through V2X messages, the vehicle can obtain information such as the target's position and posture, and can better infer the target's intentions. By dynamically adjusting the information confidence, the received perception data is judged and added to the input data of the blind spot collision warning. The present invention implements blind spot collision warnings by integrating V2X and vehicle-mounted cameras, avoiding misjudgments caused by ghosts, stains, etc. of vehicle visual sensors or missed judgments caused by related algorithms, avoiding misjudgments and missed judgments caused by blind spots and insufficient perception distances of radars, etc., and avoiding packet loss and bad packets caused by poor communication quality when there is only V2X communication, thereby affecting functional performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 A schematic diagram showing the arrangement of vehicle sensors and a V2X unit according to an embodiment of the present invention is shown;

[0038] Figure 2 A flow chart showing a blind spot collision warning method according to an embodiment of the present invention;

[0039] Figure 3 A classification diagram of application logic scenarios of a blind spot collision warning method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0040] The above and other technical features and advantages of the present invention are described in more detail below in conjunction with the accompanying drawings.

[0041] As used herein, the term "including" and its variations mean open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "based at least in part on". The terms "an example embodiment" and "an embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". Other explicit and implicit definitions may also be included below.

[0042] like Figure 1 As shown, Figure 1The layout of vehicle cameras is shown by way of example. To implement this method, sufficient cameras should be arranged to cover the blind spot behind the vehicle. There can be 3 or more cameras, and the PC5 interface should be supported to realize V2X communication.

[0043] like Figure 2 As shown, the blind spot collision warning method integrating the vehicle-mounted camera and V2X provided in the embodiment of the present application is implemented according to the following process:

[0044] 1. Multiple cameras on the vehicle collect blind spot video data in real time, and multiple video streams are time-aligned through data timestamps.

[0045] 2. The vehicle performs online target recognition through an algorithm. The algorithm can adopt a deep learning algorithm with better effects to form a list of self-vehicle targets. The list contains information such as the location, size, type, relative speed, etc. of the target. At the same time, the list needs to include a timestamp and the current position obtained from the inside of the self-vehicle.

[0046] 3. The vehicle receives standardized V2X basic safety messages (BSM, PSM messages, hereinafter referred to as BSM) from the roadside, other vehicles or vulnerable traffic participants through the V2X module through the PC5 direct connection interface to form a V2X target list, which includes the target's positioning, posture, dynamic information, timestamp and related confidence information.

[0047] 4. To perform target matching, set a time threshold T, which should be based on the frequency and time of receiving V2X messages. The recommended value is 80 milliseconds. Obtain Δt = t(self-vehicle) - t(V2X) of the two source images that need to match the target. Set the dynamic coefficient k1: k1 should be corrected according to the current speed of the self-vehicle. Preferably, the correction function is a cubic or higher polynomial function related to the driving speed; set the confidence adjustment coefficient k2 to adjust the confidence of the V2X target data. k2 should be a function of (Δt, k1, T). It is recommended to be a function of 1-Δt / (k1*T) to achieve the purpose of dynamically adjusting the confidence:

[0048] If Δt is less than k1*T, 1-Δt / k1*T is a positive value, and k2 increases the confidence of the target data.

[0049] If Δt is greater than or equal to k1*T, 1-Δt / k1*T is a negative value, and k2 reduces the confidence of the target data.

[0050] 5. Compare the V2X target list and the vehicle target list to determine whether there are any nearby targets that may conflict with the vehicle. The determination method is as follows:

[0051] a) If there is no target in either the V2X target list or the vehicle target list, it is determined that there is no target with which a conflict may occur, and the frame ends, and the next frame is judged.

[0052] b) If there is a target in one of the V2X target list and the ego vehicle target list or both of them, target matching is performed: first, based on the ego vehicle posture in the ego vehicle target list, the identified target positioning is converted into a positioning consistent with the ego vehicle reference system, and a positioning difference threshold is set. Based on the threshold and the target positioning, size, and type, the contents of the two target lists are matched through a filtering algorithm to form a fused target list.

[0053] c) Fusion target list The scenarios where targets exist can be roughly divided into Figure 3 Scenario 1: The possible targets of collision in the blind spot perceived by visual perception and V2X perception are consistent; Scenario 2: In visual perception, the possible targets of collision in the blind spot are blocked, so there are no possible targets in the blind spot perceived by visual perception, while V2X perception detects the possible targets in the blind spot; Scenario 3: V2X perception detects the possible targets of collision in the blind spot, while the visual perception of the ego vehicle detects no targets and no obstructions; Scenario 4: Both visual perception and V2X perception detect possible targets in the blind spot, but the target data are inconsistent, such as different positions; Scenario 5: There is no V2X perception target at the location where the visual perception target exists.

[0054] The above five scenarios are divided into four categories. The subsequent data processing strategies for each type are shown in the following table: It is necessary to preset equal confidence levels and a confidence difference threshold for the two target lists to determine which data should be discarded when there is a large difference between the two target lists:

[0055]

[0056] 6. Perform trajectory estimation.

[0057] a) The ego-vehicle trajectory estimation can use a filtering algorithm based on the dynamic model, with the input being the ego-vehicle position information obtained by the ego-vehicle system.

[0058] b) Target trajectory estimation can use the target's historical trajectory and real-time posture, combined with the target data confidence, and fit through a filtering algorithm.

[0059] 7. Estimate the target intention. Combined with road information such as lane width, number of vehicles, lane direction, and target information such as target type, the target's movement intention is predicted. Models can be established for different types of vehicles and other traffic participants to derive the possibility of the target moving in a certain direction under certain conditions.

[0060] 8. Evaluate collision risk. Combine the vehicle's trajectory, target trajectory and intention to compare trajectories. If there is a risk of collision according to the current trajectory, the vehicle will give appropriate sound, light and vibration reminders on the human-machine interface. If no collision risk is found, the operator will only be reminded of the presence of a target in the blind spot to a minimum extent.

[0061] The above are only preferred embodiments of the present invention, which are only illustrative and not restrictive. The structures and connection modes of the components in the present invention can be changed, and any equivalent transformations and improvements based on the technical solution of the present invention should not be excluded from the protection scope of the present invention.

Claims

1. A blind spot collision warning method integrating a vehicle-mounted camera and V2X, characterized in that: The steps include: Step 1: Collect blind spot video data in real time through the camera on the vehicle; Step 2: Perform online target recognition based on the blind spot video data to form a self-vehicle target list; Step 3: The vehicle receives standardized V2X basic safety messages through the V2X module to form a V2X target list; Step 4: Perform target matching based on the ego vehicle target list and the V2X target list; Adjust the confidence level of V2X target data; Step 5: Compare the target data in the V2X target list and the target list of the ego vehicle to determine whether there is a target in the blind spot that may conflict with the ego vehicle; if there is a target that may conflict with the ego vehicle, execute step 6; Step 6: Estimating the trajectory of the vehicle and the target. Step 7: Predict the target's movement intention; Step 8: Compare the ego vehicle trajectory, target trajectory, and target motion intention to assess collision risk.

2. A blind spot collision warning method integrating a vehicle-mounted camera and V2X as claimed in claim 1, characterized in that: In step 1, the number of cameras on the vehicle is greater than or equal to 3, and the video streams of multiple cameras are time-aligned through data timestamps.

3. A blind spot collision warning method integrating a vehicle-mounted camera and V2X as claimed in claim 1, characterized in that: In step 2, the ego vehicle target list includes the position, size, type, relative speed information, timestamp information and ego vehicle posture information of the target.

4. A blind spot collision warning method integrating a vehicle-mounted camera and V2X as claimed in claim 1, characterized in that: In step 3, the V2X target list includes the target's location, posture, dynamic information, timestamp information, and confidence information.

5. The blind spot collision warning method integrating a vehicle-mounted camera and V2X as claimed in claim 1, characterized in that: In step 4, the adjustment of the confidence of the V2X target data specifically includes: A time threshold T is set based on the frequency and time of receiving V2X messages, and the time difference Δt of two source images from two lists of the target to be matched is obtained. A dynamic coefficient k1 related to the current speed of the vehicle is set, and a confidence adjustment coefficient k2 is set. k2 should be a function of (Δt, k1, T). The dynamic adjustment of the confidence is achieved through k2.

6. A blind spot collision warning method integrating a vehicle-mounted camera and V2X as claimed in claim 5, characterized in that: In step 5, the specific method for determining whether there is a target in the blind spot that may conflict with the vehicle is as follows: ① If there is no target in the V2X target list and the vehicle target list, it is determined that there is no target with which there may be a conflict, the frame ends, and the next frame is judged; ② If there is a target in one of the V2X target list and the ego vehicle target list or both, first convert the identified target location into a location consistent with the ego vehicle reference system based on the ego vehicle posture in the ego vehicle target list, set the location difference threshold, and form a fused target list based on the location difference threshold and the location, size, and type of the target; A confidence difference threshold is preset for the two target lists, and the target data to be adopted is selected based on the fused target list.

7. A blind spot collision warning method integrating a vehicle-mounted camera and V2X as claimed in claim 6, characterized in that: In step 5, the target data to be used is selected based on the fused target list, specifically including: A. When the target data perceived by the ego vehicle is consistent with the target data perceived by V2X, the target data is used; B. When V2X senses the presence of a target at a location where the ego vehicle senses the absence of a target; Query the ego vehicle target list to find out whether there is an obstruction between the ego vehicle position and the target position. If there is an obstruction, use the target data from the V2X target list; if there is no obstruction, use the target data from the V2X target list and reduce the confidence of the data; C. When the target state perceived by the ego vehicle is inconsistent with the target state perceived by V2X, the target data in the list with high confidence is used; D. When V2X senses that there is no target at a location where the ego vehicle senses that there is a target; use the target data from the ego vehicle's target list and reduce the confidence of the data.

8. The blind spot collision warning method integrating a vehicle-mounted camera and V2X as claimed in claim 1, characterized in that: In step 6, Ego vehicle trajectory estimation: using a filtering algorithm based on a dynamic model, based on the ego vehicle position information obtained by the ego vehicle system; Target trajectory estimation: Use the target's historical trajectory and real-time pose data, combined with the target data confidence, and fit through a filtering algorithm.

9. The blind spot collision warning method integrating a vehicle-mounted camera and V2X as claimed in claim 1, characterized in that: In step 7, the prediction of the target's movement intention includes predicting the target's movement direction by combining road surface information and target information.

10. The blind spot collision warning method integrating a vehicle-mounted camera and V2X as claimed in claim 1, characterized in that: In step 8, if there is a risk of collision, a warning will be given on the human-computer interaction interface; if there is no risk of collision, the driver will only be reminded to a minimum that there is a target in the blind spot.

Images