A lane merging recognition method, system and vehicle

Abstract

This invention discloses a lane merging recognition method, system, and vehicle. The method includes the following steps: S1, determining the lane width ahead at time t+1 based on the images of the lane ahead captured by a monocular camera at time t and time t+1; S2, using the lane width ahead for lane merging detection and issuing a warning. This achieves lane merging prediction based on a monocular camera, enabling vehicles to follow the merging lane guide and merge into the main lane, improving driving safety.

Description

Technical Field

[0001] This invention belongs to the field of vehicle driver assistance technology, and more specifically, this invention relates to a lane change recognition method, system and vehicle. Background Technology

[0002] Current vehicles use monocular cameras to identify lane lines. However, the identified lane lines are subject to random and steady-state errors. Without high-precision navigation and positioning, relying solely on lane line curve equations cannot accurately determine if a lane change is imminent. This inability to accurately predict lane changes introduces uncertainty and safety hazards into assisted driving systems. Examples include the risk of the system suddenly disengaging during a lane change, the vehicle veering out of its lane, colliding with guardrails, or hitting vehicles in adjacent lanes. Summary of the Invention

[0003] This invention provides a lane merging recognition method, which aims to improve the lane merging recognition accuracy of a monocular camera.

[0004] This invention is implemented as follows: a lane merging recognition method, the method specifically includes the following steps:

[0005] S1, images of the lane ahead captured by a monocular camera at times t and t+1. t Determine the width of the lane ahead of your vehicle;

[0006] S2. Use the lane width ahead for lane merging detection and issue a warning.

[0007] Furthermore, the method for determining the width of the lane ahead of the vehicle includes the following steps:

[0008] S11, Move the image p in front t The lane where the vehicle is located is divided into n lane segments along the direction of vehicle travel;

[0009] S12. Calculate the average lane width W of each lane segment in the n lane segments. t [n] is used to predict the average lane width of each lane segment at time t+1.

[0010] S13. Fit the lane line curves on both sides of the vehicle, and calculate the lane width z of each of the n lane segments at time t+1. t+1 [n];

[0011] S14, Predicted average lane width and lane width value z t+1 [n] are merged to form the lane width W of each lane segment at time t+1 in the n lane segments. t+1[n].

[0012] Furthermore, the lane width W of each lane segment at time t+1 t+1 The specific formula for calculating [n] is as follows:

[0013]

[0014] Where K[n] is the confidence level, Let R[n] be the variance of the lane width of n lane segments at time t+1, and R[n] be the measurement error.

[0015] Furthermore, the average lane width at time t+1 The specific calculation formula is as follows:

[0016]

[0017] Furthermore, the variance of the lane width of the n lane segments at time t+1 The specific calculation formula is as follows:

[0018]

[0019] Where Q is the process noise variance, and the value of the process noise variance Q is related to the vehicle speed v at time t. ego yaw rate ψ ego Related.

[0020] Furthermore, lane merging detection specifically includes the following steps:

[0021] S21. Detect whether the vehicle is going straight and whether the angle between the vehicle and the lane is less than the angle threshold. If the detection result is yes, proceed to step S22.

[0022] S22. At time t+1, obtain the target vehicle in front of the vehicle in the monocular camera image and detect whether there is a target vehicle within a set distance in front of the vehicle. If the detection result is no, proceed to step S24.

[0023] S24. Check whether the vehicle in front and its lane meet the following conditions. If the detection result is yes, issue a lane change reminder.

[0024] 241) The confidence level of the left and right lane lines of the lane where this vehicle is located is greater than the confidence threshold.

[0025] 242) The shorter lane lines on both sides of the lane where this vehicle is located have very small curvature, while the longer lane lines have large curvature.

[0026] 243) The two lane lines intersect;

[0027] 244) The lane width of the lane segment where this vehicle is located shows a decreasing trend.

[0028] Furthermore, if a target vehicle is located within a set distance in front of the vehicle, the system checks whether the vehicle in front and its lane meet the following conditions. If the detection result is yes, a lane change reminder is issued.

[0029] 231) The car in front is changing lanes;

[0030] 232) The confidence level of the left and right lane lines of the lane where this vehicle is located is greater than the confidence threshold;

[0031] 233) The short lane lines on both sides of the lane where this vehicle is located have very small curvature, while the long lane lines have large curvature.

[0032] Furthermore, while identifying the lane width of the lane where the vehicle is located, the lane widths of the lanes on the left and right sides of the lane are also identified. When the vehicle is detected to be changing lanes, the lane width of the lane where the vehicle is located is updated to the lane width of the lane where the vehicle is located after the lane change.

[0033] This invention is implemented as follows: a lane merging recognition system, the system comprising:

[0034] A monocular camera captures images of the front of the vehicle and sends them to the processor.

[0035] The processor performs lane merging detection based on the lane merging recognition method described above.

[0036] The present invention is implemented as follows: a car that integrates the above-mentioned lane change recognition system.

[0037] This invention enables lane merging prediction based on a monocular camera, allowing vehicles to follow the lane merging line and merge into the main lane, thus improving driving safety. Attached Figure Description

[0038] Figure 1 This is a flowchart of the lane merging recognition method provided in an embodiment of the present invention. Detailed Implementation

[0039] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings, so as to help those skilled in the art to have a more complete, accurate and in-depth understanding of the inventive concept and technical solution of the present invention.

[0040] Figure 1 The flowchart of the lane merging recognition method provided in the embodiment of the present invention includes the following steps:

[0041] Part 1: Based on the images of the lane ahead captured by the monocular camera at time t and t+1, determine the lane width of the lane where the vehicle is located at time t+1. Part 2: Use the lane width ahead for lane merging detection and issue a warning. The following is an explanation of these two parts.

[0042] (a) Determine the changes in lane width ahead of the vehicle in the lane it is currently in;

[0043] S11. Divide the lane in the images of the front captured by the monocular camera at time t (the previous time) and time t+1 (the current time) into n lane segments along the vehicle's driving direction, and extract the n lane segments of the lane where the vehicle is located at time t.

[0044] In this embodiment of the invention, the lane line image is divided into equal segments along the vehicle's driving direction. A vehicle coordinate system is defined with the midpoint of the rear axle of the vehicle as the origin, the vehicle's driving direction as the x-axis, and the vehicle's width direction as the y-axis. The average width between two lane lines in a lane segment is the lane width of the corresponding lane segment.

[0045] S12. Calculate the average lane width W of each lane segment in the n lane segments. t [n] and variance P t [n], while based on the average lane width W at time t. t [n] and variance P t [n] is used to predict the average lane width of each of the n lane segments at time t+1. and variance

[0046] Since the time interval between adjacent times (t, t+1) is very short, the mean lane width changes very little between adjacent times. Based on this, the prediction method is as follows:

[0047]

[0048] Where Q is the process noise variance, and its value depends on the vehicle speed v at time t. ego yaw rate ψ ego Related, when ψ ego >0.1, Q=(0.01736×V ego ×dt) 2 +0.0003, when ψ ego ≤0.1, Q=(0.01736×V ego ×dt) 2+0.0008, where the maximum possible change in lane width for every meter the vehicle moves forward is 0.01736, and 0.0003 and 0.0008 are steady-state errors, which exist even when the vehicle is stationary. When ψ ego As the variance increases, the stability of visual recognition decreases. Therefore, by changing the steady-state component of the process noise variance, the ability of the filter to filter noise can be increased.

[0049] S13. Fit the lane line curves of the lane lines on both sides of the vehicle at time t, and read the x-coordinates of the midpoints of all lane lines in the vehicle coordinate system at time t+1. t+1 [n], calculate the lane width z at the midpoint of each lane segment at time t+1. t+1 [n];

[0050] In this embodiment of the invention, the method for fitting the lane line curve is as follows:

[0051] Extract lane line segments from both sides of the vehicle at time t, obtain the coordinates of all lane line segments in the vehicle coordinate system, and then fit the lane line curve y of both sides of the vehicle. t [n];

[0052] Lane line curve y t [n] represents the lane line curve. Lane curve Lane curve Lane curve All are polynomial curve equations, lane line curves lane line curve The fitting method is the same, using lane line curves For example, the specific expression is as follows:

[0053]

[0054] in, Let x represent the x-coordinate of the midpoint of the nth lane segment to the left at time t, where x represents the distance from the origin of the vehicle coordinate system along the direction of lane extension. Let Ct represent the ordinate of the midpoint of the nth left lane segment at time t. The ordinate represents the distance from the origin of the vehicle coordinate system in the lane width direction. C0, C1, C2, and C3 represent the fitting coefficients. It should be noted that at the same time...

[0055] In this embodiment of the invention, the lane width z t+1 The specific method for obtaining [n] is as follows:

[0056] The x-coordinates of each lane segment in the vehicle coordinate system at time t+1 are read. t+1 [n] Input the lane line curves respectively Lane curve This allows us to obtain the longitudinal coordinates of the lane segments on both sides of each lane at time t+1. So

[0057] Due to measurement errors in lane line recognition, the measurement error... Where, δ n This represents the measurement variance of the nth lane segment, which is related to distance.

[0058] S4. Predicted mean lane width of each lane segment in n lane segments at time t+1. and lane width calculation value z t+1 [n] determines the average width W of each lane segment. t+1 [n] represents the lane width of each lane segment at time t+1.

[0059]

[0060] Where K[n] is the confidence level, and its value ranges from 0 to 1.

[0061] This invention filters the lane width information. Before processing, the lane width recognition error, based on the camera parameters, is: 0.005 + 0.00034738 * x^2 (where 0.005 is the C0 error and 0.00034738 is the C2 error), meaning the error over 50 meters is 0.8734 meters. After Kalman filtering, using the variance formula: P(t+1) = ((P(t) + Q) * R) / (P(t) + Q + R), where Q is the process noise variance and R is the measurement error, the error over 50 meters after iterative convergence is 0.1673 meters. Therefore, filtering significantly improves the reliability of lane merging detection.

[0062] (ii) Detection methods for lane merging;

[0063] S21. Before performing lane merging judgment, it is necessary to first detect whether the vehicle is going straight and whether the angle between the vehicle and the lane is less than the angle threshold. When the angle between the vehicle and the lane is too large, the image will be distorted and false alarms will be easily generated. If the detection result is yes, then step S22 is executed. If the detection result is no, then lane merging detection is not performed, that is, step S22 is not executed.

[0064] S22. At time t+1, obtain the target vehicle in front of the vehicle in the monocular camera image and detect whether the target vehicle is within a set distance in front of the vehicle. If the detection result is yes, the vehicle in front is obstructed, and step S23 is executed. If the detection result is no, there is no obstruction in front, and step S24 is executed.

[0065] S23. Check whether the vehicle in front and its lane meet the following conditions. If the detection result is yes, issue a lane change reminder.

[0066] 231) The vehicle in front is changing lanes: the curvature of the vehicle in front's trajectory and the angle between the heading speed direction and the lane line are too large, the vehicle in front's turn signal is on, or the vehicle in front's lateral position deviates from the lane.

[0067] 232) The confidence level of the left and right lane lines of the lane where this vehicle is located is greater than the confidence threshold;

[0068] 233) The curvature of the shorter lane line on both sides of the lane where this vehicle is located is very small, for example, less than 0.0005 (unit 1 / m), while the curvature of the longer lane line is larger, for example, greater than 0.004 (unit 1 / m).

[0069] S24. Check whether the vehicle in front and its lane meet the following conditions. If the detection result is yes, issue a lane change reminder.

[0070] 241) The confidence level of the left and right lane lines of the lane where this vehicle is located is greater than the confidence threshold.

[0071] 242) The curvature of the shorter lane line on both sides of the lane where this vehicle is located is very small, for example, less than 0.0005 (unit 1 / m), while the curvature of the longer lane line is larger, for example, greater than 0.004 (unit 1 / m);

[0072] 243) The two lane lines intersect;

[0073] 244) The lane width of the lane segment where this vehicle is located shows a decreasing trend.

[0074] In this embodiment of the invention, in order to more closely resemble the actual driving process, the vehicle needs to change lanes during driving. In order to ensure that the vehicle can still achieve lane merging recognition after changing lanes, the lane width of the lane where the vehicle is located is identified at the same time as the lane width of the lane on the left and right sides of the lane where the vehicle is located. The identification method is the same as the identification method of the lane where the vehicle is located. When the vehicle is detected to be changing lanes, the lane width of the lane where the vehicle is located is updated to the lane width of the lane where the vehicle is located after the lane change.

[0075] The present invention also provides a lane merging recognition system, the system comprising:

[0076] A monocular camera captures images of the front of the vehicle and sends them to a processor, which then performs lane change detection based on the lane change recognition method described above.

[0077] The present invention also provides a car that integrates the above-mentioned lane change recognition system, wherein the monocular camera is integrated into the windshield, and the processor is separately located in the vehicle or integrated into the vehicle's overall controller.

[0078] The present invention has been described by way of example. Obviously, the specific implementation of the present invention is not limited to the above-described manner. Any non-substantial improvements made using the inventive concept and technical solution of the present invention, or the direct application of the inventive concept and technical solution of the present invention to other occasions without modification, are all within the protection scope of the present invention.

Claims

1. A lane merging recognition method, characterized in that, The method specifically includes the following steps: S1. Based on the images of the lane ahead captured by the monocular camera at time t and time t+1, determine the lane width of the lane where the vehicle is located at time t+1. S2. Use the lane width ahead for lane merging detection and issue a warning; Lane merging detection specifically includes the following steps: S21. Detect whether the vehicle is going straight and whether the angle between the vehicle and the lane is less than the angle threshold. If the detection result is yes, proceed to step S22. S22. At time t+1, obtain the target vehicle in front of the vehicle in the monocular camera image and detect whether there is a target vehicle within a set distance in front of the vehicle. If the detection result is no, proceed to step S24. S24. Check whether the vehicle in front and its lane meet the following conditions. If the detection result is yes, issue a lane change reminder. 241) The confidence level of the left and right lane lines of the lane where this vehicle is located is greater than the confidence threshold. 242) The shorter lane lines on both sides of the lane where this vehicle is located have very small curvature, while the longer lane lines have large curvature. 243) The two lane lines intersect; 244) The lane width of the lane segment where this vehicle is located shows a decreasing trend.

2. The lane merging recognition method as described in claim 1, characterized in that, The method for determining the width of the lane ahead of the vehicle includes the following steps: S11. Divide the lane where the vehicle is located in the image ahead into n lane segments along the direction of vehicle travel. S12. Calculate the average lane width W of each lane segment in the n lane segments. t [n] is used to predict the average lane width of each lane segment at time t+1. S13. Fit the lane line curves on both sides of the vehicle, and calculate the lane width z of each of the n lane segments at time t+1. t+1 [n]; S14, Predicted average lane width and lane width value z t+1 [n] are merged to form the lane width W of each lane segment at time t+1 in the n lane segments. t+1 [n].

3. The lane merging recognition method as described in claim 2, characterized in that, Lane width W of each lane segment at time t+1 t+1 The specific formula for calculating [n] is as follows: Where K[n] is the confidence level, Let R[n] be the variance of the lane width of n lane segments at time t+1, and R[n] be the measurement error.

4. The lane merging recognition method as described in claim 2 or 3, characterized in that, Average lane width at time t+1 The specific calculation formula is as follows:

5. The lane merging recognition method as described in claim 3, characterized in that, The variance of lane width of n lane segments at time t+1 The specific calculation formula is as follows: Where Q is the process noise variance, and the value of the process noise variance Q is related to the vehicle speed v at time t. ego yaw rate ψ ego Related.

6. The lane merging recognition method as described in claim 1, characterized in that, If a target vehicle is located within a set distance in front of this vehicle, the system will detect whether the vehicle in front and its lane meet the following conditions. If the detection result is yes, a lane change reminder will be issued. 231) The car in front is changing lanes; 232) The confidence level of the left and right lane lines of the lane where this vehicle is located is greater than the confidence threshold; 233) The short lane lines on both sides of the lane where this vehicle is located have very small curvature, while the long lane lines have large curvature.

7. The lane merging recognition method as described in claim 1, characterized in that, While identifying the lane width of the lane where the vehicle is located, the system also identifies the lane widths of the lanes on the left and right sides of the lane where the vehicle is located. When the system detects that the vehicle is changing lanes, it updates the lane width of the lane where the vehicle is located to the lane width of the lane where the vehicle is located after the lane change.

8. A lane merging recognition system, characterized in that, The system includes: A monocular camera captures images of the front of the vehicle and sends them to the processor. The processor performs lane merging detection based on the lane merging recognition method according to any one of claims 1 to 7.

9. A vehicle that integrates the lane change recognition system of claim 8.

Images