Lane line repair method and system

Abstract

The present invention discloses a lane line repair method, which includes the following steps: 1) obtaining the necessary lane line information required for the automatic driving system to calculate the expected path through a camera; 2) quickly identifying whether there is a lane line or whether the lane quality is good; 3) when there is no lane line or the lane line quality is poor, extracting, storing, calling, and converting lane line feature points, combining the feature points and the necessary lane line information required for the automatic driving system to calculate the expected path, and automatically generating new lane lines. The present invention also discloses a lane line repair system. The lane line can be repaired, fundamentally solving the problem that the automatic driving vehicle cannot drive normally or the vehicle active steering control device cannot work normally when there is no lane line or the lane line quality is poor, or when the camera system fails and cannot obtain lane line information or the camera system is attacked and outputs wrong lane line information, thereby ensuring the normal driving of the automatic driving vehicle.

Description

Technical Field

[0001] The present invention belongs to the field of automobile automatic driving, and in particular relates to a method and system for lane line repairing. Background Art

[0002] When the car is driving automatically, it needs to collect lane line information and drive in the center of the road according to the lane line information. However, some sections of the road have no lane lines or the lane line quality is poor due to being used for too long or other reasons, or the camera system fails to obtain lane line information, or the camera system is attacked and outputs wrong lane line information, resulting in the inability of the autonomous driving vehicle to drive normally or the vehicle's active steering control device to work properly (downgrade or function exit).

[0003] The patent with the publication number CN111516673A and the name of the patent is Lane Line Fusion System and Method Based on Smart Camera and High-Precision Map Positioning. It discloses a lane line fusion method based on smart camera and high-precision map positioning. The method pre-processes the lane line information output by the smart camera and high-precision map positioning controller; determines whether there is an abnormality in the pre-processed smart camera lane line; if an abnormality occurs, determines whether the high-precision map locator lane line is available at this time, and if it is available, selects the lane line output by the high-precision map locator to assist in automatic driving control; if the smart camera lane line is restored and available, selects the lane line output by the smart camera to assist in automatic driving control; tracks, verifies and processes the fused lane line. When the lane line on one side is lost, this method only performs translation and completion according to the lane width, and cannot adapt to various road conditions in real time. For example, it cannot repair the lane line when the width of the current road surface changes and the lane line width changes, and cannot ensure the normal driving of the vehicle. Summary of the invention

[0004] The purpose of the present invention is to provide a method and system for lane line repairing, which can repair lane lines, fundamentally solving the problem that there are no lane lines or the lane lines are of poor quality, or when the camera system fails and cannot obtain lane line information or the camera system is attacked and outputs wrong lane line information, the autonomous driving vehicle cannot drive normally or the vehicle's active steering control device cannot work normally (degradation or function exit), thereby ensuring the normal driving of the autonomous driving vehicle.

[0005] The technical solution adopted by the present invention is:

[0006] A method for lane line repairing comprises the following steps:

[0007] 1) Obtain the necessary lane information required by the autonomous driving system to calculate the desired path through the camera;

[0008] 2) Quickly identify whether there are lane lines or whether the lane quality is good;

[0009] 3) When there are no lane lines or the quality of the lane lines is poor, the lane line feature points are extracted, stored, called, and converted, and the necessary lane line information required for the desired path is calculated by combining the feature points and the autonomous driving system to automatically generate new lane lines.

[0010] In step 3), the method of extracting, storing, calling and converting lane feature points, combining the feature points with the necessary lane information required for the automatic driving system to calculate the desired path, and automatically generating new lanes is as follows:

[0011] The lane line data of the previously stored valid path is called, and the lane line feature points are converted to the current vehicle coordinate system through coordinate transformation. The necessary lane line information required for the autonomous driving system to calculate the desired path is combined to automatically generate new lane lines.

[0012] Or, in step 3), the method of extracting, storing, calling, and converting lane feature points, combining the feature points with the necessary lane information required for the automatic driving system to calculate the desired path, and automatically generating new lanes is:

[0013] When there is a vehicle ahead, the position coordinates of the vehicle at different sensing times are identified through cameras and millimeter-wave radars, and the historical driving trajectories of each sensing vehicle in the coordinate system of the vehicle are fitted; the historical trajectory curve is used to determine whether the vehicle ahead has changed lanes; if there is no lane change, feature points of the historical driving trajectory are extracted and stored, and new lane lines are automatically generated based on the feature points of the historical driving trajectory combined with the necessary lane line information required for the autonomous driving system to calculate the expected path.

[0014] A further solution is to remove the historical trajectory of the lane-changing vehicle and not extract the coordinate points if there is a lane-changing behavior.

[0015] A further solution is to determine whether the vehicle is changing lanes based on the extreme curvature and parallelism deviation of the historical driving trajectory curve. If the curvature value changes significantly or the parallelism deviation is large, it is determined that the vehicle is changing lanes.

[0016] A further solution is that the lane line feature points include feature points C that are continuously extracted at equal intervals within the visible range of the lane line by the automatic driving system during vehicle driving, and feature points of the driving history trajectory of the vehicle in front; the feature points of the driving history trajectory of the vehicle in front include historical trajectory feature points A of the vehicle in front and the vehicle in the same lane, and historical trajectory feature points B of the vehicle in front and the vehicle in different lanes.

[0017] A further solution is that when the lane line ahead cannot be identified, the stored feature points A, B, and C are converted to the current vehicle coordinate system through coordinate transformation, and new lane lines are automatically generated based on the extracted converted feature points and the necessary lane line information required for the automatic driving system to calculate the desired path;

[0018] When feature points A, B, and C all exist, feature points A and C are preferentially used for fitting to generate a new lane line. When there is no feature point A, that is, there is no vehicle in front of the same lane, feature point B needs to be translated in the coordinate system. The translation distance is N*D, where D is the lane line width and N is the number of lanes between the target vehicle and the vehicle. After translation, it is fitted with feature point C to generate a new lane line.

[0019] A further solution is to assume that when the lane line cannot be identified, the driver's visible distance is R, and the number of automatically stored lane line feature points A, C or B, C is n, so the extraction interval of lane line feature points is At this time, the horizontal coordinates of the lane feature points are 0, …,R;

[0020] Assume the lane line equation is y=c0+c1x+c2x 2 +c3x 3 ;

[0021] From this we can get y(0)=c0

[0022]

[0023]

[0024]

[0025] …

[0026] y(n)=c0+c1(R)+c2(R) 2 +c3(R) 3

[0027] Transform the lane feature points to the current vehicle coordinate system:

[0028] Assume the behavior of vehicle coordinate system translation The angle of rotation is θ,

[0029] The transformed coordinates are:

[0030] x′=(xa)cosθ+(yb)sinθ

[0031] y′=(yb)cosθ-(xa)sinθ

[0032] After substituting the coordinates of the lane feature points, the transformed lane feature points can be calculated:

[0033] x′(0)=(x0-a)cosθ+(y0-b)sinθ

[0034] y′(0)=(y(0)-b)cosθ-(x(0)-a)sinθ . . .

[0035] x′(n)=(x n -a)cosθ+(y n -b) sinθ

[0036] y′(n)=(y(n)-b)cosθ-(x(n)-a)sinθ

[0037] where x0, x1, x ,2 ,x3,x4. . . x n =0, . . . Ry(0)=c0, . . . y(n)=c0+c1(R)+c2(R) 2 +c3(R) 3

[0038] According to the lane line equation: y = c0 + c1x + c2x 2 +c3x 3

[0039] When there are no less than 4 lane feature points, substitute the lane feature points into the lane equation to obtain:

[0040]

[0041] The necessary lane line information c0′, c1′, c2′, c3′ required for the system to calculate the desired path can be obtained by calculation

[0042] This generates the expected path:

[0043] y″=c0′+c1′x+c2′x 2 +c3′x 3 .

[0044] The present invention also provides a lane line repair system, the system adopts the lane line repair method, and includes a collection module, a lane line signal fault diagnosis module, and a lane line repair module;

[0045] The acquisition module is used to collect lane line information and front vehicle information;

[0046] The lane line signal fault diagnosis module determines whether there is a lane line or whether the lane quality is good based on the data transmitted by the acquisition module;

[0047] The lane line repair module receives data from the acquisition module and the lane line signal fault diagnosis module, actively calls the lane line data of the valid path stored previously, converts the lane line feature points to the current vehicle coordinate system through coordinate transformation, and generates new lane lines in combination with the necessary lane line information required for the expected path of the autonomous driving system.

[0048] The acquisition module includes a camera.

[0049] The beneficial effects of the present invention are:

[0050] By setting up a lane line repair module, the problem of the autonomous driving vehicle being unable to drive normally or the vehicle's active steering control device being unable to work properly (degradation or function exit) when there are no lane lines or the lane lines are of poor quality, or when the camera system fails and cannot obtain lane line information or the camera system is attacked and outputs wrong lane line information is fundamentally solved. This reduces the error execution rate of the vehicle's active steering device and ensures the normal driving of the autonomous driving vehicle.

[0051] All information of the lane signal fault diagnosis module comes from the camera and has nothing to do with other information source components (high-precision maps, positioning equipment, radar, etc.), ensuring the accuracy of data collection and providing strong support for subsequent lane repairs.

[0052] It can be used to repair various lane lines, that is, it can be used to repair lane lines when the width of the road surface ahead changes and the width of the lane line changes.

[0053] It can repair lane line information according to the current specific situation, thereby avoiding the problem of frequent degradation of the autonomous driving system due to short-term poor lane line quality, bringing users a better driving experience, expanding the application scenarios of autonomous driving vehicles, and reducing the error execution rate of the vehicle's active steering device.

[0054] By collecting three feature points (features A, B, and C) and combining them with the necessary lane line information required for the desired path of the autonomous driving system, new lane lines can be generated quickly and accurately to ensure the normal driving of the vehicle and ensure driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0056] Figure 1 is the translation diagram of the vehicle coordinate system;

[0057] Figure 2 is a schematic diagram of a lane line repair system;

[0058] Figure 3 This is a working diagram of the lane patching module. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0060] Example 1

[0061] See also Figure 2 and Figure 3 , a lane line repair system, which includes an acquisition module, a lane line signal fault diagnosis module, and a lane line repair module. The acquisition module mainly includes a camera, which is used to collect lane line information and information of the vehicle in front. The lane line signal fault diagnosis module determines whether there is a lane line or whether the lane quality is good based on the data transmitted by the acquisition module. The lane line repair module receives the data transmitted by the acquisition module and the lane line signal fault diagnosis module, actively calls the lane line data of the valid path stored before, converts the lane line feature points to the current vehicle coordinate system through coordinate transformation, and generates new lane lines in combination with the necessary lane line information required for the expected path of the automatic driving system. The automatic driving system controls the centering of the vehicle according to the new lane line, and controls the steering actuator to steer.

[0062] Example 2

[0063] A method for lane line repairing, comprising the following steps

[0064] When the camera system fails and cannot obtain lane line information or the camera system is attacked and outputs wrong lane line information,

[0065] The lane line signal fault diagnosis module can quickly identify whether there is a lane line or whether the lane quality is good. When there is no lane line or the lane line quality is poor, the lane line repair module calls the lane line data of the previously stored valid path, converts the lane line feature points to the current vehicle coordinate system through coordinate transformation, and combines the necessary lane line information required by the automatic driving system to calculate the expected path to automatically generate new lane lines. In a short period of time, the vehicle's active steering control device will be prevented from being degraded and the function exited, ensuring the normal driving of the vehicle.

[0066] Example 3

[0067] A method for lane line repairing comprises the following steps:

[0068] 1) Obtain the necessary lane information required by the autonomous driving system to calculate the desired path through the camera;

[0069] 2) Quickly identify whether there are lane lines or whether the lane quality is good;

[0070] 3) When there are no lane lines or the quality of the lane lines is poor, the lane line feature points are extracted, stored, called, and converted. The feature points are combined with the necessary lane line information required for the autonomous driving system to calculate the desired path, and new lane lines are automatically generated. Specifically:

[0071] When there is a vehicle ahead, the camera and millimeter-wave radar identify the position coordinates of the vehicle at different sensing times and fit the historical driving trajectory of each sensing vehicle in the coordinate system of the vehicle.

[0072] Determine whether the vehicle ahead is changing lanes based on the historical trajectory curve:

[0073] Calculate the extreme curvature of all historical driving trajectory curves. If the curvature value changes significantly or the parallelism deviation is large, it can be judged that the vehicle is changing lanes. Such historical trajectories are discarded and coordinate points are not extracted. Feature points are extracted from the remaining historical driving trajectories and stored.

[0074] Through the front millimeter wave radar and camera of the vehicle, the driving history of the vehicle in front can be divided into two categories: 1. The vehicle in front is in the same lane as the vehicle; 2. The vehicle in front is in a different lane from the vehicle. The feature points extracted in the first category are collectively referred to as feature points A, and the feature points extracted in the second category are collectively referred to as feature points B.

[0075] At the same time, during the driving process of the vehicle, the autonomous driving system continuously extracts feature points C at equal intervals within the visible range of the lane line and stores them.

[0076] When the lane line ahead cannot be identified, the stored feature points A, B, and C are converted to the current vehicle coordinate system through coordinate transformation. Based on the extracted converted feature points, the necessary lane line information required for the autonomous driving system to calculate the desired path is obtained, and new lane lines are automatically generated:

[0077] When feature points A, B, and C all exist, feature points A and C are preferentially used for fitting. When there is no feature point A, that is, there is no vehicle ahead, feature point B needs to be translated in the coordinate system, and the translation distance is N*D, where D is the lane width and N is the number of lanes between the vehicle ahead and the vehicle.

[0078] Assume that when the lane line cannot be identified, the driver's visible distance is R, and the number of automatically stored lane line A, C or B, C feature points is n, so the extraction interval of lane line feature points is At this time, the horizontal coordinates of the lane feature points are 0, …, R. Let the lane line equation be y=c0+c1x+c2x 2 +c3x 3

[0079] From this we can get y(0)=c0

[0080]

[0081]

[0082]

[0083] …

[0084] y(n)=c0+c1(R)+c2(R) 2 +c3(R) 3

[0085] Transform the lane feature points to the current vehicle coordinate system:

[0086] Assume the behavior of vehicle coordinate system translation The angle of rotation is θ, see Figure 1 , then the transformed coordinates are:

[0087] x′=(xa)cosθ+(yb)sinθ

[0088] y′=(yb)cosθ-(xa)sinθ

[0089] After substituting the coordinates of the lane line feature points, the transformed lane line feature points can be calculated.

[0090] x′(0)=(x0-a)cosθ+(y0-b)sinθ

[0091] y′(0)=(y(0)-b)cosθ-(x(0)-a)sinθ . . .

[0092] x′(n)=(x n -a)cosθ+(y n -b) sinθ

[0093] y′(n)=(y(n)-b)cosθ-(x(n)-a)sinθ

[0094] where x0, x1, x ,2 ,x3,x4. . . x n =0, . . . R

[0095] y(0)=c0, . . . y(n)=c0+c1(R)+c2(R) 2 +c3(R) 3

[0096] According to the lane line equation y=c0+c1x+c2x 2 +c3x 3

[0097] When there are no less than 4 lane feature points, substitute the lane feature points into the lane equation to obtain:

[0098]

[0099] The necessary lane line information c0′, c1′, c2′, c3′ required for the system to calculate the desired path can be obtained by calculation.

[0100] This generates the expected path:

[0101] y″=c0′+c1′x+c2′x 2 +c3′x 3

[0102] Therefore, when the quality of the lane line ahead is poor, the feature points of the vehicle ahead can be extracted and combined with the feature points of the vehicle itself to obtain the lane line equation of the desired path, and a new lane line can be automatically generated.

[0103] Implementation Cases:

[0104] When the lane line ahead cannot be identified,

[0105] Assume t e There is no lane line from time t, and the duration is τ, that is, the lane line-free time T = [t e ,t e +τ]. Then, the measurement output of the camera system is:

[0106]

[0107] The lane patching module extracts, stores, calls, and converts lane feature points, and the autonomous driving system actively calculates the necessary lane information c0, c1, c2, and c3 required for the desired path. Then, the measurement output of the camera system is:

[0108]

[0109] The lane line repair module fundamentally solves the problem that the vehicle's active steering control device cannot work properly (degraded or functionally exited) when there are no lane lines or the lane line quality is poor, or when the camera system fails and cannot obtain lane line information or the camera system is attacked and outputs incorrect lane line information. It expands the application scenarios of such devices and reduces the error execution rate of the vehicle's active steering device.

[0110] All information of the lane line information fault diagnosis module comes from the camera and has nothing to do with other information source components (high-precision maps, positioning equipment, radar, etc.), which expands the application scenarios of such devices.

[0111] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the scope of protection of the appended claims of the present invention.

Claims

1. A method for lane line repairing, characterized in that: The steps include: 1) Obtain the necessary lane information required by the autonomous driving system to calculate the desired path through the camera; 2) Quickly identify whether there are lane lines or whether the lane quality is good; 3) When there are no lane lines or the quality of the lane lines is poor, the lane line feature points are extracted, stored, called, and converted, and the necessary lane line information required for the autonomous driving system to calculate the desired path is combined with the feature points to automatically generate new lane lines; The lane line feature points include feature points C and feature points of the driving history trajectory of the front vehicle, which are extracted at equal intervals within the visible range of the lane line by the automatic driving system during the driving process of the vehicle; the driving history trajectory feature points of the front vehicle include feature points A of the history trajectory of the front vehicle and the vehicle in the same lane, and feature points B of the history trajectory of the front vehicle and the vehicle in different lanes; The method for automatically generating a new lane line is: Assume that when the lane line cannot be identified, the driver's visible distance is R, and the number of automatically stored lane line feature points "A, C" or "B, C" is n+1, so the extraction interval of lane line feature points is At this time, the horizontal coordinates of the lane feature points are Assume the lane line equation is y=c0+c1x+c2x 2 +c3x 3 ; From this we can get y(0)=c0 Transform the lane feature points to the current vehicle coordinate system: Assume the behavior of vehicle coordinate system translation The angle of rotation is θ, The transformed coordinates are: x′=(xa)cosθ+(yb)sinθ y′=(yb)cosθ-(xa)sinθ After substituting the coordinates of the lane feature points, the transformed lane feature points are calculated: x′(0)=(x0-a)cosθ+(y0-b)sinθ y′(0)=(y(0)-b)cosθ-(x(0)-a)sinθ … x′(n)=(x n -a)cosθ+(y n -b)sinθ y′(n)=(y(n)-b)cosθ-(x(n)-a)sinθ in According to the lane line equation: y = c0 + c1x + c2x 2 +c3x 3 When there are no less than 4 lane feature points, substitute the lane feature points into the lane equation to obtain: Calculate the necessary lane line information c0 required to obtain the desired path ' 、c1 ' 、c2 ' 、c3 ' This generates the expected path: y″=c0′+c1′x+c2′x 2 +c3′x 3 。 2. The lane line repairing method according to claim 1, characterized in that: In step 3), the method of extracting, storing, calling and converting lane feature points, combining the feature points with the necessary lane information required for the automatic driving system to calculate the desired path, and automatically generating new lanes is as follows: The lane line data of the previously stored valid path is called, and the lane line feature points are converted to the current vehicle coordinate system through coordinate transformation. The necessary lane line information required for the autonomous driving system to calculate the desired path is combined to automatically generate new lane lines.

3. The lane line repairing method according to claim 1, characterized in that: In step 3), the method of extracting, storing, calling and converting lane feature points, combining the feature points with the necessary lane information required for the automatic driving system to calculate the desired path, and automatically generating new lanes is as follows: When there is a vehicle ahead, the position coordinates of the vehicle at different sensing times are identified through cameras and millimeter-wave radars, and the historical driving trajectories of each sensing vehicle in the coordinate system of the vehicle are fitted; the historical trajectory curve is used to determine whether the vehicle ahead has changed lanes; if there is no lane change, feature points of the historical driving trajectory are extracted and stored, and new lane lines are automatically generated based on the feature points of the historical driving trajectory combined with the necessary lane line information required for the automatic driving system to calculate the expected path.

4. The lane line repairing method according to claim 3, characterized in that: If there is a lane change, the historical trajectory of the lane-changing vehicle will be removed and no coordinate point extraction will be performed.

5. The lane line repairing method according to claim 3 or 4, characterized in that: Whether the vehicle is changing lanes is determined based on the extreme curvature and parallelism deviation of the historical driving trajectory curve. If the curvature value changes significantly or the parallelism deviation is large, it is determined that the vehicle is changing lanes.

6. The lane line repairing method according to claim 1, characterized in that: When the lane line ahead cannot be identified, the stored feature points A, B, and C are converted to the current vehicle coordinate system through coordinate transformation. Based on the extracted converted feature points and the necessary lane line information required for the automatic driving system to calculate the desired path, a new lane line is automatically generated. When feature points A, B, and C all exist, feature points A and C are used for fitting to generate a new lane line. When there is no feature point A, that is, there is no vehicle in front of the same lane, feature point B needs to be translated in the coordinate system. The translation distance is N*D, where D is the lane line width and N is the number of lanes between the target vehicle and the vehicle. After translation, it is fitted with feature point C to generate a new lane line.

7. A lane line repair system, characterized in that: The system adopts the lane line repairing method described in any one of claims 1-6.

8. The lane line repairing system according to claim 7, characterized in that: The system includes an acquisition module, a lane signal fault diagnosis module, and a lane repair module; The acquisition module is used to collect lane line information and front vehicle information; The lane line signal fault diagnosis module determines whether there is a lane line or whether the lane quality is good based on the data transmitted by the acquisition module; The lane line repair module receives data from the acquisition module and the lane line signal fault diagnosis module, actively calls the lane line data of the valid path stored previously, converts the lane line feature points to the current vehicle coordinate system through coordinate transformation, and generates new lane lines in combination with the necessary lane line information required for the expected path of the autonomous driving system.

Images