Lane Edge Correction Method and System Based on Camera Perception

Abstract

The present invention discloses a lane boundary line correction method and system based on camera perception. By using the heading angle rate provided by the IMU and the wheel speed measured by the odometer, the relative displacement of the vehicle during the time interval between the previous frame and the current frame of camera data is deduced. Then, equally spaced points are taken from the lane boundary lines perceived by the camera in the front and rear frames, and multiple sampling points on each lane boundary line are obtained respectively. The sampling points in the current frame are translated and corrected according to the relative displacement of the vehicle. Finally, curve fitting is performed based on the corrected sampling points in the current frame and the sampling points in the previous frame to obtain the corrected lane boundary line perceived by the camera, thereby ensuring the matching accuracy between the high-precision map and the lane boundary line perceived by the camera, and further improving the accuracy of curve guidance in autonomous driving.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous vehicle guidance, and particularly to a lane boundary line correction method and system based on camera perception. Background Art

[0002] During high-precision autonomous driving, when the vehicle is driving on a curve, the curvature of the lane boundary lines perceived by the vehicle camera changes. Due to the current insufficient technical ability to perceive the change curvature of the lane boundary lines by the camera, the error in detecting the lane boundary lines of the curve is relatively large. If the curve curvature is large, that is, in the case of an overly sharp curve, the error of the lane boundary lines perceived by the camera increases, resulting in a large deviation when matching the lane boundary lines of the high-precision map with the lane boundary lines perceived by the camera, and the accuracy of the matching positioning result decreases. Therefore, it is necessary to correct the lane boundary lines perceived by the camera to ensure the matching accuracy of the high-precision map and the lane boundary lines perceived by the camera. Summary of the Invention

[0003] The purpose of the present invention is to overcome the above technical deficiencies, and propose a lane boundary line correction method and system based on camera perception to solve the problem of large deviation when matching the lane boundary lines of the existing high-precision map with the lane boundary lines perceived by the camera.

[0004] To achieve the above technical purpose, the first aspect of the technical solution of the present invention provides a lane boundary line correction method based on camera perception, which includes the following steps:

[0005] Calculate the relative displacement of the vehicle within the time interval between the previous frame and the current frame of camera data by using the heading angle rate provided by the IMU and the wheel speed measured by the odometer;

[0006] Take equally spaced points on the lane boundary lines perceived by the camera in the front and rear two frames, respectively obtain a plurality of sampling points on each lane boundary line, and translate and correct the sampling points of the current frame according to the relative displacement of the vehicle;

[0007] Perform curve fitting on the corrected sampling points of the current frame and the sampling points of the previous frame to obtain the corrected lane boundary lines perceived by the camera.

[0008] The second aspect of the present invention provides a lane boundary line correction system based on camera perception, which includes the following functional modules:

[0009] Relative displacement calculation module, which is used to calculate the relative displacement of the vehicle within the time interval between the previous frame and the current frame of camera data by using the heading angle rate provided by the IMU and the wheel speed measured by the odometer;

[0010] The translation correction module is used to equally spaced sample points by using the parametric equations of the lane boundary lines sensed by the cameras in the previous and current frames, respectively obtain multiple sampling points on each lane boundary line, and translate and correct the sampling points of the current frame according to the relative displacement of the vehicle.

[0011] The curve fitting module is used to perform curve fitting on the corrected sampling points of the current frame and the sampling points of the previous frame to obtain the corrected lane boundary lines sensed by the camera.

[0012] A third aspect of the present invention provides a server, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned lane boundary line correction method based on camera perception is implemented.

[0013] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the above-mentioned lane boundary line correction method based on camera perception is implemented.

[0014] Compared with the prior art, the lane boundary line correction method and system based on camera perception of the present invention calculate the relative displacement of the vehicle during the time interval between the camera data of the previous frame and the current frame by using the heading angle rate provided by the IMU and the wheel speed measured by the odometer; then equally spaced sample points on the lane boundary lines sensed by the cameras in the previous and current frames are obtained, respectively obtaining multiple sampling points on each lane boundary line, and the sampling points of the current frame are translated and corrected according to the relative displacement of the vehicle; finally, curve fitting is performed on the corrected sampling points of the current frame and the sampling points of the previous frame to obtain the corrected lane boundary lines sensed by the camera, thereby ensuring the matching accuracy between the high-precision map and the lane boundary lines sensed by the camera, and further improving the accuracy of curve guidance in autonomous driving. Description of the Drawings

[0015] Figure 1 is a flowchart of the lane boundary line correction method based on camera perception according to an embodiment of the present invention;

[0016] Figure 2 is a step flowchart of the lane boundary line correction method based on camera perception according to an embodiment of the present invention;

[0017] Figure 3 is a block diagram of the modules of the lane boundary line correction system based on camera perception according to an embodiment of the present invention;

[0018] Figure 4 is a sub-module block diagram of the translation correction module in the lane boundary line correction system based on camera perception according to an embodiment of the present invention;

[0019] Figure 5It is the block diagram of the sub-module of the curve fitting module in the lane boundary correction system based on camera perception according to the embodiments of the present invention. Detailed implementation manners

[0020] In order to make the objectives, technical solutions and advantages of the present invention more clear and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0021] As Figure 1 and Figure 2 shown, the embodiments of the present invention provide a lane boundary correction method based on camera perception, which includes the following steps:

[0022] S1. Calculate the relative displacement of the vehicle within the time interval between the previous frame and the current frame of camera data by using the heading angle rate provided by the IMU and the wheel speed measured by the odometer.

[0023] That is, calculate the change in the heading angle and the driving distance of the vehicle within the time interval between the current frame and the previous frame of the camera according to the IMU data and the wheel speed data at the previous frame and the current frame of the camera respectively. Then, obtain the lateral displacement and the longitudinal displacement of the vehicle relative displacement within the time interval between the previous frame and the current frame of camera data according to the cosine value and the sine value of the change in the heading angle relative to the driving distance of the vehicle.

[0024] The step S1 specifically includes the following sub-steps:

[0025] Collect the IMU data at the previous frame and the current frame of the camera, take the average of the heading angle rates in the two groups of IMU data to obtain the average heading angle rate;

[0026] Collect the wheel speed data at the previous frame and the current frame of the camera, take the average of the two groups of wheel speed data to obtain the average wheel speed;

[0027] Calculate the change in the heading angle within the time interval between the current frame and the previous frame of the camera according to the time difference between the previous frame and the current frame of the camera and the average heading angle rate;

[0028] Calculate the driving distance of the vehicle within the time interval between the current frame and the previous frame of the camera according to the time difference between the previous frame and the current frame of the camera and the average wheel speed;

[0029] Multiply the driving distance of the vehicle within the time interval between the current frame and the previous frame of the camera by the cosine value of the change in the heading angle to obtain the lateral displacement of the vehicle relative displacement within the time interval between the previous frame and the current frame of camera data;

[0030] Multiply the vehicle driving distance within the time interval between the current frame and the previous frame of the camera by the sine value of the change in the heading angle to obtain the longitudinal displacement of the vehicle's relative displacement during the time interval between the previous frame and the current frame of camera data.

[0031] The calculation formula for the vehicle's relative displacement during the time interval between the previous frame and the current frame of camera data is as follows:

[0032]

[0033]

[0034]

[0035]

[0036] Where v is the average speed, the time interval is Δt, p is the driving distance, θ is the change in the heading angle, is the eastward displacement of the vehicle relative to the previous frame of camera data time, is the northward displacement of the vehicle relative to the previous frame of camera data time, is the displacement vector of the vehicle relative to the previous frame of camera data time.

[0037] S2. Take equally spaced points on the lane boundary lines perceived by the front and rear frames of the camera, respectively obtain multiple sampling points on each lane boundary line, and translate and correct the sampling points of the current frame according to the vehicle's relative displacement.

[0038] The specific steps of step S2 include the following sub-steps:

[0039] Use the parametric equations of the lane boundary lines perceived by the front and rear frames of the camera to take equally spaced points, and respectively obtain multiple sampling points on each lane boundary line;

[0040] Translate and correct the abscissa and ordinate of the sampling points of the current frame by the horizontal displacement and longitudinal displacement respectively.

[0041] Among them, the number of sampling points obtained on the lane boundary lines perceived by the front and rear frames of the camera is the same. Preferably, 30 sampling points are obtained on each lane boundary line, so there are a total of 60 sampling points on the corresponding lane boundary lines perceived by the front and rear frames of the camera.

[0042] Taking the coordinate sequence of 30 sampling points on the m-th lane boundary line of the current frame as an example, the sampling point sequence is , where are the coordinates of the 1st to 30th sampling points, and the displacement vector obtained in step S1 is , , ……, , after sorting, , It is the sampling point coordinate sequence of the m-th lane boundary line of the current frame after relative displacement.

[0043] S3. Perform curve fitting on the sampled points of the current frame after correction and the sampled points of the previous frame to obtain the corrected camera-perceived lane boundary line.

[0044] The step S3 specifically includes the following sub-steps:

[0045] Concatenate the coordinate sequences of multiple sampled points of the lane boundary line perceived by the previous frame camera with the coordinate sequences of multiple sampled points of the lane boundary line perceived by the current frame camera after translation correction.

[0046] Use the least squares method to perform curve fitting on the sampled points of the current frame after correction and the sampled points of the previous frame to obtain the corrected camera-perceived lane boundary line.

[0047] After obtaining the corrected camera-perceived lane boundary line, intercept the lane boundary line within the threshold range centered on the vehicle body position at the time of the previous frame camera data in the high-precision map, and convert the lane boundary line to the vehicle body coordinate system at the time of the previous frame camera data to obtain the lane boundary line of the high-precision map in the vehicle body coordinate system. Fit the parametric equations of each lane boundary line in the high-precision map, and match each lane boundary line in the high-precision map with the corresponding corrected camera-perceived lane boundary line to improve the accuracy of curve guidance in autonomous driving.

[0048] The method for converting the lane boundary line to the vehicle body coordinate system at the current time is as follows:

[0049] Subtract the longitude and latitude of each point on each lane boundary line in the high-precision map from the longitude and latitude of the vehicle body position at the time of the previous frame camera data, and then project the longitude and latitude difference onto the north and east directions of the vehicle body using the heading angle of the vehicle body at the time of the previous frame camera data to obtain the coordinates of each point on each lane boundary line in the high-precision map with the vehicle body position at the time of the previous frame camera data as the origin in the vehicle body coordinate system, that is, complete the conversion of the lane boundary line of the high-precision map to the vehicle body coordinate system at the time of the previous frame camera data.

[0050] The present invention calculates the relative displacement of the vehicle during the time interval between the previous frame and the current frame camera data by using the heading angle rate provided by the IMU and the wheel speed measured by the odometer; then equally spaced points are taken from the lane boundary lines perceived by the front and rear frame cameras respectively to obtain multiple sampled points on each lane boundary line, and the sampled points of the current frame are translationally corrected according to the relative displacement of the vehicle; finally, curve fitting is performed on the sampled points of the current frame after correction and the sampled points of the previous frame to obtain the corrected camera-perceived lane boundary line, thereby ensuring the matching accuracy of the lane boundary line of the high-precision map and the camera perception, and further improving the accuracy of curve guidance in autonomous driving.

[0051] like Figure 3 As shown, the embodiment of the present invention also discloses a lane edge correction system based on camera perception, which includes the following functional modules:

[0052] The relative displacement calculation module 10 is used to calculate the relative displacement of the vehicle within the interval between the previous frame and the current frame of camera data through the heading angular rate provided by the IMU and the wheel speed measured by the odometer;

[0053] The translation correction module 20 is used to use the lane edge parameter equations perceived by the two frames before and after the camera to take points at equal intervals, obtain multiple sampling points on each lane edge, and perform translation correction on the sampling points of the current frame according to the relative displacement of the vehicle;

[0054] The curve fitting module 30 is used to perform curve fitting based on the corrected sampling points of the current frame and the sampling points of the previous frame to obtain the corrected lane edge perceived by the camera.

[0055] Among them, the relative displacement calculation module 10 is specifically used to: calculate the heading angle change and the vehicle travel distance within the time interval between the current frame and the previous frame of the camera according to the IMU data and wheel speed data at the camera moment of the previous frame and the current frame, and obtain the lateral displacement and longitudinal displacement of the vehicle's relative displacement within the time interval between the camera data of the previous frame and the current frame according to the cosine value and sine value of the heading angle change relative to the vehicle travel distance.

[0056] like Figure 4 As shown, the translation correction module 20 includes the following sub-functional modules:

[0057] The lane edge sampling submodule 21 is used to take points at equal intervals using the lane edge parameter equations perceived by the two previous and next frames of cameras, and obtain multiple sampling points on each lane edge;

[0058] The coordinate displacement correction submodule 22 is used to perform translation correction of the horizontal displacement and the vertical displacement of the horizontal coordinate and the vertical coordinate of the sampling point of the current frame respectively.

[0059] like Figure 5 As shown, the curve fitting module 30 includes the following sub-functional modules:

[0060] The sampling point series connection submodule 31 is used to connect the coordinates of multiple sampling points of the lane edge perceived by the camera in the previous frame in series with the coordinates of multiple sampling points of the lane edge perceived by the camera in the current frame after translation correction;

[0061] The square method fitting correction submodule 32 is used to use the least square method to perform curve fitting on the corrected sampling points of the current frame and the sampling points of the previous frame to obtain the corrected camera-perceived lane edge.

[0062] The execution manner of the lane boundary line correction system based on camera perception in this embodiment is basically the same as the above-mentioned lane boundary line correction method based on camera perception, so it will not be elaborated in detail.

[0063] The server in this embodiment is a device that provides computing services, usually referring to a computer with relatively high computing power and provided to multiple consumers through a network. The server of this embodiment includes: a memory, a processor, and a system bus. The memory includes a runnable program stored thereon. Those skilled in the art can understand that the structure of the terminal device in this embodiment does not constitute a limitation on the terminal device, and it may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0064] The memory can be used to store software programs and modules. The processor executes various functional applications and data processing of the terminal by running the software programs and modules stored in the memory. The memory may mainly include a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created according to the use of the terminal (such as audio data, phone book, etc.). In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0065] A runnable program of a lane boundary line correction method based on camera perception is included in the memory. The runnable program can be divided into one or more modules / units. The one or more modules / units are stored in the memory and executed by the processor to complete the information acquisition and implementation process. The one or more modules / units can be a series of computer program instruction segments capable of completing specific functions, and these instruction segments are used to describe the execution process of the computer program in the server. For example, the computer program can be divided into a relative displacement calculation module 10, a translation correction module 20, and a curve fitting module 30.

[0066] The processor is the control center of the server, connecting various parts of the entire terminal device through various interfaces and lines. By running or executing the software programs and / or modules stored in the memory, and calling the data stored in the memory, it executes various functions of the terminal and processes data, thereby monitoring the terminal as a whole. Optionally, the processor may include one or more processing units; preferably, the processor may integrate an application processor and a modulation and demodulation processor. Among them, the application processor mainly processes the operating system, application programs, etc., and the modulation and demodulation processor mainly processes wireless communication. It can be understood that the above modulation and demodulation processor may not be integrated into the processor.

[0067] The system bus is used to connect various functional components inside a computer and can transmit data information, address information, and control information. Its types can be, for example, PCI bus, ISA bus, VESA bus, etc. The instructions of the processor are transmitted to the memory through the bus, and the memory feeds back data to the processor. The system bus is responsible for the data and instruction interaction between the processor and the memory. Of course, the system bus can also be connected to other devices, such as network interfaces, display devices, etc.

[0068] The server should at least include a CPU, chipset, memory, disk system, etc. Other components will not be elaborated here.

[0069] In the embodiment of the present invention, the executable program executed by the processor included in the terminal is specifically: a method for correcting lane boundary lines based on camera perception, which includes the following steps:

[0070] Based on the heading angle rate provided by the IMU and the wheel speed measured by the odometer, the relative displacement of the vehicle during the time interval between the previous frame and the current frame of camera data is calculated.

[0071] Equidistant sampling points are taken from the lane boundary lines perceived by the camera in the front and rear two frames, and multiple sampling points on each lane boundary line are obtained respectively. The sampling points of the current frame are corrected by translation according to the relative displacement of the vehicle.

[0072] Curve fitting is performed based on the corrected sampling points of the current frame and the sampling points of the previous frame to obtain the corrected lane boundary lines perceived by the camera.

[0073] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated here.

[0074] In the above embodiments, the descriptions of each embodiment have their own emphases. For the parts not elaborated or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0075] Those of ordinary skill in the art can realize that the modules, units, and / or method steps of each embodiment described in combination with the embodiments disclosed in this article can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of the present invention.

[0076] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A lane boundary correction method based on camera perception, characterized in that The method includes the following steps: Based on the heading angle rate provided by the IMU and the wheel speed measured by the odometer, calculate the relative displacement of the vehicle during the time interval between the previous frame and the current frame of camera data; Take equally spaced points on the lane boundary lines perceived by the front and rear frames of cameras respectively, obtain multiple sampling points on each lane boundary line, and translate and correct the sampling points of the current frame according to the relative displacement of the vehicle; Perform curve fitting on the sampling points of the current frame after correction and the sampling points of the previous frame to obtain the corrected lane boundary line perceived by the camera; 2. The lane edge correction method based on camera perception according to claim 1, wherein The step of calculating the relative displacement of the vehicle during the time interval between the previous frame and the current frame of camera data based on the heading angle rate provided by the IMU and the wheel speed measured by the odometer specifically includes: Calculate the change in heading angle and the driving distance of the vehicle during the time interval between the current frame and the previous frame of the camera respectively based on the IMU data and wheel speed data at the previous frame and current frame of the camera, and obtain the lateral displacement and longitudinal displacement of the relative displacement of the vehicle during the time interval between the previous frame and the current frame of camera data according to the cosine value and sine value of the change in heading angle relative to the driving distance of the vehicle respectively; 3. The lane boundary correction method based on camera perception according to claim 1, wherein The step of taking equally spaced points on the lane boundary lines perceived by the front and rear frames of cameras respectively, obtaining multiple sampling points on each lane boundary line, and translating and correcting the sampling points of the current frame according to the relative displacement of the vehicle specifically includes: Use the parametric equations of the lane boundary lines perceived by the front and rear frames of cameras to take equally spaced points, and obtain multiple sampling points on each lane boundary line respectively; Translate and correct the abscissa and ordinate of the sampling points of the current frame by the lateral displacement and longitudinal displacement respectively; 4. The lane boundary correction method based on camera perception according to claim 1, characterized in that The step of performing curve fitting on the sampling points of the current frame after correction and the sampling points of the previous frame to obtain the corrected lane boundary line perceived by the camera specifically includes: Concatenate the coordinates of multiple sampling points of the lane boundary line perceived by the previous frame of the camera and the coordinates of multiple sampling points of the lane boundary line perceived by the current frame of the camera after translation correction; Use the least squares method to perform curve fitting on the sampling points of the current frame after correction and the sampling points of the previous frame to obtain the corrected lane boundary line perceived by the camera; 5. A lane boundary correction system based on camera perception, characterized in that, The method includes the following functional modules: A relative displacement calculation module, which is used to calculate the relative displacement of the vehicle during the time interval between the previous frame and the current frame of camera data based on the heading angle rate provided by the IMU and the wheel speed measured by the odometer; A translation correction module, which is used to take equally spaced points using the parametric equations of the lane boundary lines perceived by the front and rear frames of cameras, obtain multiple sampling points on each lane boundary line respectively, and translate and correct the sampling points of the current frame according to the relative displacement of the vehicle; A curve fitting module, which is used to perform curve fitting on the sampling points of the current frame after correction and the sampling points of the previous frame to obtain the corrected lane boundary line perceived by the camera; 6. The lane boundary correction system based on camera perception according to claim 5, wherein The relative displacement calculation module is specifically used to: calculate the change in heading angle and the driving distance of the vehicle during the time interval between the current frame and the previous frame of the camera respectively based on the IMU data and wheel speed data at the previous frame and current frame of the camera, and obtain the lateral displacement and longitudinal displacement of the relative displacement of the vehicle during the time interval between the previous frame and the current frame of camera data according to the cosine value and sine value of the change in heading angle relative to the driving distance of the vehicle respectively; 7. The lane edge correction system based on camera perception according to claim 5, wherein The translation correction module includes the following sub-functional modules: The lane boundary sampling sub-module is used to equally space sample points by using the parametric equations of the lane boundaries sensed by the front and rear cameras, and respectively obtain multiple sampling points on each lane boundary; The coordinate displacement correction sub-module is used to respectively perform translational corrections of the horizontal displacement and the vertical displacement on the abscissa and ordinate of the sampling points in the current frame.

8. The lane edge correction system based on camera perception according to claim 5, characterized in that, The curve fitting module includes the following sub-function modules: The sampling point concatenation sub-module is used to concatenate the coordinates of multiple sampling points of the lane boundary sensed by the previous frame camera with the coordinates of multiple sampling points of the lane boundary sensed by the current frame camera after translational correction; The least squares fitting correction sub-module is used to perform curve fitting on the sampling points after correction in the current frame and the sampling points in the previous frame by using the least squares method to obtain the lane boundary sensed by the camera after correction.

9. A server, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the lane boundary correction method based on camera sensing according to any one of claims 1 to 4.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the lane boundary correction method based on camera sensing according to any one of claims 1 to 4.

Images