Lane line compensation method and device and computer storage medium

Abstract

The invention provides a lane line compensation method and device and a computer storage medium. The lane line compensation method comprises the steps that a current lane line image is acquired; extracting a current lane line point set of the current lane line image; obtaining a predicted lane line point set of the historical lane line image at the current moment; compensating the current lane line point set by using the predicted lane line point set to obtain a compensated lane line point set; and fitting a lane line equation of the current lane line image according to the compensation lane line point set. By means of the mode, the current lane line is compensated by combining the historical data, the accuracy, robustness and real-time performance of lane line detection can be remarkably improved, and more reliable support is provided for automatic driving and advanced driving assistance systems.

Description

Technical Field

[0001] The present application relates to the technical field of autonomous driving, and particularly to a lane line compensation method, device, and computer storage medium. Background Art

[0002] In recent years, more and more intelligent driving assistance systems have been applied to vehicles. While providing more high-order functions, higher requirements are also put forward for the perception units of vehicle systems. Among them, lane line perception is one of the important technologies. Functions such as lane keeping systems and high-speed piloting systems mainly rely on lane lines for vehicle control. Therefore, the stability of lane line detection is crucial for vehicle control. However, due to reasons such as road damage and poor model detection effects, lane line loss may occur. Lane line loss may cause large fluctuations in vehicle lateral control and other aspects, and the driving experience will be reduced. Therefore, it is crucial to timely and effectively compensate for lane lines that existed in the previous frame but are lost in the current frame for driving safety and comfort.

[0003] In the prior art, after the lane lines are fitted and found to be missing, the lane lines are compensated, which requires repeated calculations and cannot guarantee accuracy, and has poor robustness. Summary of the Invention

[0004] The present application provides a lane line compensation method, device, and computer storage medium.

[0005] To solve the above technical problems, the present application proposes a lane line compensation method, which includes: obtaining a current lane line image; extracting a current lane line point set of the current lane line image; obtaining a predicted lane line point set of a historical lane line image at the current moment; compensating the current lane line point set with the predicted lane line point set to obtain a compensated lane line point set; and fitting a lane line equation of the current lane line image according to the compensated lane line point set.

[0006] Among them, the step of compensating the current lane line point set with the predicted lane line point set to obtain a compensated lane line point set includes: obtaining a clustering result of the current lane line point set; selecting a plurality of to-be-matched clustering points from the clustering result; matching the plurality of to-be-matched clustering points with the predicted lane line point set to obtain successfully matched clustering points; and when the number of the successfully matched clustering points is less than a preset threshold, adding the predicted lane line point set to the current lane line point set to generate the compensated lane line point set.

[0007] Among them, adding the predicted lane line point set to the current lane line point set to generate the compensated lane line point set includes: obtaining the lane line serial numbers of the unsuccessfully matched points based on the unsuccessfully matched clustered points; obtaining the predicted lane line points associated with the lane line serial numbers of the unsuccessfully matched points from the predicted lane line point set; adding the predicted lane line points to the current lane line point set to generate the compensated lane line point set.

[0008] Among them, the lane line compensation method further includes: when the number of successfully matched clustered points is greater than or equal to the preset threshold, obtaining the lane line serial numbers associated with the successfully matched clustered points; and fitting the lane line equation of the current lane line image according to the lane line serial numbers by using the current lane line points belonging to the same lane line serial number.

[0009] Among them, obtaining the predicted lane line point set of the historical lane line image at the current moment includes: obtaining the historical lane line points of the historical lane line image; and mapping the historical lane line points to the vehicle coordinate system of the current lane line image to generate the predicted lane line point set.

[0010] Among them, mapping the historical lane line points to the vehicle coordinate system of the current lane line image includes: obtaining the first extrinsic camera parameters at the current moment and the second extrinsic camera parameters at the historical moment; mapping the historical lane line points to the world coordinate system by using the second extrinsic camera parameters; and mapping the historical lane line points from the world coordinate system to the vehicle coordinate system of the current lane line image by using the first extrinsic camera parameters.

[0011] Among them, obtaining the historical lane line points of the historical lane line image includes: extracting all the lane line points and the fitted historical lane lines of the historical lane line image; and downsampling all the lane line points by using the historical lane lines to obtain the historical lane line points.

[0012] To solve the above technical problems, the present application provides a lane line compensation device, which includes: an image acquisition module, an image processing module, and a calculation module; the image acquisition module is configured to acquire a current lane line image; the image processing module is configured to extract the current lane line point set of the current lane line image; obtain the predicted lane line point set of the historical lane line image at the current moment; compensate the current lane line point set by using the predicted lane line point set to obtain a compensated lane line point set; and the calculation module is configured to fit the lane line equation of the current lane line image according to the compensated lane line point set.

[0013] To solve the above technical problems, the present application provides a lane line compensation device, which includes a memory and a processor coupled to the memory; wherein, the memory is used to store program data, and the processor is used to execute the program data to implement the above-mentioned lane line compensation method.

[0014] To solve the above technical problems, the present application provides a computer storage medium, which is used to store program data, and when the program data is executed by a computer, it is used to implement the above-mentioned lane line compensation method.

[0015] Different from the prior art, the beneficial effect of the present application lies in that the lane line compensation device acquires the current lane line image; extracts the current lane line point set of the current lane line image; acquires the predicted lane line point set of the historical lane line image at the current moment; uses the predicted lane line point set to compensate the current lane line point set to obtain a compensated lane line point set; and fits the lane line equation of the current lane line image according to the compensated lane line point set. In this way, by compensating the current lane line by combining historical data, the accuracy, robustness, and real-time performance of lane line detection can be significantly improved, providing more reliable support for autonomous driving and advanced driver assistance systems. Description of the Drawings

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other drawings without creative efforts based on these drawings.

[0017] Figure 1 It is a schematic flowchart of the first embodiment of the lane line compensation method provided by the present application;

[0018] Figure 2 It is the Figure 1 schematic flowchart of the sub-steps of step S13 in the lane line compensation method provided by the present application;

[0019] Figure 3 It is a schematic diagram of a classical lane line fitting model provided by the present application;

[0020] Figure 4 It is a schematic flowchart of the second embodiment of the lane line compensation method provided by the present application;

[0021] Figure 5 It is a schematic diagram of the matching and compensation of the lane line provided by the present application;

[0022] Figure 6 It is a schematic structural diagram of an embodiment of the lane line compensation device provided by the present application;

[0023] Figure 7 It is a schematic structural diagram of another embodiment of the lane line compensation device provided by the present application;

[0024] Figure 8 It is a schematic structural diagram of an embodiment of the computer storage medium provided by the present application. Detailed implementation manners

[0025] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0026] Among them, the lane line compensation method of the present application is applied to a lane line compensation device. Among them, the lane line compensation device of the present application can be a server or a system in which the server and the local terminal cooperate with each other. Correspondingly, each part included in the lane line compensation device, such as each unit, sub-unit, module, and sub-module, can be all arranged in the server or can be respectively arranged in the server and the local terminal.

[0027] Furthermore, the above-mentioned server can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster composed of multiple servers or as a single server. When the server is software, it can be implemented as multiple software or software modules, such as software or software modules for providing a distributed server, or can be implemented as a single software or software module, which is not specifically limited herein. In some possible implementation manners, the lane line compensation method in the embodiments of the present application can be implemented by a processor calling computer-readable instructions stored in a memory.

[0028] The prior art can improve the stability of lane lines to a certain extent, but the lane line parallelism needs to be in a structured road to meet the lane line parallelism. For example, ramps, special-shaped lines, etc. that do not meet the lane line parallelism cannot be compensated.

[0029] Furthermore, if only the left and right lane lines are compensated, there will be a lack of lane lines during the process of vehicle lane change, and the vehicle cannot smoothly complete the lane change.

[0030] In addition, the prior art focuses on post-compensation of lane lines, that is, after the lane line fitting is completed and a lack is found, the lane line is compensated. In this calculation process, it needs to be recalculated once, and the accuracy can also be guaranteed.

[0031] To solve the above technical problems, the present application provides a lane line compensation method. After lane line matching is completed, it is checked whether the lane line existed in the previous frame and whether there is a situation in the current frame where there are no points to be fitted and matched. If such a situation exists, the inferred points of the missing lane line of the current lane line in the previous frame predicted by the inertial measurement unit are filled into the queue to be fitted. Next, the queue to be fitted is sequentially fitted and tracked, thereby completing the missing compensation of the lane line, making the output lane line more stable and maximizing the stability of lane line detection.

[0032] To solve the above technical problems, the present application proposes a lane line compensation method. Please refer to Figure 1 , Figure 1 which is a schematic flowchart of the first embodiment of the lane line compensation method provided by the present application.

[0033] As Figure 1 shown, the specific steps are as follows:

[0034] Step S11: Obtain the current lane line image.

[0035] Among them, the current lane line image is the lane line image at the current moment t, which can be a certain frame intercepted from a video or a real-time image taken at the current moment.

[0036] In the embodiment of the present application, a vision sensor system is used to collect lane line information. Specifically, the vision sensing system can combine data from multiple sensors such as cameras, radars, and lidar, and obtain the lane line image through a fusion algorithm. The color and texture features of the lane line are extracted from the image captured by the camera, the geometric features of the lane line are extracted from the lidar point cloud, and the distance features of the lane line are extracted from the radar data. Finally, the features of each sensor are input into the fusion model, and the final lane line detection result is generated through weighted average or deep learning algorithms.

[0037] It should be noted that the present application does not limit the specific algorithm type. The camera can be installed at any position of the vehicle to capture the image of the road ahead.

[0038] Step S12: Extract the current lane line point set of the current lane line image.

[0039] Among them, the current lane line point set is a set composed of discrete points extracted from the current lane line image through model inference.

[0040] Step S13: Obtain the predicted lane line point set of the historical lane line image at the current moment.

[0041] Among them, the historical lane line image is any frame image before the time when the current lane line image is located, and the current moment is the moment when the current lane line image is located.

[0042] Specifically, in an embodiment of the present application, the predicted lane line point set of the historical lane line image at the current moment can be obtained in the following manner. For details, please refer to Figure 2 , Figure 2 which is the Figure 1 flow schematic diagram of the sub-step of step S13 in the lane line compensation method provided by the present application.

[0043] As Figure 2 shown, the specific steps are as follows:

[0044] Step S131: Obtain the historical lane line points of the historical lane line image.

[0045] Specifically, the method for the lane line compensation device to obtain historical lane line points is the same as that for current lane line points, which will not be elaborated here.

[0046] Step S132: Map the historical lane line points to the vehicle coordinate system of the current lane line image to generate a predicted lane line point set.

[0047] In an embodiment of the present application, the lane line compensation device obtains the first camera extrinsic parameters at the current moment and the second camera extrinsic parameters at the historical moment; maps the historical lane line points to the world coordinate system by using the second camera extrinsic parameters; and maps the historical lane line points from the world coordinate system to the vehicle coordinate system of the current lane line image by using the first camera extrinsic parameters.

[0048] In an embodiment of the present application, the lane line compensation device extracts all lane line points of the historical lane line image and the fitted historical lane line; downsamples all the lane line points by using the historical lane line to obtain the historical lane line points.

[0049] Specifically, the lane line compensation device obtains the sensor data of the vision at time t in real time, sends it into the lane line detection model, samples, clusters, and fits the output result of the model, obtains the pose and offset information of the inertial measurement unit at time t in real time, generates the extrinsic parameters at this moment, which are composed of the extrinsic rotation matrix and the translation vector, samples the lane line generated at time t to generate a point set in the vehicle coordinate system, combines the extrinsic parameters at time t to obtain the coordinates of the sampled points at time t in the world coordinate system, obtains the pose and offset information of the inertial measurement unit at time t + 1 in real time, generates the extrinsic parameter matrix at this moment, combines the sampled points obtained at time t and the inverse of the extrinsic parameter matrix generated at time t + 1 to obtain the coordinates of the historical points at time t in the vehicle coordinate system corresponding to time t + 1, that is, the predicted lane prediction point set.

[0050] Specifically, as Figure 3 shown, Figure 3It is a schematic diagram of the classic lane line fitting model provided by this application. Lane line model curve equation:

[0051] Among them, y0 represents the lateral offset, ε represents the angle of the vehicle relative to the road curve, C0 represents the curvature of the curve, and C1 represents the curvature change rate.

[0052] Read the yaw angle, pitch angle, roll angle and offsets x, y, z of the inertial measurement unit, and the external parameter matrix of the vehicle body to the world coordinate system read above:

[0053]

[0054] Among them, R 3×3 is the rotation matrix at time t read from the inertial measurement unit, and T 3×1 is the offset at time t read from the inertial measurement unit.

[0055]

[0056] Obtain the coordinates of the sampling points of the lane line at time t corresponding to the world coordinate system through the above calculation method.

[0057] Through the above method, the predicted lane line point set of the historical lane line image can be accurately obtained.

[0058] Step S14: Use the predicted lane line point set to compensate the current lane line point set to obtain a compensated lane line point set.

[0059] The lane line compensation device obtains the prediction result of the point at time t at time t + 1. At time t + 1, the current lane line is sampled, clustered, and matched. When there is a clustering result that exists at time t but has no matching at time t + 1, the point information of time t + 1 predicted based on the inertial measurement unit at time t is used to compensate the clustering result at time t + 1, and then the lane line is updated with the fused clustering information, thus completing the lane line compensation at time t + 1.

[0060] Specifically, this application proposes an embodiment for generating a compensated lane line point set. For details, please refer to Figure 4 and Figure 5 , Figure 4 is the schematic flow diagram of the second embodiment of the lane line compensation method provided by this application, Figure 5 is the schematic diagram of the matching and compensation of the lane line provided by this application.

[0061] As Figure 4 shown, the specific steps are as follows:

[0062] Step S21: Obtain the clustering result of the current lane line point set.

[0063] Specifically, perform clustering algorithm calculation on the current lane line point set to obtain the clustering set of current several lane lines, and cluster multiple lane line clusters.

[0064] Step S22: Select several to-be-matched clustering points from the clustering result.

[0065] Specifically, the lane line compensation device selects a preset number of to-be-matched points from the clustered points.

[0066] Step S23: Match the several to-be-matched clustering points with the predicted lane line point set to obtain successfully matched clustering points.

[0067] Specifically, when the number of points that can be matched when matching several to-be-matched clustering points with the predicted lane line point set is greater than a certain threshold, it is considered a successful match; otherwise, it is considered a failed match.

[0068] In the matching process, 5 to-be-matched points are selected from the clustered points. If the number of matched points is greater than a certain threshold, it is considered a successful match; otherwise, it is considered a failed match.

[0069] Based on the inertial measurement unit, obtain the external parameters (including the rotation matrix and translation vector) of the vehicle from the vehicle coordinate system to the world coordinate system at time t, and obtain the corresponding position of the point information at time t in the world coordinate system. Combine the external parameters at time t+1 to obtain the prediction result of the point at time t at time t+1. At time t+1, sample, cluster, and match the current lane lines. When there is a clustering result that exists at time t but has no matching at time t+1, use the point information predicted at time t+1 based on the inertial measurement unit to compensate the clustering result at time t+1, and then use the fused clustering information to update the lane lines, thereby completing the lane line compensation at time t+1.

[0070] Step S24: When the number of successfully matched clustering points is less than the preset threshold, add the predicted lane line point set to the current lane line point set to generate the compensated lane line point set.

[0071] Match the clustering result at time t+1 with the lane lines generated at time t. If it is found that there are lane lines at time t but the number of clustering points matched at time t and time t+1 is less than the preset threshold, then use the prediction result of the point at time t at time t+1 for filling and compensation.

[0072] Specifically, in the embodiments of the present application, the lane line compensation device obtains the lane line numbers of the unsuccessfully matched clustering points based on the unsuccessfully matched clustering points; obtains the predicted lane line points associated with the lane line numbers of the unsuccessfully matched clustering points from the predicted lane line point set; and adds the predicted lane line points to the current lane line point set to generate the compensated lane line point set.

[0073] In the embodiments of the present application, when the number of successfully matched clustering points is greater than or equal to the preset threshold, the lane line numbers associated with the successfully matched clustering points are obtained, and according to the lane line numbers, the lane line equation of the current lane line image is fitted by using the current lane line points belonging to the same lane line number.

[0074] There is no matching phenomenon in the current clustering result. If such a phenomenon exists, check whether the IDs of the predicted points correspond one-to-one with the IDs of the lane lines that are not successfully matched. If there is a one-to-one correspondence, associate the predicted clustering result with the lane line, add the predicted result to the set of clustering result points to be updated, and finally fit all the clustering results to be updated to obtain the lane line fitting equation.

[0075] Step S15: Fit the lane line equation of the current lane line image according to the compensated lane line point set.

[0076] In the present application, by using the predicted lane line point set of the historical lane line image to compensate the current lane line point set, the interference of factors such as noise, occlusion, or illumination change on the current lane line detection can be effectively reduced, thereby improving the accuracy of lane line detection. In a complex environment, simply relying on the current image may lead to false detection or missed detection. By combining historical data, the system can better cope with these challenges and improve robustness. Using historical data for compensation can reduce the jitter of the lane line detection result, make the fitted lane line equation smoother, and help the vehicle control system make more stable decisions. By compensating the current lane line point set with the predicted lane line point set, the dependence on the current image processing can be reduced, thereby reducing the computational complexity and improving the real-time performance of the system. Through the compensation mechanism, the false detection and missed detection caused by poor image quality or environmental interference can be effectively reduced, and the reliability of lane line detection can be improved. More accurate and smooth lane line detection results help the autonomous driving system better plan the path and control the vehicle, improving driving safety and comfort.

[0077] The lane line fitting and updating of the present application uses the root mean square Kalman filter, and all Kalman filters are root mean square Kalman filters. The lane line fitting coefficients can be obtained from the matrix finally output by the Kalman filter. The finally generated lane line can be one or more. For the convenience of description, this example takes one lane line as an example to outline the root mean square Kalman filter process.

[0078] The prediction formulas (1) and (2) in the Kalman filter algorithm:

[0079] X(k|k - 1) = AX(k - 1|k - 1) + BU(k) (1)

[0080] P(k|k - 1) = AP(k - 1|k - 1)A T + Q (2)

[0081] Among them, X(k|k - 1) is the predicted value of the state variable, X(k - 1|k - 1) is the result of the state variable history, A is the state matrix of the Kalman filter, U(k) is the external input, P(k|k - 1) is the predicted value of the covariance matrix at the current detection time, P(k - 1|k - 1) is the covariance matrix at the historical detection time, and Q is the system noise covariance matrix (initialized with empirical values based on actual experience).

[0082] Among them,

[0083]

[0084] Among them, Δd = v c (k - 1)Δt, yawrate and v c are real - time vehicle body signals.

[0085] The lane line update process of this real - time example is shown in formulas (3), (4), and (5):

[0086] K k = P(k|k - 1)H T (HP(k|k - 1)H T + R) -1 (3)

[0087] X(k|k) = X(k|k - 1) + K k (Z(k) - HX(k|k - 1))(4)

[0088] P(k|k) = (I - K k H)P(k|k - 1)(5)

[0089] Among them, P(k|k - 1) is the predicted value of the covariance matrix at the current detection time, and its value can be calculated through formula (2). H is a preset observation matrix, which can be preset through experiments or empirical values. R is the measurement noise covariance, which can be determined through experiments or empirical constants.

[0090] Among them, X(k|k - 1) is used to represent the lane line at the current detection moment, X(k|k) is used to represent the final lane line, and P(k|k) is the covariance matrix, and its final value is used to predict the covariance matrix value of the current updated and fused lane line in the next frame. P(k|k - 1) is the covariance matrix calculated according to Formula 5 in the previous frame.

[0091] To improve the calculation efficiency and enhance the numerical stability in the calculation process of updating the state covariance matrix, the Cholesky decomposition factor P(k|k - 1) = Cp * Cp is used T to directly propagate and update the square root of the state covariance matrix. The advantage of this method is that it updates the covariance matrix P in a more stable way.

[0092] The formula mentioned in Equation (2) above is updated to Equation (6) after Cholesky decomposition:

[0093] Cp(k)Cp T (k) = A(x)Cp(k - 1)Cp T (k - 1)A(x) T +CqCq T (6)

[0094] Through the Kalman filter update mentioned above, the final covariance matrix and the final lane line fitting coefficient are obtained. The Kalman filter completes the selection of the single-frame lane line and also completes the tracking of the lane line. The Kalman filter algorithm combines the motion parameters of the vehicle itself during operation.

[0095] This application discloses a lane line compensation method based on an inertial measurement unit. The method includes: obtaining the real-time images of each vision sensor in real time, detecting and recognizing the lane lines in the images, and fitting the points after downsampling the recognition results to obtain a lane line equation; collecting the current attitude information of the vehicle body (including pitch angle, yaw angle, roll angle) based on the inertial measurement unit, and obtaining the historical rotation matrix and translation vector according to the attitude information mentioned above, sampling the historically fitted lane line equation to obtain some discrete points, and transforming these discrete points to the world coordinate system according to the obtained attitude, and then transforming the above points according to the attitude information at the current moment to obtain the prediction of the historical points at the current moment; according to whether the state of the current lane line is missing, whether the length jumps, etc., if it is found that there are scenes of missing and length jumps, the historical predicted points are put into the point set to be fitted, and the lane line is fitted to obtain a complete lane line, thus completing the lane line compensation. The present invention improves the stability of the lane line by predicting historical points based on the inertial measurement unit.

[0096] The method of the present application takes less time. Only one determination is needed before fitting, and then the compensation points and the detection results are fitted together. Only one fitting is required to complete the update compensation of the lane line. It can also achieve full scenarios, without relying on parallelism, nor determining the main lane of the lane line and other external conditions. Only by checking whether the fitting result is successfully matched can the compensation of the lane line be completed, which makes the output of the lane line more stable and reliable.

[0097] To implement the lane line compensation method of the above embodiment, the present application also provides a lane line compensation device. For details, please refer to Figure 6 , Figure 6 which is a schematic structural diagram of an embodiment of the lane line compensation device provided by the present application.

[0098] As Figure 6 shown, the lane line compensation device 400 includes an image acquisition module 41, an image processing module 42, and a calculation module 43;

[0099] The image acquisition module 41 is used to acquire the current lane line image;

[0100] The image processing module 42 is used to extract the current lane line point set of the current lane line image; acquire the predicted lane line point set of the historical lane line image at the current moment; use the predicted lane line point set to compensate the current lane line point set to obtain a compensated lane line point set;

[0101] The calculation module 43 is used to fit the lane line equation of the current lane line image according to the compensated lane line point set.

[0102] To implement the lane line compensation method of the above embodiment, the present application also provides a lane line compensation device. For details, please refer to Figure 7 , Figure 7 which is a schematic structural diagram of another embodiment of the lane line compensation device provided by the present application.

[0103] As Figure 7 shown, the lane line compensation device 600 of this embodiment includes a processor 61, a memory 62, an input / output device 63, and a bus 64.

[0104] The processor 61, the memory 62, and the input / output device 63 are respectively connected to the bus 64. The memory 62 stores a computer program, and the processor 61 is used to execute the computer program to implement the lane line compensation method of the above embodiment.

[0105] In this embodiment, the processor 61 can also be referred to as a CPU (Central Processing Unit). The processor 61 may be an integrated circuit chip with signal processing capabilities. The processor 61 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The processor 61 can also be a GPU (Graphics Processing Unit), also known as a display core, a visual processor, or a display chip, which is a microprocessor specifically for image computing on computers, workstations, game consoles, and some mobile devices (such as tablets, smartphones, etc.). The purpose of the GPU is to convert and drive the display information required by the computer system and provide a line scan signal to the display to control the correct display of the display. It is an important component connecting the display and the computer motherboard. The graphics card, as an important part of the computer host, undertakes the task of outputting and displaying graphics. The general-purpose processor can be a microprocessor or the processor 61 can also be any conventional processor, etc.

[0106] This application also provides a computer storage medium, as Figure 8 shown, the computer storage medium 700 is used to store a computer program 71. When the computer program 71 is executed by the processor, it is used to implement the method described in the embodiment of the lane line compensation method of this application.

[0107] The method involved in the embodiment of the lane line compensation method of this application, when implemented in the form of a software functional unit and sold or used as an independent product, can be stored in a device, such as a computer-readable storage medium. Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs, etc., which can store program codes.

[0108] The above are only the embodiments of the present invention, and do not thereby limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall similarly be included within the patent protection scope of the present invention.

Claims

1. A lane line compensation method, characterized in that, The lane line compensation method includes: Obtain the current lane line image; Extract the current lane line point set of the current lane line image; Obtain the predicted lane line point set of the historical lane line image at the current moment; Compensate the current lane line point set by using the predicted lane line point set to obtain a compensated lane line point set; Fit the lane line equation of the current lane line image according to the compensated lane line point set.

2. The lane line compensation method according to claim 1, wherein The step of compensating the current lane line point set by using the predicted lane line point set to obtain a compensated lane line point set includes: Obtain the clustering result of the current lane line point set; Select several to-be-matched clustering points from the clustering result; Match the several to-be-matched clustering points with the predicted lane line point set to obtain successfully matched clustering points; When the number of the successfully matched clustering points is less than a preset threshold, add the predicted lane line point set to the current lane line point set to generate the compensated lane line point set.

3. The lane line compensation method according to claim 2, wherein The step of adding the predicted lane line point set to the current lane line point set to generate the compensated lane line point set includes: Based on the unsuccessfully matched clustering points, obtain the lane line numbers of the unsuccessfully matched clustering points; Obtain the predicted lane line points associated with the lane line numbers of the unsuccessfully matched clustering points from the predicted lane line point set; Add the predicted lane line points to the current lane line point set to generate the compensated lane line point set.

4. The lane line compensation method according to claim 2, wherein The lane line compensation method further includes; When the number of the successfully matched clustering points is greater than or equal to the preset threshold, obtain the lane line numbers associated with the successfully matched clustering points; Fit the lane line equation of the current lane line image according to the current lane line points belonging to the same lane line number according to the lane line numbers.

5. The lane line compensation method according to claim 1, wherein The step of obtaining the predicted lane line point set of the historical lane line image at the current moment includes: Obtain the historical lane line points of the historical lane line image; Map the historical lane line points to the vehicle coordinate system of the current lane line image to generate a predicted lane line point set.

6. The lane line compensation method according to claim 5, wherein The step of mapping the historical lane line points to the vehicle coordinate system of the current lane line image includes: Obtain the first camera extrinsic parameters at the current moment and the second camera extrinsic parameters at the historical moment; Map the historical lane line points to the world coordinate system by using the second camera extrinsic parameters; Map the historical lane line points from the world coordinate system to the vehicle coordinate system of the current lane line image by using the first camera extrinsic parameters.

7. The lane line compensation method according to claim 5, wherein The step of obtaining the historical lane line points of the historical lane line image includes: Extract all lane line points of the historical lane line image and the fitted historical lane line; Downsample all the lane line points using the historical lane lines to obtain the historical lane line points.

8. A lane line compensation device, characterized in that, The lane line compensation device includes: an image acquisition module, an image processing module, and a calculation module; The image acquisition module is configured to acquire a current lane line image; The image processing module is configured to extract a set of current lane line points of the current lane line image; acquire a set of predicted lane line points of a historical lane line image at the current moment; compensate the set of current lane line points using the set of predicted lane line points to obtain a set of compensated lane line points; The calculation module is configured to fit a lane line equation of the current lane line image according to the set of compensated lane line points.

9. A lane line compensation device, characterized in that, The lane line detection and compensation device includes a memory and a processor coupled to the memory; Wherein, the memory is used to store program data, and the processor is used to execute the program data to implement the lane line compensation method according to any one of claims 1 to 7.

10. A computer storage medium, characterized in that, The computer storage medium is used to store program data, and the program data, when executed by a computer, is used to implement the lane line compensation method according to any one of claims 1 to 7.

Images