Automatic driving lane line processing method and device
By extracting and matching lane line data, combining the merge and interruption of historical frames and preset conditions, the problem of low noise signals and matching accuracy in autonomous driving lane line detection is solved, and the high accuracy and continuity of lane line is achieved , improving the driving safety of autonomous vehicles.
Patent Information
- Application Number
- CN202411056925.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-02
- Publication Date
- 2024-11-08
AI Technical Summary
The existing autonomous driving lane line detection technology has problems such as noise signals, low line segment matching accuracy, and poor line segment merging and interruption effects in complex environments, resulting in insufficient continuity and accuracy of target lane lines.
By acquiring the driving environment images, extracting lane line data based on the preset algorithm, performing depth-first search, clustering and fitting operations, combining historical frame data for matching, segment merge and interruption based on the preset conditions, and smoothing and The filtering algorithm de-jitter processing is used to finally determine the target lane line.
It improves the accuracy and robustness of lane line identification, ensures the continuity and smoothness of lane line detection, and provides a continuous and accurate target lane line for use by autonomous vehicles.
Smart Images

Figure CN118918556A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of autonomous driving technology, and in particular to an autonomous driving lane line processing method and device. Background Art
[0002] In the field of autonomous driving, lane detection is one of the key technologies to ensure safe driving of vehicles. Currently, lane detection mainly relies on the acquisition and processing of environmental images to identify and track lane lines on the road. However, existing technologies still face some technical problems in complex driving environments, including:
[0003] Existing autonomous driving systems use cameras to obtain images of the driving environment and use various image processing algorithms and deep learning techniques to extract lane line data from the images. However, due to environmental factors such as light, weather, and road conditions, the obtained lane line data often contains noise signals, resulting in inaccurate and unstable detection results.
[0004] After extracting lane line data, it needs to be further processed into line segment data and matched with the line segment data of the historical frame. Existing methods often have difficulty in handling line segment position deviations caused by environmental changes and sensor errors in this process, resulting in low matching accuracy and affecting subsequent lane line tracking and positioning.
[0005] In order to generate continuous lane lines, the extracted line segment data needs to be merged and interrupted. However, the existing technology often lacks effective strategies when processing line segment merging and interruption, resulting in the generated lane lines being not smooth enough, or even intermittent or incorrectly connected. Summary of the invention
[0006] The technical problem to be solved by the present application is to provide a method and device for processing lane lines in an autonomous driving system in view of the deficiencies in the prior art, so as to solve the problems of the existing autonomous driving lane line detection technology in complex environments, such as the presence of noise signals, low line segment matching accuracy, poor line segment merging and interruption effects, and insufficient continuity and accuracy of the target lane lines.
[0007] In order to solve the above technical problems, the first aspect of the embodiments of the present application provides a method for processing lane lines in an autonomous driving system, the method comprising:
[0008] Acquire a driving environment image, and obtain lane line data based on the environment image;
[0009] Extracting the lane line data to obtain first line segment data corresponding to the lane line;
[0010] Performing a matching operation on the first line segment data based on the line segment data of the historical frame to obtain matched second line segment data;
[0011] The second line segment data are integrated based on a preset condition to obtain third line segment data, and a target lane line is determined based on the third line segment data.
[0012] The process of obtaining lane line data based on the environment image includes:
[0013] A recognition operation is performed on the environment image based on a preset algorithm to obtain lane line data, where the lane line data contains a noise signal.
[0014] The process of extracting the lane line data to obtain the first line segment data corresponding to the lane line specifically includes:
[0015] Extracting regular line segment data from the lane line data;
[0016] Performing a screening operation on the line segment data to obtain screened line segment data;
[0017] Performing a de-redundancy operation on the filtered line segment data to obtain de-redundancy line segment data;
[0018] A fitting operation is performed on the de-redundant line segment data to obtain first line segment data, wherein the first line segment data includes multi-segment straight line and polynomial curve data.
[0019] The process of performing a matching operation on the first line segment data based on the line segment data of the historical frame to obtain the matched second line segment data specifically includes:
[0020] Acquire line segment data of the historical frame, wherein the line segment data of the historical frame includes multi-segment straight line data of the historical frame and polynomial curve data of the historical frame;
[0021] Matching the multi-segment straight line data of the first line segment data based on the multi-segment straight line data of the historical frame to obtain matched multi-segment straight line data;
[0022] Performing a matching operation on the polynomial curve data of the first line segment data based on the historical frame polynomial curve data to obtain matched polynomial curve data;
[0023] The second line segment data is determined based on the matched multi-segment straight line data and the matched polynomial curve data.
[0024] The process of integrating the second line segment data based on the preset condition to obtain the third line segment data specifically includes:
[0025] Based on the distance, slope and overlapping area of the second line segment data, each line segment in the second line segment data is merged and interrupted to obtain the third line segment data.
[0026] The process of integrating the second line segment data based on the preset condition to obtain the third line segment data further includes:
[0027] Calculating the confidence of each line segment in the second line segment data, and filtering the line segments based on a preset threshold and the confidence to obtain filtered line segments;
[0028] The filtered line segments are merged and interrupted to obtain third line segment data.
[0029] The process of determining the target lane line based on the third line segment data specifically includes:
[0030] Smoothing the third line segment data, and de-jittering the data in the third line segment based on a filtering algorithm to obtain a line segment to be fused;
[0031] The line segments to be fused are fused to determine the target lane line.
[0032] A second aspect of an embodiment of the present application provides an automatic driving lane line processing device, including:
[0033] An image acquisition module, used to acquire a driving environment image and obtain lane line data based on the environment image;
[0034] A first processing module, used for extracting the lane line data to obtain first line segment data corresponding to the lane line;
[0035] A second processing module, configured to perform a matching operation on the first line segment data based on the line segment data of the historical frame to obtain matched second line segment data;
[0036] The lane confirmation module is used to integrate the second line segment data based on a preset condition to obtain third line segment data, and determine the target lane line based on the third line segment data.
[0037] A third aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the autonomous driving lane line processing method.
[0038] A fourth aspect of an embodiment of the present application provides a terminal device, which includes: a processor and a memory;
[0039] The memory stores a computer-readable program executable by the processor;
[0040] When the processor executes the computer-readable program, the steps in the automatic driving lane line processing method are implemented.
[0041] Beneficial effects: The present invention overcomes the problem of lane line data noise signals in the prior art by acquiring driving environment images and extracting lane line data based on a preset algorithm, thereby improving the accuracy of lane line recognition;
[0042] The technical means of matching the current line segment data based on the line segment data of the historical frame overcomes the problem of low line segment matching accuracy, thereby improving the robustness and stability of lane line detection;
[0043] Secondly, the technical means of merging and interrupting line segment data based on preset conditions overcomes the problem of poor line segment merging and interrupting effects, thereby ensuring the continuity and smoothness of the generated lane lines;
[0044] Thirdly, by using smoothing and filtering algorithms to de-jitter and fuse the line segment data, the problems of insufficient continuity and accuracy of the target lane line are overcome, thereby providing a continuous and accurate target lane line for use by autonomous driving vehicles. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without inventive work.
[0046] Figure 1 A flowchart of an autonomous driving lane line processing method provided in this application.
[0047] Figure 2 An algorithm flow chart of an autonomous driving lane line processing method provided in this application.
[0048] Figure 3 The figure is a flowchart of the specific process of initially screening the line segments of the current frame.
[0049] Figure 4 A flowchart showing the specific process of merging short lines that are approximately at the beginning and the end.
[0050] Figure 5 This is a flowchart of the specific process of Track I tem matching processing.
[0051] Figure 6 A flowchart of the specific process of processing the matched Track Item.
[0052] Figure 7 A flowchart of the specific process of merging Track I tem.
[0053] Figure 8 This is a structural schematic diagram of an autonomous driving lane line processing method device provided in this application.
[0054] Fig. 9 This is a schematic diagram of the structure of the terminal device provided in this application.
[0055] Description of Figure Numbers:
[0056] 100. Image acquisition module;
[0057] 200, first processing module;
[0058] 300, second processing module;
[0059] 400. Lane confirmation module;
[0060] 20. Processor;
[0061] 21. Display screen;
[0062] 22. Memory;
[0063] 23. Communication interface;
[0064] 24. Bus. DETAILED DESCRIPTION
[0065] The present application provides a method and device for processing lane lines in an autonomous driving system. To make the purpose, technical solution and effect of the present application clearer and more specific, the present application is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0066] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.
[0067] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.
[0068] It should be understood that the sequence numbers and sizes of the steps in this embodiment do not mean the order of execution. The execution order of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.
[0069] Autonomous driving: refers to the technology that allows vehicles to perceive the environment through built-in sensors, cameras, radars and other equipment without human intervention, and achieve autonomous driving through computer algorithms.
[0070] Lane detection: refers to the autonomous driving system obtaining road images through cameras or other sensors, and using image processing or deep learning algorithms to identify lane lines on the road.
[0071] Depth-first search (DFS): An algorithm for traversing or searching a tree or graph data structure by going as deep as possible down the branches of the tree until the goal is found or all nodes are traversed.
[0072] Non-maximum suppression (NMS): A technique used to eliminate duplicate detections in image processing by retaining the detection with the highest confidence and suppressing similar detections around it.
[0073] In the field of autonomous driving, lane detection is one of the key technologies to ensure safe driving of vehicles. Currently, lane detection mainly relies on cameras to obtain environmental images and recognize lane data through image processing and deep learning technology. However, these methods still face many technical problems in practical applications.
[0074] First, there is the problem of noise signals. In complex driving environments, factors such as light changes, weather effects, and complex road conditions can cause the image quality obtained by the camera to be unstable, resulting in a large amount of noise signals in the lane line detection results. These noise signals interfere with the accurate recognition of lane lines and reduce the reliability of detection.
[0075] Secondly, there is the problem of low segment matching accuracy. In the prior art, the extracted lane line data needs to be matched with the data of the historical frame to improve the continuity and stability of the detection. However, due to the influence of environmental changes and sensor errors, the accuracy of segment matching is often low, resulting in errors in lane line tracking and positioning.
[0076] In addition, there is the problem of poor segment merging and interruption effects. When generating the final lane line, the extracted segment data needs to be merged and interrupted to ensure the continuity and smoothness of the lane line. However, the existing methods lack effective strategies, resulting in the generated lane lines often being discontinuous or incorrectly connected.
[0077] Finally, there is the problem of insufficient continuity and accuracy of the target lane line. When comprehensively processing lane line data after multiple steps, existing methods are difficult to ensure the continuity and accuracy of lane lines, which in turn affects the path planning and control of autonomous vehicles.
[0078] In order to solve the above problems, an embodiment of the present application discloses a method and device for processing lane lines in an autonomous driving, the method comprising: acquiring a driving environment image, obtaining lane line data based on the environment image; extracting the lane line data to obtain first line segment data corresponding to the lane line; matching the first line segment data based on the line segment data of the historical frame to obtain the matched second line segment data; integrating the second line segment data based on preset conditions to obtain third line segment data, and determining the target lane line based on the third line segment data. The method of the present invention improves the robustness and stability of lane line detection, thereby ensuring that the autonomous driving vehicle can still accurately identify and track lane lines in complex environments and improves driving safety.
[0079] The application content is further explained below through the description of embodiments in conjunction with the accompanying drawings.
[0080] This embodiment provides a method and device for processing lane lines in an autonomous driving system. Figure 1 As shown, the method includes:
[0081] S100: Acquire a driving environment image, and obtain lane line data based on the environment image.
[0082] Specifically, the process of obtaining lane line data based on the environment image includes:
[0083] A recognition operation is performed on the environment image based on a preset algorithm to obtain lane line data, where the lane line data contains a noise signal.
[0084] First, a high-resolution camera is used to capture images of the driving environment in real time. Generally speaking, in an autonomous driving system, the vehicle is equipped with a set of high-resolution cameras installed in different locations of the vehicle, such as the front windshield, roof, and rearview mirrors. The high-resolution camera can capture images of the driving environment in real time at a high frame rate, covering a 360-degree field of view around the vehicle.
[0085] The image of the driving environment is processed by a computing unit on the vehicle. The image captured by the camera is transmitted to the on-board computing unit via a high-speed data bus. The computing unit is usually composed of a high-performance central processing unit (CPU) and a graphics processing unit (GPU), which can perform real-time image processing. The image data also includes preliminary compression and decompression processing during the transmission process to ensure the efficiency and stability of data transmission.
[0086] When the on-board computing unit receives the image data, it first performs decoding processing to restore the original resolution and quality of the image. Through high-resolution cameras and efficient data processing units, the system can obtain clear and detailed images of the driving environment, providing high-quality basic data for subsequent image analysis and lane line detection.
[0087] After obtaining the processed driving environment image, the captured image is preprocessed. The preprocessing operation may include one or more of a filtering operation, a contrast enhancement operation, and a brightness adjustment operation. Specifically, the filtering operation is used to remove random noise and interference signals in the image and retain the main feature information of the image, wherein the filtering operation may adopt an image denoising algorithm such as a Gaussian filter and a median filter. The contrast enhancement operation is used to enhance the contrast between the lane line and the background in the image to enhance the clarity of the lane line, wherein the contrast enhancement operation may adopt techniques such as histogram equalization and adaptive contrast enhancement. The brightness adjustment operation is used to improve the uniformity and stability of the image brightness, wherein when performing a brightness adjustment operation on the image, the ambient lighting information corresponding to the image may be first obtained, and then the image brightness may be dynamically adjusted based on the ambient lighting information to ensure the uniformity and stability of the image brightness under different lighting environments (such as daytime, night, and cloudy days).
[0088] This application improves the quality and usability of the image by adopting the above-mentioned preprocessing operation, and provides more accurate and clearer input data for the subsequent lane line recognition algorithm, thereby improving the accuracy of the extracted lane line data.
[0089] Furthermore, after preprocessing the image, lane line data can be extracted from the preprocessed image, wherein the lane line data can be extracted using a deep learning method. Specifically, the preprocessed image data is input into a trained deep learning algorithm, and lane line recognition is performed in the deep learning algorithm to obtain lane line data, wherein the deep learning algorithm is usually composed of a convolutional neural network (CNN), which can automatically extract features from the image and perform classification and detection. The neural network model is trained with a large amount of driving environment image data so that it can learn and recognize the features of the lane lines. The training process includes feature extraction, model optimization, and parameter adjustment to ensure that the model has high recognition accuracy and robustness in practical applications.
[0090] After the model inputs the preprocessed image, it gradually extracts the lane line features in the image through multiple layers of convolution, pooling and activation operations. The lane line features include the shape, color, position and other information of the lane line. Based on the extracted features, the model outputs the location information of the lane line, which includes the starting point, end point and path curve of each lane line. The location information data is represented in the form of coordinate points or polynomial curves as the basis for subsequent processing.
[0091] It should be noted that the identified lane line data may contain some noise signals: In actual applications, the lane line recognition results may contain some noise signals. These noise signals may come from light changes, weather conditions (such as rain and snow), or other objects on the road (such as shadows, road signs).
[0092] In order to eliminate these noise signals, filtering and denoising algorithms can be introduced to further process the identified lane line data. Use Kalman filtering, mean filtering and other technologies to smooth the lane line data, eliminate short-term fluctuations and jitters caused by environmental changes, and maintain the continuity of the lane line. And use the non-maximum suppression (NMS) algorithm to filter and optimize the lane line data. The NMS algorithm retains the lane line detection results with the highest confidence and removes repeated detections with lower confidence, ensuring that the final output lane line data is accurate and reliable. Through the above noise processing steps, the system can remove the noise signals in the lane line recognition results, generate clear and stable lane line data, and provide a reliable path planning reference for autonomous driving vehicles.
[0093] Of course, in practical applications, other methods may be used to extract lane line data, for example, lane line data may be extracted using a traditional search algorithm (such as a depth-first search (DFS) algorithm).
[0094] S200: Extract the lane line data to obtain first line segment data corresponding to the lane line.
[0095] Specifically, the process of extracting the lane line data to obtain the first line segment data corresponding to the lane line specifically includes:
[0096] S210, extracting regular line segment data from the lane line data;
[0097] S211, performing a screening operation on the line segment data to obtain screened line segment data;
[0098] S212, performing a de-redundancy operation on the filtered line segment data to obtain de-redundancy line segment data;
[0099] S213. Perform a fitting operation on the de-redundant line segment data to obtain first line segment data, wherein the first line segment data includes multi-segment straight line and polynomial curve data.
[0100] In step S210, regular line segment data is extracted from lane line data, wherein the regular line segment data can be extracted according to an algorithm. Specifically, the regular line segment data can be based on a depth-first search (DFS) algorithm and / or a clustering algorithm, wherein the depth-first search (DFS) algorithm is used to search for candidate lane line data in lane line data that meets preset conditions, and the clustering algorithm is used to cluster the candidate lane line data. The embodiment of the present application can filter out noise signals in the environmental image and improve the accuracy and consistency of the lane line data by extracting line segment data from the lane line data.
[0101] In step S211, the regular line segment data is screened. The screening operation mainly includes two aspects: first, the line segment length is screened according to the preset threshold value to remove the line segments that are too short or too long; second, the line segment is screened according to the direction and position to ensure that the screened line segments meet the characteristics of the actual lane line, thereby improving the accuracy of the line segment data and ensuring the effectiveness of the subsequent processing steps.
[0102] In step S212, the de-redundancy operation may adopt a non-maximum suppression (NMS) algorithm, that is, the non-maximum suppression (NMS) algorithm is used to perform a de-redundancy operation on the filtered line segment data. The NMS algorithm effectively removes redundant line segments by retaining the line segments with the highest confidence and suppressing similar line segments around them, thereby retaining important line segment information and improving the purity and accuracy of the data.
[0103] In step S213, the de-redundant line segment data is fitted. A preset fitting method is used, and the fitting method includes multi-segment straight line fitting and polynomial curve fitting. By fitting the line segment data, the discrete line segment data can be converted into a continuous straight line or curve expression. Multi-segment straight line fitting is suitable for shorter lane segments that are approximately straight, while polynomial curve fitting is suitable for longer lane segments with large curvature changes. The expression accuracy and stability of the lane line are improved by the fitting method.
[0104] It should be noted that in the process of autonomous driving lane line detection, straight line fitting is an important step in processing lane line data. Traditional straight line fitting methods usually use the polynomial form of y=ax+b, because this form is simple and easy to understand. However, in practical applications, especially in autonomous driving scenarios, this fitting form has some limitations.
[0105] Dependence on a specific coordinate system: The form of y=ax+b requires a specific coordinate system and assumes that y is a function of the independent variable x. This is not applicable when the road environment is complex and changeable.
[0106] Insufficient ability to handle vertical lines: When the lane line is close to vertical, the form of y = ax + b will cause the slope a to become very large, which makes the fitting result unstable and has large errors.
[0107] In the specific embodiment of the present application, the mathematical form of ax+by=c is used for straight line fitting. The method has the following advantages:
[0108] Coordinate system independence: The form of ax+by=c does not depend on a specific coordinate system, so it can more flexibly adapt to different road environments and lane directions. It is more suitable for application scenarios that need to handle a variety of complex road conditions.
[0109] Processing vertical line segments: The mathematical form of ax+by=c can well process nearly vertical line segments because it does not depend on the size of the slope, thereby avoiding the numerical instability problem that may occur in the form of y=ax+b when processing vertical line segments.
[0110] The steps of the mathematical form of ax+by=c are as follows:
[0111] Data preprocessing: Preprocess the deduplicated line segment data and extract the endpoint coordinates of each line segment.
[0112] Linear fitting: Use the least squares method and other optimization algorithms to fit the linear equation of ax+by=c according to the endpoint coordinates. The specific steps are as follows:
[0113] For each line segment data, a linear equation system is established to minimize the error and solve the optimal values of a, b and c.
[0114] Use the solved a, b, and c parameters to describe the equation of the line segment to achieve straight line fitting.
[0115] Because the mathematical form of ax+by=c is used for straight line fitting, the limitations of the form of y=ax+b in dealing with vertical line segments and complex road environments are overcome. The specific effects are as follows:
[0116] Improve adaptability: Since the form of ax+by=c is independent of the coordinate system, the adaptability and robustness of the system under different road conditions are improved.
[0117] Enhanced stability: When dealing with nearly vertical line segments, the fitting results are more stable, avoiding errors caused by excessive slopes, thereby improving fitting accuracy.
[0118] Optimize computational efficiency: This form is more suitable for the application of numerical calculations and optimization algorithms, thereby improving the overall computational efficiency and thus improving the real-time performance of lane line detection.
[0119] Through the above technical means, this embodiment can solve the problems of noise signals and low line segment matching accuracy in complex environments in existing autonomous driving lane line detection technology. Specifically:
[0120] The lane line data is quickly extracted through the DFS algorithm, and similar line segments are clustered together using the clustering algorithm, which overcomes the problem of noise signals and improves the accuracy and consistency of the lane line data.
[0121] By screening the length and direction of regular line segment data, we ensure that the screened line segments meet the characteristics of actual lane lines, further improving the accuracy of the line segment data.
[0122] The NMS algorithm is used to remove redundant line segments, retain important line segment information, overcome the problem of high data redundancy, and thus improve the purity and accuracy of line segment data.
[0123] By adopting multi-segment straight line fitting and polynomial curve fitting methods, discrete line segment data is converted into a continuous expression form, thereby improving the expression accuracy and stability of lane lines.
[0124] S300: performing a matching operation on the first line segment data based on the line segment data of the historical frame to obtain matched second line segment data.
[0125] Specifically, the process of performing a matching operation on the first line segment data based on the line segment data of the historical frame to obtain the matched second line segment data specifically includes:
[0126] S310, acquiring line segment data of the historical frame, wherein the line segment data of the historical frame includes multi-segment straight line data of the historical frame and polynomial curve data of the historical frame;
[0127] S311, matching the multi-segment straight line data of the first line segment data based on the multi-segment straight line data of the historical frame to obtain matched multi-segment straight line data;
[0128] S312, performing a matching operation on the polynomial curve data of the first line segment data based on the historical frame polynomial curve data to obtain matched polynomial curve data;
[0129] S313, determining second line segment data based on the matched multi-segment straight line data and the matched polynomial curve data.
[0130] Specifically, the line segment data of the historical frame is determined based on the historical image corresponding to the image, and the acquisition time point of the historical image is earlier than the acquisition time point of the image. Among them, the line segment data of the historical frame includes multi-segment straight line data of the historical frame and polynomial curve data of the historical frame. In order to ensure the comprehensiveness of the data, the historical frame data needs to cover a variety of driving environments and road conditions, including lane line data under different conditions such as day, night, rainy days, and sunny days. By using the historical frame data, a reference and verification of the line segment data of the current frame can be provided to ensure the continuity and accuracy of the matching process.
[0131] Furthermore, the line segment data of the historical frame may be determined and stored when the automatic driving lane line processing method provided by the embodiment of the present application is executed on the historical frame, or may be obtained by extracting the line segment data of the historical frame corresponding to the image when processing the current image, wherein the line segment extraction process is the same as the above-mentioned line segment extraction process and is not described in detail here. In a specific implementation, the acquisition process of the line segment data of the historical frame may specifically include: reading lane line detection data of historical frames at multiple time points from a storage device, and classifying the read lane line detection data according to the line segment type to obtain multi-segment straight line data and polynomial curve data.
[0132] The embodiment of the present application uses the line segment data of the historical frame as prior knowledge, which can provide a stable reference basis, reduce the instability caused by noise and errors in the single frame data, and thus improve the reliability of matching. In addition, the diversity and comprehensiveness of the historical frame data enhances the algorithm's adaptability to lane lines in different environments, further improving the robustness of lane line detection.
[0133] Next, the multi-segment straight line data in the historical frame is matched with the first line segment data in the current frame. The matching operation is performed by comparing the starting point, end point, slope and other parameters of the line segment to determine the corresponding relationship between the two.
[0134] Specifically, the matching step includes:
[0135] 1. Calculate the starting point, end point, slope and other characteristic parameters of the multi-segment straight line data of the current frame.
[0136] 2. Compare these parameters with the corresponding parameters of the multi-segment straight line data in the historical frame.
[0137] 3. Use the least squares method or other optimization algorithms to determine the optimal matching relationship and generate matched multi-segment straight line data.
[0138] By comparing the multi-segment straight line data of the historical frame and the current frame, the corresponding line segment can be accurately found, the mismatch caused by environmental changes can be reduced, the matching accuracy of the straight line data can be improved, and the robustness of lane line detection can be improved. The matching operation can effectively deal with slight deviations caused by vehicle movement and environmental changes, ensuring the continuity and stability of lane line detection.
[0139] On the basis of matching the straight line data, the polynomial curve data in the historical frame is further matched with the polynomial curve data in the current frame. The matching process of the polynomial curve data in the historical frame and the polynomial curve data in the current frame is achieved by comparing the parameters (such as coefficients, curvatures, etc.) of the curves, wherein the matching step includes:
[0140] 1. Extract characteristic parameters such as coefficients and curvature of the polynomial curve data of the current frame.
[0141] 2. Compare the parameters with the corresponding parameters of the polynomial curve data in the historical frame.
[0142] 3. Use a curve fitting algorithm, such as spline curve fitting or Bezier curve fitting, to determine the optimal matching relationship and generate matched polynomial curve data.
[0143] By matching the polynomial curve data of the historical frame and the current frame, the changing trend of complex lane lines can be accurately captured, overcoming the limitations of the simple straight line matching method and improving the comprehensiveness and accuracy of lane line detection. The method is particularly suitable for processing lane line data of curved and curved sections, ensuring that lane lines can be accurately detected and tracked under various road conditions.
[0144] Finally, the matched multi-segment straight line data and polynomial curve data are comprehensively matched to determine the second line segment data of the current frame. The second line segment data is the result of historical frame verification and matching optimization, and has the characteristics of high accuracy and strong stability.
[0145] Specifically, the comprehensive step includes:
[0146] 1. Merge the matched multi-segment straight line data and polynomial curve data.
[0147] 2. According to the continuity and connectivity of the line segments, adjust and optimize the fusion results to ensure that the generated line segment data is smooth and has no breakpoints.
[0148] 3. Output the final second line segment data as the result of lane line detection in the current frame.
[0149] By combining the matching results of multi-segment straight line data and polynomial curve data, the deficiencies caused by a single matching method can be effectively eliminated, ensuring that the final second line segment data has both the stability of a straight line and the flexibility of a curve. This improves the overall accuracy and reliability of lane line detection. Ultimately, the generated second line segment data provides a more stable, continuous and accurate lane line reference for the autonomous driving system, improving the driving safety and stability of autonomous vehicles in complex road environments.
[0150] S400: Integrate the second line segment data based on a preset condition to obtain third line segment data, and determine a target lane line based on the third line segment data.
[0151] Specifically, the integration includes performing a breaking operation and / or a merging operation on the second line segment data, wherein the merging operation is used to merge two or more second line segment data into one second line segment data, and the breaking operation is used to divide the second line segment data into two or more second line segment data.
[0152] In one implementation, the step of integrating the second line segment data based on a preset condition to obtain the third line segment data specifically includes:
[0153] S401, calculating the confidence of each line segment in the second line segment data, and filtering the line segments based on a preset threshold and the confidence to obtain filtered line segments;
[0154] S402: Merge and break the filtered line segments to obtain third line segment data.
[0155] Specifically, the confidence level is used to reflect the credibility of the second line segment data. When the confidence level is greater than a preset threshold, it indicates that the second line segment data is credible, and the second line segment data is retained. Conversely, when the confidence level is less than or equal to the preset threshold, it indicates that the second line segment data is uncredible, and the second line segment data is filtered out to obtain the filtered line segment.
[0156] Furthermore, when merging and interrupting the filtered line segments, the distance, slope and overlapping area of the second line segment data can be analyzed to determine which line segments need to be merged and which need to be interrupted, thereby solving the problems of discontinuity and discontinuity of line segments, as well as the problems of incorrect connection and overlap of line segments. Specifically, through the merging operation, the discontinuity of lane lines caused by the discontinuity of line segments is eliminated, and the smoothness and continuity of lane lines are improved. And through the interruption operation, the incorrect connection of line segments is corrected to ensure the accuracy and correctness of lane lines.
[0157] The embodiment of the present application adopts merging and breaking operations based on distance, slope and overlapping area, so as to overcome the problems of discontinuity and wrong connection in the line segment data, and thus obtain more continuous and smooth third line segment data.
[0158] In step S401, before the merging and breaking operations, it is necessary to calculate the confidence of each line segment in the second line segment data. The specific steps are as follows:
[0159] For each line segment, its confidence is calculated based on its features in the image and its matching with historical data.
[0160] Based on the preset confidence threshold, line segments with confidence levels lower than the threshold are filtered to remove unreliable line segment data.
[0161] Through the filtering operation, the negative impact of low-confidence line segments on the overall quality of the lane lines is reduced, and noise and false detection in the data are avoided.
[0162] After completing the confidence filtering, the filtered line segment data is merged and interrupted. The specific steps of the merging and interrupting operations are as follows:
[0163] 1. Merge operation: Based on the distance, slope and overlapping area of the line segments, determine whether adjacent or overlapping line segments need to be merged, and merge the qualified line segments into a continuous line segment.
[0164] 2. Interruption operation: For long line segments, interruption operation is performed according to preset conditions to divide them into multiple short line segments to improve the matching and tracking accuracy of the line segments.
[0165] Through the above means, the problem of low tracking and matching accuracy of long line segments and the data redundancy and complexity caused by multiple short line segments are solved.
[0166] Specifically, the merging operation reduces the number of line segments, reduces data complexity, and improves the continuity and smoothness of lane lines. The interruption operation divides long line segments into short line segments that are easier to match and track, improving the matching accuracy and tracking stability of line segments.
[0167] Because of the merging and interruption operations, the problem of low tracking and matching accuracy of long line segments is overcome, while data redundancy is reduced, thereby obtaining a set of third line segment data that is both continuous and smooth, providing higher quality basic data for determining the target lane line.
[0168] The process of determining the target lane line based on the third line segment data specifically includes:
[0169] S410, performing smoothing processing on the third line segment data, and performing de-jitter processing on the data in the third line segment based on a filtering algorithm to obtain a line segment to be fused;
[0170] S411, performing fusion processing on the line segments to be fused to determine a target lane line.
[0171] For step S410, in an actual driving environment, the collected lane line data is often affected by factors such as environmental noise, road sign wear and sensor errors, resulting in jitter and unevenness in the lane line data, and thus generating uneven and jittery data. The uneven and jittery data will directly affect the path planning and control of the autonomous driving vehicle, and further affect the safety and comfort of driving.
[0172] Specifically, by smoothing the third line segment data, the mutation points in the line segment can be effectively removed, making the line segment smoother and more continuous. At the same time, the filtering algorithm is applied to the line segment data to de-jitter, further eliminating high-frequency noise and jitter in the data. Specifically, common filtering algorithms such as Gaussian filtering and Kalman filtering can be used to process the line segment data, making the line segment data more stable and smooth.
[0173] After smoothing and de-jittering, as described in step S411, the line segment data becomes more stable and smooth, but there may still be multiple line segments that need to be merged into a complete lane line. There may be some slight deviations between the multiple line segments, and these line segment data need to be integrated into a continuous lane line through fusion processing.
[0174] In the fusion process, a fusion algorithm based on line segment features is used to process the line segments to be fused. Specifically, the following steps are included:
[0175] Determine which line segments can be fused based on their spatial position, direction, length and other features.
[0176] Adjacent line segments are fused and the fused line segment data is calculated using methods such as least squares method and weighted average.
[0177] The fused line segments are fitted to ensure that the final generated target lane line has good continuity and accuracy.
[0178] By using a fusion algorithm to process the fused line segments, the deviation problem between multiple line segments is solved, the continuity and consistency of the line segment data are ensured, and a complete target lane line is generated. The target lane line can provide more accurate and reliable lane line information for the autonomous driving system, improve the path planning and control accuracy of the vehicle, and thus improve the safety and comfort of driving.
[0179] Through the coordination of the above steps S410 and S411, the problems of jitter, unevenness and difficulty in multi-segment fusion of lane line data in the prior art are overcome, ensuring that the target lane line finally generated has high precision and high stability.
[0180] In the specific implementation of the present application, the method can also be implemented as follows: Figure 2 The algorithm flow is implemented, specifically:
[0181] 1. Preliminary extraction of lane line data
[0182] Input image data of the driving environment and preliminarily extract lane line data.
[0183] 2. Preliminary screening of line segments in the current frame, specifically, Figure 3 The line segments of the current frame are initially screened as follows:
[0184] Check whether the line segment of the current frame meets the preset condition, which is that the line segment is in the intersection area and is approximately parallel to the XY axis of the vehicle. The line segment that meets the preset condition is considered to be an unobstructed and observable line segment of the current lane. A high credibility can be set for the line segment that meets the preset condition so that the line segment that meets the preset condition can be trusted.
[0185] · Calculate the confidence threshold coefficient according to the distance between the line segment and the vehicle, wherein the confidence threshold coefficient of the line segment with a small distance is large, and the confidence threshold coefficient of the line segment with a large distance is small, so that the line segment with a small distance can be trusted.
[0186] Estimate the length of line segments and delete line segments with extreme lengths (e.g., greater than a preset length threshold, etc.).
[0187] Create the Cur Item (current project) after the initial screening.
[0188] 3. Non-maximum suppression processing
[0189] Perform non-maximum suppression on the line segments after the initial screening. If there is a large crossover expectation, retain the line segments with high confidence.
[0190] 4. Merge short lines that are close to the beginning and the end. Specifically, Figure 4 As shown, the short lines that merge the approximate beginning and end specifically include:
[0191] Merge the short lines Cur Item that are close to the beginning and the end according to the beginning and the end merging conditions, where:
[0192] The head-to-tail merging conditions include that the head-to-tail distance is close; and the derivative direction of a group of close endpoints does not exceed the angle threshold.
[0193] If there is a curve in the merged Cur Item, refit the curve using the confidence as the weight, otherwise fit a straight line.
[0194] Resample points on the newly fitted line or curve, generate a new Cur Item, and delete the original Cur Item;
[0195] Continue to match with other Cur Items. When a match is possible, re-merge the short Cur Items that are close to the beginning and the end according to the head-to-tail merging condition. When a match cannot be made, perform Track Item matching with Cur Item.
[0196] 5. Track Item and Cur Item matching
[0197] Match Track Item with Cur Item.
[0198] ·Transform the Track Item back to the current coordinate system and delete the part that exceeds the current ROI (region of interest).
[0199] 6. Track I tem matching process, specifically, Figure 5 As shown, the matching process of Track I tem specifically includes:
[0200] Match according to the line shape of Track Item, specifically,
[0201] When Track Item is a curve, the polygonal intersection area of Track Item and Cur Item is calculated, and it is considered a match if it exceeds the threshold.
[0202] Calculate the polygonal intersection area of the two lines, the distance and angle difference between the endpoints to each other, and determine the matching result based on the threshold.
[0203] For each Cur Item, only the best matching Track Item is retained. For each Track Item, multiple Cur Items can be matched.
[0204] According to the matching results, use the Cur Item of the latest frame to update the Track Item.
[0205] 7. Process the matching Track I tem, specifically, Figure 6 As shown, processing the matching Track I tem specifically includes:
[0206] For a Track Item that matches two straight lines, if the two matched Cur Items are both straight line segments and the confidence level is higher than the threshold, the track Item is interrupted.
[0207] For the matched Track Item, refit the straight line or curve with different weights according to the matching results, resample the points based on the new fitting function, and generate a new Track Item.
[0208] For unmatched Cur Item, a secondary screening is performed with a higher confidence threshold. If it passes the screening, it is added to the Track Item.
[0209] 8. Merge Track I tem, specifically, Figure 7 As shown, merging Track I tem specifically includes:
[0210] If the Track Items can be merged, they are merged; if the overlap is very high, the Track Item with high confidence is retained.
[0211] Merge the Track Items according to the merging conditions, where the merging conditions are:
[0212] The intersection area is large enough;
[0213] The angle difference is small.
[0214] Refit the line according to the weights and resample to generate a new Track Item.
[0215] Continue matching with other Track Items until no match is found (if a match is found, return to the merging step).
[0216] 9. Delete untracked Track Item
[0217] ·Delete TrackItems that have not been tracked for multiple frames.
[0218] 10. Track Item non-maximum suppression processing
[0219] Perform non-maximum suppression on Track Item, and retain Track Item with high confidence for those with large intersection areas.
[0220] 11. Output the final Cur Item
[0221] Output the final Cur Item as the target lane line data.
[0222] In summary, the present embodiment provides a method and device for processing lane lines in an autonomous driving system. The present application obtains a driving environment image, processes the environment image based on a preset algorithm, obtains lane line data containing noise signals, and further performs depth-first search, clustering algorithm, non-maximum suppression and fitting operations on these data to extract regular first line segment data. Matching operations are performed on these first line segment data based on the line segment data of the historical frame to obtain matched second line segment data. Then, based on preset conditions, these second line segment data are merged and interrupted to obtain third line segment data. Furthermore, by performing smoothing processing and filtering algorithm de-jitter processing on the third line segment data, the line segments to be fused are obtained, and finally these line segments to be fused are fused to determine the target lane line.
[0223] The technical solution of the present application processes the environmental image through a preset algorithm, reduces the noise signal in the lane line data, and improves the accuracy of lane line recognition; overcomes the problem of low segment matching accuracy through matching operations between historical frame data and current frame data, and improves the robustness and stability of lane line detection; merges and interrupts the segment data based on preset conditions to solve the problem of poor segment merging and interruption effects in the prior art, and ensures that the generated lane line has high continuity and smoothness; de-jitters the third segment data through smoothing and filtering algorithms, and further determines the target lane line through fusion processing, which solves the problem of insufficient continuity and accuracy of the target lane line and provides a continuous and accurate lane line for use by autonomous driving vehicles.
[0224] Based on the above-mentioned automatic driving lane line processing method, this embodiment provides an automatic driving lane line processing device, such as Figure 8 As shown, the device comprises:
[0225] An image acquisition module 100 is used to acquire a driving environment image and obtain lane line data based on the environment image;
[0226] A first processing module 200 is used to extract the lane line data to obtain first line segment data corresponding to the lane line;
[0227] The second processing module 300 is used to perform a matching operation on the first line segment data based on the line segment data of the historical frame to obtain matched second line segment data;
[0228] The lane confirmation module 400 is used to integrate the second line segment data based on a preset condition to obtain third line segment data, and determine the target lane line based on the third line segment data.
[0229] Based on the above-mentioned autonomous driving lane line processing method, this embodiment provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in the autonomous driving lane line processing method as described in the above-mentioned embodiment.
[0230] Based on the above-mentioned automatic driving lane line processing method, the present application also provides a terminal device, such as Fig. 9 As shown, it includes at least one processor 20; a display screen 21; and a memory 22, and may also include a communication interface 23 and a bus 24. The processor 20, the display screen 21, the memory 22 and the communication interface 23 may communicate with each other through the bus 24. The display screen 21 is configured to display a preset user guide interface in the initial setting mode. The communication interface 23 may transmit information. The processor 20 may call the logic instructions in the memory 22 to execute the method in the above embodiment.
[0231] In addition, the logic instructions in the memory 22 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.
[0232] The memory 22 is a computer-readable storage medium that can be configured to store software programs, computer executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions or modules stored in the memory 22, that is, implementing the methods in the above embodiments.
[0233] The memory 22 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 22 may include a high-speed random access memory and may also include a non-volatile memory. For example, a variety of media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, may also be a transient storage medium.
[0234] In addition, the specific process of loading and executing the multiple instruction processors in the above-mentioned storage medium and the terminal device has been described in detail in the above-mentioned method, and will not be described one by one here.
[0235] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A method for processing lane lines in an automatic driving system, characterized in that: The method comprises: Acquire a driving environment image, and obtain lane line data based on the environment image; Extracting the lane line data to obtain first line segment data corresponding to the lane line; Performing a matching operation on the first line segment data based on the line segment data of the historical frame to obtain matched second line segment data; The second line segment data are integrated based on a preset condition to obtain third line segment data, and a target lane line is determined based on the third line segment data.
2. The method for processing lane lines for autonomous driving according to claim 1, characterized in that: The process of obtaining lane line data based on the environment image includes: A recognition operation is performed on the environment image based on a preset algorithm to obtain lane line data, where the lane line data contains a noise signal.
3. The method for processing lane lines for automatic driving according to claim 1, characterized in that: The process of extracting the lane line data to obtain the first line segment data corresponding to the lane line specifically includes: Extracting regular line segment data from the lane line data; Performing a screening operation on the line segment data to obtain screened line segment data; Performing a de-redundancy operation on the filtered line segment data to obtain de-redundancy line segment data; A fitting operation is performed on the de-redundant line segment data to obtain first line segment data, wherein the first line segment data includes multi-segment straight line and polynomial curve data.
4. The method for processing lane lines for autonomous driving according to claim 1, characterized in that: The process of performing a matching operation on the first line segment data based on the line segment data of the historical frame to obtain the matched second line segment data specifically includes: Acquire line segment data of the historical frame, wherein the line segment data of the historical frame includes multi-segment straight line data of the historical frame and polynomial curve data of the historical frame; Matching the multi-segment straight line data of the first line segment data based on the multi-segment straight line data of the historical frame to obtain matched multi-segment straight line data; Performing a matching operation on the polynomial curve data of the first line segment data based on the historical frame polynomial curve data to obtain matched polynomial curve data; The second line segment data is determined based on the matched multi-segment straight line data and the matched polynomial curve data.
5. The method for processing lane lines for autonomous driving according to claim 1, characterized in that: The process of integrating the second line segment data based on the preset condition to obtain the third line segment data specifically includes: Based on the distance, slope and overlapping area of the second line segment data, each line segment in the second line segment data is merged and interrupted to obtain the third line segment data.
6. The method for processing lane lines for automatic driving according to claim 5, characterized in that: The process of integrating the second line segment data based on the preset condition to obtain the third line segment data further includes: Calculating the confidence of each line segment in the second line segment data, and filtering the line segments based on a preset threshold and the confidence to obtain filtered line segments; The filtered line segments are merged and interrupted to obtain third line segment data.
7. The method for processing lane lines for autonomous driving according to claim 1, characterized in that: The process of determining the target lane line based on the third line segment data specifically includes: Smoothing the third line segment data, and de-jittering the data in the third line segment based on a filtering algorithm to obtain a line segment to be fused; The line segments to be fused are fused to determine the target lane line.
8. An automatic driving lane line processing device, characterized in that: The device comprises: An image acquisition module, used to acquire a driving environment image and obtain lane line data based on the environment image; A first processing module, used for extracting the lane line data to obtain first line segment data corresponding to the lane line; A second processing module, configured to perform a matching operation on the first line segment data based on the line segment data of the historical frame to obtain matched second line segment data; The lane confirmation module is used to integrate the second line segment data based on a preset condition to obtain third line segment data, and determine the target lane line based on the third line segment data.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the autonomous driving lane line processing method as described in any one of claims 1-7.
10. A terminal device, characterized in that: include: Processor and memory; The memory stores a computer-readable program executable by the processor; When the processor executes the computer-readable program, the steps in the automatic driving lane line processing method as described in any one of claims 1 to 7 are implemented.
