Lane line detection model reasoning optimization method based on generalization data training

Abstract

The invention discloses a lane line detection model reasoning optimization method based on generalization data training, and the main design concept of the invention lies in providing a lane line recognition scheme based on associative learning and sequential control based on a deep learning visual model. Specifically, in the training stage, a visual generation model is used for repairing abnormal lane line pictures and predicting lane line pictures which are rare or difficult to capture, and the accuracy and efficiency of lane line marking are improved in a man-machine coupling mode; in an actual reasoning stage, a time sequence concept is introduced, and a mechanism for evaluating a detection result is provided by combining time sequence information of front and back frames. According to the invention, the training data of the detection model is effectively enhanced and generalized, and the robustness and accuracy of the lane line detection result are improved, so that many problems of lane line identification in automatic driving application can be solved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a lane line detection model reasoning optimization method using generalized data training. Background Art

[0002] Lane line recognition is crucial in autonomous driving. It can help the system monitor and track lane markings in real time, ensuring that the vehicle always drives in the center of the lane, thereby improving driving safety. By accurately identifying lane lines, the system can effectively prevent lane departure, assist vehicle decisions such as changing lanes or overtaking, and collaborate with other autonomous driving functions to optimize the driving experience and overall safety.

[0003] However, in reality, lane lines may not be clear, especially in cities, where some lane lines may be blocked by other vehicles or there are no lane lines on some roads. These factors may affect the accuracy and reliability of lane line recognition. At present, in response to the problems existing in lane line recognition, the technical route mostly adopts a solution that combines high-precision maps and real-time positioning systems (such as GPS) to obtain road geometric topology information.

[0004] However, it has been found in practice that the combination of high-precision maps and real-time positioning has the following problems:

[0005] (1) Maps are not updated in a timely manner. High-precision maps need to be updated regularly. If the map data is outdated, it may lead to inaccurate lane line recognition;

[0006] (2) Map coverage is limited. Some roads in certain areas, such as parks and country roads, do not include lane information in the map;

[0007] (3) Highly dependent on real-time positioning accuracy. The error caused by positioning will affect the lane line information, resulting in inaccurate lane line recognition;

[0008] (4) Road data changes. High-precision maps cannot reflect changes in temporary roads, such as construction or accidents, in real time, resulting in the map not being consistent with the actual situation. Summary of the invention

[0009] In view of the above, the present invention aims to provide a lane detection model inference optimization method using generalized data training to solve the above-mentioned technical problems.

[0010] The technical solution adopted by the present invention is as follows:

[0011] The present invention provides a lane detection model reasoning optimization method using generalized data training, which includes:

[0012] Using the first training set and the second training set respectively, a first visual generation model and a second visual generation model are constructed correspondingly;

[0013] Get the image to be annotated and determine whether there are abnormal lane lines in the image;

[0014] Inputting the picture with abnormal lane lines into the first visual generation model to obtain a lane line extension picture, in which the abnormal lane lines are repaired;

[0015] Inputting the picture without abnormal lane lines and / or the lane line extension picture into the second visual generation model to generate a plurality of lane line prediction pictures;

[0016] Label the lane line extension image and lane line prediction image;

[0017] Use the annotated images to build a lane detection model, and use the lane detection model to detect lane lines in the actual input images;

[0018] The detection results of the current frame are evaluated by combining the detection results of the previous and next frames.

[0019] In at least one possible implementation, the output result of the second visual generation model is fed back into the first visual generation model for restoration processing.

[0020] In at least one possible implementation, a defect judgment is performed on the output result of the first visual generation model, and if a defect is determined to exist, the output result is sent to the first visual generation model again for expansion.

[0021] In at least one possible implementation, the first training set is constructed using real lane line images under multiple scenes and multiple lighting conditions, and the images in the first training set include lane line anomalies.

[0022] In at least one possible implementation manner, the second training set is constructed using real lane line images containing time series information, and the images in the second training set include lane line conditions whose states change within a continuous time period.

[0023] In at least one possible implementation manner, the evaluation includes several combinations of the following scoring dimensions: distance consistency score, shape consistency score, smoothness score, and feature matching score.

[0024] Compared with the prior art, the main design concept of the present invention is to propose a lane line recognition scheme of associative learning + timing control based on a deep learning visual model. Specifically, in the training stage, the visual generation model is used to repair abnormal lane line images and predict rare or hard-to-capture lane line images, so as to improve the accuracy and efficiency of lane line marking in a human-machine coupling manner; in the actual reasoning stage, the concept of timing is introduced, and a mechanism for evaluating the detection results is proposed by combining the timing information of the previous and next frames. The present invention effectively enhances and generalizes the training data of the detection model, and improves the robustness and accuracy of the lane line detection results, thereby being able to solve many problems of lane line recognition in autonomous driving applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described below with reference to the accompanying drawings, in which:

[0026] Figure 1 A schematic diagram of a lane detection model inference optimization method using generalized data training provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] Embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0028] The present invention proposes an embodiment of a lane detection model reasoning optimization method using generalized data training. Specifically, Figure 1 shown, including:

[0029] Step S1, pre-constructing a first visual generation model and a second visual generation model;

[0030] Step S2: Obtain the image to be annotated and determine whether there is an abnormal lane line in the image;

[0031] Step S3, inputting the picture with abnormal lane lines into the first visual generation model to obtain a lane line extension picture, in which the abnormal lane lines are repaired;

[0032] Step S4, inputting the picture without abnormal lane lines and / or the lane line extension picture into the second visual generation model to generate a plurality of lane line prediction pictures;

[0033] In practice, a high-quality dataset can be prepared in advance, which includes lane line images in a variety of scenes and lighting conditions. The dataset can contain annotated data of real lane lines and cover different occlusion conditions, such as vehicles, trees, or roadblocks. These annotated data will serve as the basis for training the first generative model to ensure that the model can learn the characteristics and occlusion status of the lane lines.

[0034] Next, select a suitable aforementioned first visual generative model architecture, such as but not limited to deep generative models such as generative adversarial networks (GANs) and variational autoencoders (VAEs), which can learn features from input images and generate corresponding lane line images; after selecting a suitable model architecture, it is necessary to train the first model using the aforementioned dataset.

[0035] In addition, in actual operation, the architecture of the second visual generation model can be selected, for example but not limited to, a time series generation model architecture, which can ensure that predicted images for multiple future times are generated based on the current lane line image. Specifically, in the second model training phase, a data set containing time series information is used to train the model. The data set includes: lane line images for continuous time periods, so that the data model can learn the patterns and laws of lane line changes over time. During training, the second model will start from the current image and generate future image sequences. These image sequences cover various possible changes, such as deformation of lane lines, changes in background, and movement of other elements on the road.

[0036] After the two visual generation models are trained and built, they are deployed to the data annotation tool or platform. When the annotators process the images:

[0037] On the one hand, the first visual generation model automatically expands the defective lane lines to assist in subsequent data supplementation.

[0038] On the other hand, the second vision generation model can be used to predict a single lane line image and generate multiple predicted images, which show the possible changes of the lane line in the future. The annotator can select typical predicted images for subsequent annotation processing.

[0039] Step S4: manually labeling the lane line extension image and the lane line prediction image;

[0040] Through the above scheme of the present invention, on the one hand, the labeling efficiency can be improved, and on the other hand, the error caused by manual intervention can be reduced. Based on this, a feedback mechanism can also be designed, that is, the output results of the first and second visual generation models can be corrected and adjusted to iterate the performance effect of the visual generation model. For example, the output results of the second visual generation model can be screened and sent back to the first visual generation model to repair the predicted defective pictures; the output results of the first visual generation model can also be directly judged for defects (the defects are the anomalies mentioned above, such as occlusion, unclearness, missing, etc.). If it is determined that there are defects, it will be sent to the first visual generation model again for expansion. If it is determined that there are no defects, it will enter the subsequent manual labeling link.

[0041] Step S5: construct a lane line detection model using the annotated image, and use the lane line detection model to detect lane lines in the actual input image;

[0042] Step S6: Evaluate the detection result of the current frame by combining the detection results of the previous and next frames.

[0043] Understandably, the present invention believes that the reasoning results of a single frame image are easily affected by factors such as occlusion, missing or blurred lane lines, resulting in inaccurate detection results. Therefore, in order to improve the reliability of lane line detection, the present invention proposes a method of combining the results of the previous and next frames to construct an evaluation model, thereby significantly enhancing the detection results.

[0044] In actual operation, the main technical concept of constructing a time-series-based reasoning result evaluation model is to use time-series data to evaluate and optimize lane detection results, as follows: after obtaining the lane detection result of the current frame, the detection results of the previous and next frames are input into the reasoning result evaluation model together with the detection result of the current frame. The evaluation model gives a score or correction suggestion for the detection result of the current frame based on the consistency of the previous and next frames and the time series law. In this way, the lane detection result of the current frame can be adjusted to improve the accuracy of detection.

[0045] The specific evaluation dimensions and process of the inference result evaluation model can be referred to as follows:

[0046] Distance consistency score: For each lane line in the current frame, find the lane line that is most similar to the previous and next frames and calculate their distance;

[0047] Shape consistency score: Shape consistency is calculated using a shape similarity metric (Hausdorff distance);

[0048] Smoothness score: Use a smoothness metric (such as differential variance) to calculate the change between the lane line of the current frame and the lane line of the previous and next frames to evaluate whether the lane line changes smoothly;

[0049] Feature matching score: Extract the lane feature vector and calculate the cosine similarity between the lane feature vector of the current frame and the lane feature vectors of the previous and next frames.

[0050] Weighted comprehensive score: Set weights for different scoring dimension indicators, and then calculate the weighted average score. For example, the weights of distance consistency score, shape consistency score, smoothness score, and feature matching score are all 25%.

[0051] In summary, the main design concept of the present invention is to propose a lane line recognition scheme of associative learning + timing control based on a deep learning visual model. Specifically, in the training stage, the visual generation model is used to repair abnormal lane line images and predict rare or hard-to-capture lane line images, so as to improve the accuracy and efficiency of lane line marking in a human-machine coupling manner; in the actual reasoning stage, the concept of timing is introduced, and a mechanism for evaluating the detection results is proposed by combining the timing information of the previous and next frames. The present invention effectively enhances and generalizes the training data of the detection model, and improves the robustness and accuracy of the lane line detection results, thereby being able to solve many problems of lane line recognition in autonomous driving applications.

[0052] If the expressions expressing orientation are mentioned in the embodiments of the present invention, they are relative concepts based on the embodiments. In addition, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, a and b, a and c, b and c, or a and b and c, wherein a, b, c can be single or multiple.

[0053] The above describes in detail the structure, features and effects of the present invention based on the embodiments shown in the drawings, but the above is only a preferred embodiment of the present invention. It should be noted that the technical features involved in the above embodiments and their preferred methods can be reasonably combined and matched into a variety of equivalent schemes by those skilled in the art without departing from or changing the design ideas and technical effects of the present invention; therefore, the present invention is not limited to the scope of implementation shown in the drawings, and all changes made in accordance with the concept of the present invention, or modifications to equivalent embodiments with equivalent changes, which still do not exceed the spirit covered by the specification and drawings, should be within the protection scope of the present invention.

Claims

1. A lane detection model inference optimization method using generalized data training, characterized in that: include: Using the first training set and the second training set respectively, a first visual generation model and a second visual generation model are constructed correspondingly; Get the image to be annotated and determine whether there are abnormal lane lines in the image; Inputting the picture with abnormal lane lines into the first visual generation model to obtain a lane line extension picture, in which the abnormal lane lines are repaired; Inputting the picture without abnormal lane lines and / or the lane line extension picture into the second visual generation model to generate a plurality of lane line prediction pictures; Label the lane line extension image and lane line prediction image; Use the annotated images to build a lane detection model, and use the lane detection model to detect lane lines in the actual input images; The detection results of the current frame are evaluated by combining the detection results of the previous and next frames.

2. The lane detection model inference optimization method using generalized data training according to claim 1, characterized in that: The output result of the second visual generation model is sent back to the first visual generation model for repair processing.

3. The lane detection model inference optimization method using generalized data training according to claim 1, characterized in that: The output result of the first visual generation model is subjected to defect judgment, and if it is judged that there is a defect, it is sent to the first visual generation model again for expansion.

4. The lane detection model inference optimization method using generalized data training according to claim 1, characterized in that: The first training set is constructed using real lane line images under multiple scenes and multiple lighting conditions, and the images in the first training set include lane line anomalies.

5. The lane detection model inference optimization method using generalized data training according to claim 1, characterized in that: The second training set is constructed using real lane line images containing time series information, and the images in the second training set include lane line conditions whose states change within a continuous time period.

6. The lane detection model inference optimization method using generalized data training according to any one of claims 1 to 5, characterized in that: The evaluation includes several combinations of the following scoring dimensions: distance consistency score, shape consistency score, smoothness score, and feature matching score.

Images