Lane line completion method and device

Abstract

The invention provides a lane line completion method and device, and relates to the technical field of electronic maps, and the method comprises the steps: obtaining lane line vector data corresponding to a map region to be subjected to lane line completion; according to the lane line vector data, determining a lane line image and a mapping relation between position coordinates of an original lane line in the lane line vector data and pixel point coordinates of the original lane line in the lane line image; according to the lane line image, determining a missing lane line and complementing the missing lane line to obtain a target lane line image; and according to the target lane line image and the mapping relation, determining the position coordinates of the complemented lane line after the missing lane line is complemented, and generating target lane line vector data. Therefore, the part with the missing lane line in the lane line vector data can be automatically complemented, compared with manual drawing complementation, the processing efficiency is greatly improved, and accurate complementation of the lane line in different map road scenes can be realized.

Description

Technical Field

[0001] The present disclosure relates to the technical field of electronic maps, and in particular to a lane line completion method and device. Background Art

[0002] In the process of collecting road data for HQ (High Quality) maps, the collected map data may be missing due to the obstruction of obstacles (motor vehicles, non-motor vehicles, pedestrians, etc.) on the road, resulting in some missing lane lines in the high-precision lane line vector data generated after processing, affecting the quality of the final map data. The processing method in the related art is to manually search and compare the actual photos and manually draw the missing lane lines to complete them, which is inefficient. Summary of the invention

[0003] In view of this, the present invention provides a lane line completion method and device, the main purpose of which is to automatically complete the part of the lane line vector data where there are missing lane lines. Compared with manual drawing completion, it greatly improves the processing efficiency and can realize accurate lane line completion in different map road scenes.

[0004] In order to solve the above problems, the present disclosure mainly provides the following technical solutions:

[0005] In a first aspect, the present disclosure provides a lane line completion method, including: obtaining lane line vector data corresponding to a map area to be lane line completed; determining a lane line image based on the lane line vector data, and a mapping relationship between the position coordinates of the original lane line in the lane line vector data and the coordinates of the original lane line pixel points in the lane line image; determining the missing lane line and completing it based on the lane line image to obtain a target lane line image; determining the position coordinates of the completed lane line after the missing lane line is completed based on the target lane line image and the mapping relationship, and generating the target lane line vector data.

[0006] In the second aspect, the present disclosure also provides a lane line completion device, including: a data acquisition unit, used to obtain lane line vector data corresponding to the map area to be lane line completed; a first processing unit, used to determine the lane line image according to the lane line vector data, and the mapping relationship between the position coordinates of the original lane line in the lane line vector data and the pixel point coordinates of the original lane line in the lane line image; a second processing unit, used to determine the missing lane line and complete it according to the lane line image to obtain a target lane line image; a third processing unit, used to determine the position coordinates of the completed lane line after the missing lane line is completed according to the target lane line image and the mapping relationship, and generate the target lane line vector data.

[0007] In a third aspect, the present disclosure further provides a storage medium, which includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the lane line completion method as described above.

[0008] In a fourth aspect, the present disclosure further provides a processor, which is used to run a program, wherein the lane line completion method as described above is executed when the program is running.

[0009] In a fifth aspect, the present disclosure also provides a terminal device, comprising at least one processor; the storage medium is used to store the program executed by the processor and the data required by the processor during the execution of the program; the processor is used to run the program, wherein the lane line completion method as described above is executed when the program is running.

[0010] The lane line completion method and device provided by the present disclosure obtain the lane line vector data corresponding to the map area to be completed; determine the lane line image and the mapping relationship between the position coordinates of the original lane line in the lane line vector data and the pixel point coordinates of the original lane line in the lane line image according to the lane line vector data; determine the missing lane line and complete it according to the lane line image to obtain the target lane line image; determine the position coordinates of the completed lane line after the missing lane line is completed according to the target lane line image and the mapping relationship, and generate the target lane line vector data. In this way, the part with missing lane lines in the lane line vector data can be automatically completed, which greatly improves the processing efficiency compared with manual drawing completion, and can realize accurate lane line completion in different map road scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present disclosure. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0012] Figure 1 A flow chart of a lane completion method provided by an embodiment of the present disclosure is shown;

[0013] Figure 2 A schematic diagram of lane completion provided by an embodiment of the present disclosure is shown;

[0014] Figure 3 A schematic diagram of another lane completion method provided by an embodiment of the present disclosure is shown;

[0015] Figure 4 A flow chart of another lane completion method provided by an embodiment of the present disclosure is shown;

[0016] Figure 5 A structural diagram of a lane line completion device provided by an embodiment of the present disclosure is shown;

[0017] Figure 6 A structural diagram of a second processing unit in a lane completion device provided by an embodiment of the present disclosure is shown;

[0018] Figure 7 A structural diagram of another lane line completion device provided by an embodiment of the present disclosure is shown.

[0019] Figure 8 A structural diagram of another lane line completion device provided by an embodiment of the present disclosure is shown.

[0020] Fig. 9 A structural diagram of a third processing unit in a lane completion device provided in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0021] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0022] It should be noted that before introducing the embodiments of the present disclosure, the related terms involved are first introduced.

[0023] HQ (High Quality) map: A high-precision map that contains road elements such as lane line vectors and traffic sign vectors. Its accuracy and / or update frequency is between that of standard maps and high-precision maps. It is mainly used in autonomous driving, assisted driving, and lane-level navigation scenarios. In addition, HQ maps are often referred to as lane-level maps, light maps, etc.

[0024] In the related art, the missing lane lines are completed by manually searching and comparing the actual photos, which is inefficient.

[0025] Based on this, the disclosed embodiment provides a lane line completion method and device, wherein the method includes: obtaining lane line vector data corresponding to the map area to be completed; determining the lane line image and the mapping relationship between the position coordinates of the original lane line in the lane line vector data and the pixel point coordinates of the original lane line in the lane line image according to the lane line vector data; determining the missing lane line and completing it according to the lane line image, generating the completed lane line, and obtaining the target lane line image; determining the position coordinates of the completed lane line according to the target lane line image and the mapping relationship, and generating the target lane line vector data. In this way, the part of the lane line vector data with missing lane lines can be automatically completed, which greatly improves the processing efficiency compared to manual drawing completion, and can realize accurate lane line completion in different map road scenes.

[0026] The present disclosure provides a lane line completion method, such as Figure 1 As shown, the method includes:

[0027] S1: Obtain lane line vector data corresponding to the map area to be completed.

[0028] It should be noted that the lane line completion method of the embodiment of the present disclosure may be executed by the lane line completion device of the embodiment of the present disclosure, and the lane line completion device may be implemented by software and / or hardware.

[0029] Among them, the lane line completion device can be an electronic device, or it can be a chip, chip system, or processor configured in the electronic device to support the electronic device to implement the lane line completion method.

[0030] Among them, the electronic device can install and run the map generation program, and the electronic device can include but is not limited to smart phones, tablet computers, computers and other hardware devices with various operating systems.

[0031] Of course, the lane line completion device may also be other devices other than electronic devices, such as computer program products.

[0032] Among them, when the computer program product is executed by a processor, it can implement the lane line completion method provided by the embodiment of the present disclosure.

[0033] It is understandable that in the process of collecting road data for the HQ map, it is necessary to obtain the lane line vector data corresponding to the road.

[0034] Of course, in the process of collecting road data for maps other than HQ maps, it is also necessary to obtain the lane line vector data corresponding to the road, such as high-precision maps.

[0035] The lane line is a traffic marking used to divide and indicate different driving lanes on the road. The lane line vector data includes the location coordinates of the lane line, including longitude and latitude coordinates and elevation coordinates. The longitude and latitude coordinates include longitude coordinates and latitude coordinates, and the elevation coordinates can also be called altitude.

[0036] It can be understood that the lane line vector data is three-dimensional data of the lane line, wherein the longitude coordinates and the latitude coordinates can be regarded as coordinates on a plane, and the elevation coordinates can be regarded as height coordinates relative to the plane.

[0037] In the disclosed embodiment, road data is collected to obtain lane line vector data corresponding to a specified map area.

[0038] Among them, when the lane line vector data corresponding to the specified map area is obtained, the specified map area can be determined as the map area to be completed with lane lines, that is, the lane line vector data corresponding to the specified map area is the lane line vector data corresponding to the map area to be completed with lane lines, or part of the specified map area can be determined as the map area to be completed with lane lines, that is, the lane line vector data corresponding to part of the map area in the specified map area is the lane line vector data corresponding to the map area to be completed with lane lines.

[0039] Exemplarily, when the lane line vector data corresponding to the specified map area is obtained, the scope of the specified map area is relatively large, and the specified map area can be divided according to the geographic network coordinates of a preset size, and the specified map area can be divided into appropriate sizes. Each divided map area is a map area for lane line completion.

[0040] It should be noted that the method for obtaining lane line vector data in the embodiment of the present disclosure can refer to the method in the related art and will not be repeated here.

[0041] It can be understood that in the embodiment of the present disclosure, lane line vector data corresponding to the map area to be lane line completed is obtained, wherein the map area may have missing lane lines, or there may be no missing lane lines. In the embodiment of the present disclosure, it can be determined whether there are missing lane lines based on the lane line vector data corresponding to the map area, and the lane lines can be completed.

[0042] S2: Determine a lane line image according to the lane line vector data, and a mapping relationship between the position coordinates of the original lane line in the lane line vector data and the pixel point coordinates of the original lane line in the lane line image.

[0043] In the disclosed embodiment, after acquiring the new lane vector data, the lane line image can be determined based on the lane line vector data, as well as the mapping relationship between the position coordinates of the original lane line in the lane line vector data and the pixel point coordinates of the original lane line in the lane line image.

[0044] In some embodiments, the lane line vector data is projected onto a plane to generate a lane line image, and a mapping relationship between the position coordinates of the original lane line in the lane line vector data and the pixel point coordinates of the original lane line in the lane line image can also be determined.

[0045] In some embodiments, determining a lane line image based on lane line vector data includes: projecting the lane line vector data onto a ground plane to determine the lane line image.

[0046] In the disclosed embodiment, the lane line vector data is projected onto the ground plane to determine the lane line image. Projecting onto the ground plane can obtain accurate lane line information to improve the accuracy of subsequent recognition and completion of missing lane lines.

[0047] It can be understood that the lane line vector data includes the position coordinates of the shape points of the lane line, including longitude coordinates, latitude coordinates and elevation coordinates. In the lane line image determined by projecting the lane line vector data onto the ground plane, the shape points of the lane line correspond to pixel point coordinates, and the mapping relationship between the position coordinates of the original lane line in the lane line vector data and the pixel point coordinates of the original lane line in the lane line image can be determined.

[0048] In the disclosed embodiment, the mapping relationship between the position coordinates of the original lane line in the lane line vector data and the pixel point coordinates of the original lane line in the lane line image can be the mapping relationship between the latitude and longitude coordinates and elevation coordinates of the original lane line in the lane line vector data and the pixel point coordinates of the original lane line in the lane line image.

[0049] In the disclosed embodiment, the mapping relationship between the position coordinates of the original lane line in the lane line vector data and the pixel point coordinates of the original lane line in the lane line image is determined, and the lane line image can be converted back to the lane line vector data based on the mapping relationship.

[0050] S3: According to the lane line image, determine the missing lane lines and complete them to obtain the target lane line image.

[0051] In the disclosed embodiment, after the lane line image is obtained, the missing lane lines can be determined and completed based on the lane line image to generate completed lane lines and obtain the target lane line image.

[0052] In some embodiments, based on the lane line image, an artificial intelligence model that can be used for image processing is used to determine the missing lane lines and complete them, generate completed lane lines, and obtain a target lane line image.

[0053] In some embodiments, a lane line image is input into an artificial intelligence model for image processing to generate a target lane line image, wherein there are missing lane lines in the lane line image, and the target lane line image includes completed lane lines after the missing lane lines are completed.

[0054] For example, Figure 2 As shown, after obtaining the lane line image ( Figure 2 As shown in the left figure), the missing lane line is determined according to the lane line image ( Figure 2 As shown in the middle figure), the missing lane lines are completed to generate the completed lane lines ( Figure 2 As shown in the dashed box in the right figure), the target lane line image is obtained.

[0055] S4: According to the target lane line image and the mapping relationship, the position coordinates of the completed lane line after the missing lane line is completed are determined, and the target lane line vector data is generated.

[0056] In the disclosed embodiment, when a target lane line image is obtained after the missing lane line is completed, the position coordinates of the completed lane line can be determined according to the target lane line image and the mapping relationship to generate the target lane line vector data.

[0057] It can be understood that the lane line image is obtained by projecting the lane line vector data onto the ground plane, and there is a mapping relationship between the position coordinates of the original lane line in the lane line vector data and the coordinates of the original lane line pixel points in the lane line image.

[0058] Based on this, in the embodiment of the present disclosure, back projection is performed based on the target lane line image and the mapping relationship to determine the position coordinates of the completed lane line (longitude coordinates and latitude coordinates, as well as the elevation coordinates of the endpoints where the completed lane line connects with the original lane line), and then the elevation coordinates of the shape points of the completed lane line are determined based on the elevation coordinates of the endpoints where the completed lane line connects with the original lane line to generate the target lane line vector data.

[0059] Among them, according to the target lane line image and the mapping relationship, the latitude and longitude coordinates and elevation coordinates of the completed lane line are determined, and the target lane line vector data is generated.

[0060] In some embodiments, the elevation coordinates of the completed lane line are determined based on the elevation coordinates of the original lane lines at both adjacent ends of the completed lane line, so that the generated elevation coordinates of the completed lane line are smoothed.

[0061] In some embodiments, the position coordinates of the completed lane line include longitude and latitude coordinates and elevation coordinates, wherein the position coordinates of the completed lane line are determined based on the target lane line image and the mapping relationship, including: determining the longitude and latitude coordinates of the completed lane line, as well as the first elevation coordinates of the first endpoint of the first original lane line adjacent to the completed lane line and the second elevation coordinates of the second endpoint of the second original lane line according to the pixel point coordinates of the completed lane line in the target lane line image and the mapping relationship; determining the elevation coordinates of the completed lane line based on the first elevation coordinates and the second elevation coordinates.

[0062] In the disclosed embodiment, the longitude and latitude coordinates of the completed lane line, as well as the first elevation coordinates of the first end point of the first original lane line adjacent to the completed lane line and the second elevation coordinates of the second end point of the second original lane line are determined based on the pixel point coordinates of the completed lane line in the target lane line image and the mapping relationship. Then, based on the first elevation coordinates and the second elevation coordinates, the elevation coordinates of the completed lane line are determined, so that the missing lane line can be automatically completed to obtain an accurate completed lane line.

[0063] In some embodiments, the elevation coordinates of the completed lane line are determined according to the first elevation coordinate and the second elevation coordinate, including: assigning the first elevation coordinate and the second elevation coordinate to the two endpoints of the completed lane line respectively; performing interpolation and smoothing processing on the completed lane line according to the first elevation coordinate and the second elevation coordinate to determine the elevation coordinates of the completed lane line.

[0064] In the embodiment of the present disclosure, when determining the first elevation coordinate of the first endpoint of the first original lane line adjacent to the completed lane line and the second elevation coordinate of the second endpoint of the second original lane line, the elevation coordinate of the completed lane line can be determined based on the first elevation coordinate and the second elevation coordinate.

[0065] The first elevation coordinate and the second elevation coordinate are respectively assigned to the two end points of the completed lane line; the completed lane line is interpolated and smoothed according to the first elevation coordinate and the second elevation coordinate to determine the elevation coordinate of the completed lane line.

[0066] For example, Figure 3 As shown, the first endpoint of the first original lane line adjacent to the completed lane line and the second endpoint of the second original lane line adjacent to the completed lane line are determined, the first elevation coordinate of the first endpoint and the second elevation coordinate of the second endpoint are determined, the first elevation coordinate of the first endpoint is assigned to the left endpoint of the completed lane line, and the second elevation coordinate of the second endpoint is assigned to the right endpoint of the completed lane line. When the elevation coordinates of the two ends of the completed lane line are determined, the elevation coordinates of the completed lane line can be interpolated and smoothed according to the first elevation coordinate and the second elevation coordinate to determine the elevation coordinates of the completed lane line.

[0067] In a possible implementation, the difference between the first elevation coordinate and the second elevation coordinate, as well as the length of the completed lane line, is determined, and arithmetic interpolation is performed to obtain a completed lane line with overall smooth elevation coordinates.

[0068] In some embodiments, a rule-based strategy-based judgment method is used to complete the lane lines. After obtaining the lane line vector data, each lane line can be traversed to find the endpoints of the lane lines that are not connected to other lane lines, called hanging points. By matching with the hanging points of other lane lines around the hanging points, it is determined according to certain rule strategies (determining whether the categories and directions of the original lane lines where the two hanging points are located are consistent, etc.) whether there is a missing lane line between the two hanging points. If there is, a lane line is generated to connect the two hanging points. The geometry of the generated lane line can be calculated by interpolation according to the geometry (position, angle) of the original lane lines corresponding to the hanging points. However, the accuracy of lane line completion using the rule-based strategy-based judgment method is low, and it is difficult to exclude areas such as intersections and forks where there are no lane lines. It is also difficult to handle rare scenes, and redundancy or omissions are prone to occur.

[0069] In the disclosed embodiment, the lane line vector data corresponding to the map area to be lane line completed is obtained; based on the lane line vector data, the lane line image is determined, as well as the mapping relationship between the position coordinates of the original lane line in the lane line vector data and the pixel coordinates of the original lane line in the lane line image; based on the lane line image, the missing lane line is determined and completed, the completed lane line is generated, and the target lane line image is obtained; based on the target lane line image and the mapping relationship, the position coordinates of the completed lane line are determined, and the target lane line vector data is generated. In this way, the problem that the judgment method based on the rule strategy has a low accuracy rate for lane line completion and it is difficult to exclude areas such as intersections and forks where there are no lane lines can be overcome, and the lane line can be accurately completed in different scenarios, and high-quality lane line completion can be completed.

[0070] The present disclosure provides another lane line completion method, such as Figure 4 As shown, the method includes:

[0071] S10: Obtain lane line vector data corresponding to the map area to be lane line completed.

[0072] S20: Determine a lane line image according to the lane line vector data, and a mapping relationship between the position coordinates of the original lane line in the lane line vector data and the coordinates of the original lane line pixel points in the lane line image.

[0073] For the related description of S10 and S20, reference may be made to the related description in the above embodiment, which will not be repeated here.

[0074] S30: Determine a target detection frame according to the lane line image.

[0075] In the disclosed embodiment, after the lane line image is acquired, the area where the missing lane line exists can be determined based on the lane line image, and a target detection frame can be generated, wherein the area in the lane line image selected by the target detection frame is the area where the missing lane line exists.

[0076] Among them, the object detection box is used to mark the area where there are missing lane lines.

[0077] In the disclosed embodiment, the area where the lane line is missing is marked using a target detection frame, so that the area where the lane line is missing can be accurately and automatically identified. Compared with manual identification, the efficiency of missing lane line identification can be improved.

[0078] In some embodiments, a neural network model is used to generate a target detection box based on the lane line image.

[0079] In some embodiments, a target detection frame is determined based on a lane line image, including: inputting the lane line image into a target detection model to generate a detection frame and a confidence level corresponding to the detection frame; and determining a detection frame whose confidence level is not less than a threshold as a target detection frame.

[0080] In the disclosed embodiment, the lane line image is input into the target detection model, the area with missing lane lines is detected, and a detection frame and a confidence level corresponding to the detection frame are generated; the detection frame with a confidence level not less than a threshold is determined to be the target detection frame.

[0081] Among them, the detection frame is used to mark the area where the missing lane line exists, and the confidence corresponding to the detection frame is used to reflect the accuracy of the area where the missing lane line is marked by the detection frame.

[0082] Among them, the confidence level corresponding to the detection box is used to indicate the area in the lane line image selected by the detection box, which is the accuracy of the area where there are missing lane lines.

[0083] In some embodiments, the threshold value is 0.8, or 0.7, or 0.6, or 0.5, etc.

[0084] In some embodiments, the target detection model is a YOLO (You Only Look Once) target detection network model, or a DETR (Detection Transformer) target detection network model.

[0085] YOLO is a popular target detection network model that can provide fast and accurate object detection with high speed and precision. In the detection task, the YOLO target detection network model only needs one forward propagation to complete the detection task, without the need for complex post-processing steps such as non-maximum suppression.

[0086] DETR is a Transformer-based target detection network model. The DETR target detection network model regards target detection as a set prediction problem and directly outputs the final detection result through the Transformer encoder-decoder architecture. It eliminates the dependence on anchor boxes and post-processing steps (such as non-maximum suppression) in traditional target detection, and realizes end-to-end training and reasoning. The characteristics of the DETR target detection network model include simplifying the detection process, effectively eliminating the need for many manually designed components, and being able to process all object detection tasks in the image in parallel.

[0087] In the disclosed embodiment, a target detection model is used to detect areas where lane lines are missing in a lane line image, and a confidence level is output, thereby ensuring accurate positioning of areas where lane lines are missing and preventing omission of areas where lane lines are missing or redundant generation of lane lines in non-missing areas due to direct completion in the entire image.

[0088] In the disclosed embodiment, a target detection model is used to detect areas where missing lane lines are present. The missing lane lines can be detected automatically, and a detection frame with a confidence level not less than a threshold is determined as a target detection frame. Compared with manual recognition, the efficiency and accuracy of recognition can be improved.

[0089] In some embodiments, the lane line completion method provided by the embodiments of the present disclosure also includes: obtaining a target detection training data set, wherein the target detection training data set includes sample lane line images and sample detection frames marking areas where missing lane lines are missing; training an initial target detection model based on the target detection training data set to generate a target detection model.

[0090] In an embodiment of the present disclosure, a target detection training data set is obtained, including a sample lane line image and a sample detection frame marking an area where lane lines are missing; an initial target detection model is trained based on the target detection training data set, the sample lane line image is input into the initial target detection model, a predicted detection frame marking an area where lane lines are missing is generated, and parameters of the initial target detection model are adjusted based on the sample detection frame and the predicted detection frame to generate a target detection model.

[0091] In the disclosed embodiment, an initial target detection model is trained using a target detection training data set to generate a target detection model, which can obtain a detection frame and confidence level for accurately marking areas where lane lines are missing.

[0092] S40: According to the lane line image and the target detection frame, the missing lane line is completed to obtain the target lane line image.

[0093] In the embodiment of the present disclosure, when a target detection frame is obtained to mark the area where the missing lane line is located, the missing lane line can be completed based on the lane line image and the target detection frame to generate the completed lane line and obtain the target lane line image. Completing the missing lane line based on the target detection frame can improve the efficiency and accuracy of lane line completion compared to manual completion.

[0094] In some embodiments, the missing lane lines are completed according to the lane line image and the target detection frame to obtain the target lane line image, including: masking the lane line image according to the target detection frame; inputting the masked lane line image into the lane line generation model to complete the missing lane lines and obtain the target lane line image.

[0095] In the disclosed embodiment, after obtaining the target detection frame, the lane line image can be masked according to the target detection frame, and the lane line image after masking can be input into the lane line generation model to complete the missing lane lines, generate the completed lane lines, and obtain the target lane line image.

[0096] In some embodiments, the lane line generation model is an image generation deep neural network model (such as a MAT (Mask-Aware Transformer) deep neural network model, a Stable Diffusion diffusion neural network model).

[0097] In some embodiments, the lane line completion method provided by the embodiments of the present disclosure also includes: obtaining a lane line generation training data set, wherein the lane line generation training data set includes a sample missing lane line image, a sample detection box marking an area of ​​missing lane lines in the sample missing lane line image, and a sample completed lane line image after completing the missing lane lines in the sample missing lane line image; training an initial lane line generation model according to the lane line generation training data set to generate a lane line generation model.

[0098] In the disclosed embodiment, a lane line generation training data set is obtained, including a sample missing lane line image, a sample detection frame marking an area of ​​missing lane lines in the sample missing lane line image, and a sample completed lane line image after completing the missing lane lines in the sample missing lane line image; an initial lane line generation model is trained based on the lane line generation training data set, the sample missing lane line image is masked based on the sample detection frame marking an area of ​​missing lane lines in the sample missing lane line image, the sample missing lane line image after masking is input into the initial lane line generation model to generate a predicted completed lane line image, and then, based on the predicted completed lane line image and the sample completed lane line image, the parameters of the initial lane line generation model are adjusted to generate a lane line generation model.

[0099] In the disclosed embodiment, when acquiring a lane line generation training data set, if the number of sample-completed lane line images after completing the missing lane lines in the sample-missing lane line image is small, the image with complete lane lines can be used as the sample-completed lane line image after completing the missing lane lines in the sample-missing lane line image, and some lane lines in the image with complete lane lines are manually deleted to obtain the sample-missing lane line image, and a sample detection box is drawn in the sample-missing lane line image to mark the area of ​​the missing lane lines to obtain the lane line generation training data set.

[0100] In the disclosed embodiment, lane line completion is performed using a method based on a lane line generation model driven by a massive true value lane line generation training data set, ensuring that the lane line generation model can learn completion methods for different scenarios from samples, has strong generalization capabilities, and achieves accurate lane line completion in different map road scenarios.

[0101] In the disclosed embodiment, a lane line generation training data set is used to train an initial lane line generation model to generate a lane line generation model, which can accurately complete missing lane lines and obtain accurate completed lane lines. Compared with manual completion, efficiency can be improved.

[0102] S50: Determine the position coordinates of the lane line to be completed based on the target lane line image and the mapping relationship, and generate target lane line vector data.

[0103] For the relevant description of S50, reference may be made to the relevant description in the above embodiment, which will not be repeated here.

[0104] In the disclosed embodiment, lane line vector completion is performed by projecting lane line vector data into a lane line image. By visually expressing the vector, it is ensured that the computer vision algorithm can perceive the neighborhood context features, find the area that needs to be completed from a global perspective, and complete high-quality lane line completion based on the features of the surrounding lane lines.

[0105] By implementing the embodiments of the present disclosure, it is possible to automatically complete the parts of the lane line vector data where there are missing lane lines. Compared with manual drawing completion, the processing efficiency is greatly improved, and accurate lane line completion can be achieved in different map road scenes.

[0106] Based on the above method embodiment, the present disclosure also provides a lane line completion device 1, such as Figure 5 As shown, the device includes: a data acquisition unit 11, a first processing unit 12, a second processing unit 13 and a third processing unit 14.

[0107] The data acquisition unit 11 is used to acquire lane line vector data corresponding to the map area to be lane line completed.

[0108] The first processing unit 12 is used to determine a lane line image and a mapping relationship between the position coordinates of the original lane line in the lane line vector data and the pixel point coordinates of the original lane line in the lane line image according to the lane line vector data.

[0109] The second processing unit 13 is used to determine and complete the missing lane lines according to the lane line image to obtain a target lane line image.

[0110] The third processing unit 14 is used to determine the position coordinates of the completed lane line after the missing lane line is completed according to the target lane line image and the mapping relationship, and generate target lane line vector data.

[0111] like Figure 6 As shown, in some embodiments, the second processing unit 13 includes: a detection frame generating module 131 and a completion processing module 132.

[0112] The detection frame generation module 131 is used to determine a target detection frame according to the lane line image, wherein the area in the lane line image selected by the target detection frame is an area where there is a missing lane line.

[0113] The completion processing module 132 is used to complete the missing lane lines according to the lane line image and the target detection frame to obtain the target lane line image.

[0114] In some embodiments, the detection frame generation module 131 is specifically used to: input the lane line image into the target detection model, generate a detection frame and a confidence level corresponding to the detection frame, wherein the confidence level corresponding to the detection frame is used to indicate the area in the lane line image selected by the detection frame, and is the accuracy of the area where the lane line is missing; determine that the detection frame with a confidence level not less than a threshold is the target detection frame.

[0115] like Figure 7 As shown, in some embodiments, the device further includes: a first data set acquisition unit 15 and a first training processing unit 16.

[0116] The first data set acquisition unit 15 is used to acquire a target detection training data set, wherein the target detection training data set includes a sample lane line image and a sample detection frame marking an area where a lane line is missing.

[0117] The first training processing unit 16 is used to train the initial target detection model according to the target detection training data set to generate a target detection model.

[0118] In some embodiments, the completion processing module 132 is specifically used to: perform mask processing on the lane line image according to the target detection frame; input the lane line image after mask processing into the lane line generation model, complete the missing lane lines, and obtain the target lane line image.

[0119] like Figure 8 As shown, in some embodiments, the device further includes: a second data set acquisition unit 17 and a second training processing unit 18.

[0120] The second data set acquisition unit 17 is used to acquire a lane line generation training data set, wherein the lane line generation training data set includes a sample missing lane line image, a sample detection frame marking an area of ​​a missing lane line in the sample missing lane line image, and a sample completed lane line image after completing the missing lane lines in the sample missing lane line image.

[0121] The second training processing unit 18 is used to generate a training data set according to the lane lines, train the initial lane line generation model, and generate a lane line generation model.

[0122] like Fig. 9 As shown, in some embodiments, the position coordinates of the completed lane line include latitude and longitude coordinates and elevation coordinates, wherein the third processing unit 14 includes: a first coordinate determination module 141 and a second coordinate determination module 142.

[0123] The first coordinate determination module 141 is used to determine the longitude and latitude coordinates of the completed lane line, as well as the first elevation coordinates of the first endpoint of the first original lane line adjacent to the completed lane line and the second elevation coordinates of the second endpoint of the second original lane line based on the pixel point coordinates of the completed lane line in the target lane line image and the mapping relationship.

[0124] The second coordinate determination module 142 is used to determine the elevation coordinate of the completed lane line according to the first elevation coordinate and the second elevation coordinate.

[0125] In some embodiments, the second coordinate determination module 142 is specifically used to: assign the first elevation coordinate and the second elevation coordinate to the two endpoints of the completed lane line respectively; perform interpolation and smoothing processing on the completed lane line according to the first elevation coordinate and the second elevation coordinate to determine the elevation coordinate of the completed lane line.

[0126] In some embodiments, the first processing unit 12 is specifically used to project the lane line vector data onto the ground plane to determine the lane line image.

[0127] It should be noted that, for the relevant description of the functional units and functional modules of the lane line completion device in the embodiment of the present disclosure, reference can be made to the description in the method embodiment, and will not be repeated here.

[0128] It should be noted that the beneficial effects of the lane line completion device in the embodiment of the present disclosure are the same as the beneficial effects of the lane line completion method. Please refer to the description in the method embodiment and will not be repeated here.

[0129] The present disclosure also provides a storage medium, which includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the method as described above.

[0130] The present disclosure also provides a processor, which is used to run a program, wherein the program executes the method as described above when running.

[0131] The present disclosure also provides a terminal device, comprising at least one processor; the storage medium is used to store a program executed by the processor and data required by the processor during the execution of the program; the processor is used to run the program, wherein the program executes the method described above when it is run.

[0132] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0133] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0134] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0135] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0136] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0137] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0138] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0139] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0140] It will be appreciated by those skilled in the art that the embodiments of the present disclosure may be provided as methods, systems or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0141] The above are only embodiments of the present disclosure and are not intended to limit the present disclosure. For those skilled in the art, the present disclosure may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the scope of the claims of the present disclosure.

Claims

1. A lane line completion method, characterized in that: include: Obtain lane line vector data corresponding to the map area to be completed; Determine a lane line image according to the lane line vector data, and a mapping relationship between the position coordinates of the original lane line in the lane line vector data and the pixel point coordinates of the original lane line in the lane line image; According to the lane line image, determine the missing lane lines and complete them to obtain a target lane line image; According to the target lane line image and the mapping relationship, the position coordinates of the completed lane line after the missing lane line is completed are determined to generate target lane line vector data.

2. The method according to claim 1, characterized in that The determining and completing the missing lane lines according to the lane line image to obtain the target lane line image includes: Determine a target detection frame according to the lane line image, wherein the area in the lane line image selected by the target detection frame is an area where there is a missing lane line; According to the lane line image and the target detection frame, the missing lane line is completed to obtain the target lane line image.

3. The method according to claim 2, characterized in that The determining of the target detection frame according to the lane line image includes: Input the lane line image into the target detection model to generate a detection frame and a confidence level corresponding to the detection frame, wherein the confidence level corresponding to the detection frame is used to indicate the accuracy of the area in the lane line image selected by the detection frame, which is the area where the missing lane line exists; Determine the detection frame whose confidence is not less than a threshold as the target detection frame.

4. The method according to claim 2, characterized in that: The method of completing the missing lane line according to the lane line image and the target detection frame to obtain the target lane line image includes: Performing mask processing on the lane line image according to the target detection frame; The lane line image after mask processing is input into the lane line generation model to complete the missing lane lines to obtain the target lane line image.

5. The method according to claim 1, characterized in that The position coordinates of the completed lane line include latitude and longitude coordinates and elevation coordinates, wherein determining the position coordinates of the completed lane line after the missing lane line is completed according to the target lane line image and the mapping relationship includes: Determine, according to the pixel coordinates of the completed lane line in the target lane line image and the mapping relationship, the longitude and latitude coordinates of the completed lane line, and the first elevation coordinates of the first endpoint of the first original lane line adjacent to the completed lane line and the second elevation coordinates of the second endpoint of the second original lane line; The elevation coordinates of the completed lane line are determined according to the first elevation coordinates and the second elevation coordinates.

6. The method according to claim 5, characterized in that The step of determining the elevation coordinate of the completed lane line according to the first elevation coordinate and the second elevation coordinate includes: Assigning the first elevation coordinate and the second elevation coordinate to two endpoints of the completed lane line respectively; The completed lane line is interpolated and smoothed according to the first elevation coordinate and the second elevation coordinate to determine the elevation coordinate of the completed lane line.

7. A lane line completion device, characterized in that: include: A data acquisition unit, used to acquire lane line vector data corresponding to the map area to be lane line completed; A first processing unit, configured to determine a lane line image and a mapping relationship between position coordinates of an original lane line in the lane line vector data and pixel point coordinates of the original lane line in the lane line image according to the lane line vector data; A second processing unit is used to determine and complete the missing lane lines according to the lane line image to obtain a target lane line image; The third processing unit is used to determine the position coordinates of the completed lane line after the missing lane line is completed according to the target lane line image and the mapping relationship, and generate target lane line vector data.

8. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the method according to any one of claims 1 to 6.

9. A processor, characterized in that: The processor is used to run a program, wherein the program executes the method according to any one of claims 1 to 6 when running.

10. A terminal device, characterized in that: The device comprises at least one processor; the storage medium is used to store a program executed by the processor and data required by the processor in the process of executing the program; The processor is used to run a program, wherein the program executes the method according to any one of claims 1 to 6 when running.

Images