Lane line detection method, device and storage medium

Abstract

The application provides a lane line detection method, device and storage medium, the method comprising: inputting an acquired lane line scene graph into a lane line network detection model trained in advance, and performing fitting processing on the obtained result to obtain first lane line information, wherein the lane line network detection model is used for outputting key points of lane lines in the lane line scene graph; inputting the acquired lane line scene graph into a lane line semantic segmentation model trained in advance, and performing extraction and fitting processing on the obtained result to obtain second lane line information; and performing fusion processing on the first lane line information and the second lane line information to determine lane line information corresponding to the lane line scene graph. The lane line detection method provided by the application can improve the accuracy of lane line detection.

Description

Technical Field

[0001] This invention relates to the field of intelligent driving technology, and in particular to a lane line detection method, device and storage medium. Background Technology

[0002] With the development of intelligent driving technology, lane detection has become an important basic function of vehicle-assisted driving and autonomous driving. Accurately detecting and identifying lane lines is an important prerequisite for functions such as lane departure warning, lane keeping, and lane changing.

[0003] Current lane line detection methods mainly require post-processing of the detection results after semantic segmentation model detection, and finally fitting of the post-processed results to obtain the lane lines.

[0004] However, current detection methods suffer from issues such as missing lane lines and inaccurate predictions. Improving the accuracy of lane line detection has become a pressing technical problem that needs to be solved. Summary of the Invention

[0005] This invention provides a lane line detection method, device, and storage medium to solve the problem of low accuracy in current lane line detection.

[0006] In a first aspect, embodiments of the present invention provide a lane line detection method, comprising:

[0007] The acquired lane line scene map is input into a pre-trained lane line network detection model, and the result is fitted to obtain the first lane line information. The lane line network detection model is used to output the key points of the lane lines in the lane line scene map.

[0008] The acquired lane line scene image is input into a pre-trained lane line semantic segmentation model, and the results are extracted and fitted to obtain the second lane line information. The lane line semantic segmentation model is used to classify all pixels in the lane line scene image.

[0009] The first lane line information and the second lane line information are fused to determine the lane line information corresponding to the lane line scene map.

[0010] In one possible implementation, the first lane line information and the second lane line information are fused to determine the lane line information corresponding to the lane line scene map, including:

[0011] Based on the distances of all lane lines in the first and second lane line information from the camera that captured the lane line scene image in the world coordinate system, all lane lines are sorted.

[0012] When the distance between two adjacent lane lines is less than a first preset distance, the lane lines that need to be deleted are determined based on the confidence levels of the two adjacent lane lines.

[0013] The lane lines that need to be deleted are deleted from the first lane line information and the second lane line information to obtain the lane line information corresponding to the lane line scene map.

[0014] In one possible implementation, the confidence of each of the two adjacent lane lines includes the confidence of the number of points at which each lane line was fitted and / or the confidence of the fitting residuals at those points.

[0015] In one possible implementation, the confidence level of the fitting residuals for points includes the confidence level of the fitting residuals when fitting points in the lane line scene map and / or the confidence level of the fitting residuals when fitting points in world coordinates.

[0016] In one possible implementation, the acquired lane line scene map is input into a pre-trained lane line semantic segmentation model, and the result is extracted and fitted to obtain second lane line information, including:

[0017] The lane line scene map is input into the lane line semantic segmentation model, and the contour of the obtained lane line semantic label map is extracted to obtain the lane line contour.

[0018] Based on the lane line contour and the key points output by the lane line network detection model, the fitting points used to fit the second lane line information are determined.

[0019] The fitting points of the second lane line information after world coordinate transformation are fitted to obtain the second lane line information.

[0020] In one possible implementation, based on the lane line contour and key points output by the lane line network detection model, fitting points for fitting the second lane line information are determined, including:

[0021] The lane lines are merged to obtain the processed outline.

[0022] Based on the pixels on the processing contour line and the key points output by the lane line network detection model, the point where the position of the pixel on the processing contour line coincides with the position of the key points output by the lane line network detection model is set as the fitting point of the second lane line information.

[0023] In one possible implementation, before determining the fitting points for fitting the second lane line information based on the lane line contour and key points output by the lane line network detection model, the following steps are also included:

[0024] The contours of the lane lines are grouped, and based on the degree of overlap between the contours in the target group and the contours in other groups in the Y direction, as well as the distance between the fitted lines of the contours in the target group and the contours in other groups, it is determined whether the contours in the target group should be merged into other groups.

[0025] In one possible implementation, the acquired lane line scene map is input into a pre-trained lane line network detection model, and the result is fitted to obtain the first lane line information, including:

[0026] Based on the key points output by the lane line network detection model and the coordinates of the first vanishing point, the key points are transformed into world coordinates. The first vanishing point is the convergence point of lane lines on the image output by the lane line network detection model.

[0027] The key points after world coordinate transformation are fitted to obtain the information of the first lane line.

[0028] Secondly, embodiments of the present invention provide a lane line detection device, comprising:

[0029] The first acquisition module is used to input the acquired lane line scene map into a pre-trained lane line network detection model and perform fitting processing on the obtained results to obtain the first lane line information. The lane line network detection model is used to output the key points of the lane lines in the lane line scene map.

[0030] The second acquisition module is used to input the acquired lane line scene map into the pre-trained lane line semantic segmentation model, and to extract and fit the results to obtain the second lane line information. The lane line semantic segmentation model is used to classify all pixels in the lane line scene map.

[0031] The determination module is used to fuse the first lane line information and the second lane line information to determine the lane line information corresponding to the lane line scene map.

[0032] In one possible implementation, a determining module is used to sort all lane lines based on the distances of all lane lines in the first lane line information and the second lane line information from the camera that captured the lane line scene map in the world coordinate system.

[0033] When the distance between two adjacent lane lines is less than a first preset distance, the lane lines that need to be deleted are determined based on the confidence levels of the two adjacent lane lines.

[0034] The lane lines that need to be deleted are deleted from the first lane line information and the second lane line information to obtain the lane line information corresponding to the lane line scene map.

[0035] In one possible implementation, the confidence of each of the two adjacent lane lines includes the confidence of the number of points at which each lane line was fitted and / or the confidence of the fitting residuals at those points.

[0036] In one possible implementation, the confidence level of the fitting residuals for points includes the confidence level of the fitting residuals when fitting points in the lane line scene map and / or the confidence level of the fitting residuals when fitting points in world coordinates.

[0037] In one possible implementation, the second acquisition module is used to input the lane line scene map into the lane line semantic segmentation model, and extract the contour of the obtained lane line semantic label map to obtain the lane line contour.

[0038] Based on the lane line contour and the key points output by the lane line network detection model, the fitting points used to fit the second lane line information are determined.

[0039] The fitting points of the second lane line information after world coordinate transformation are fitted to obtain the second lane line information.

[0040] In one possible implementation, the second acquisition module is used to merge the contours of the lane lines to obtain the processed contour lines.

[0041] Based on the pixels on the processing contour line and the key points output by the lane line network detection model, the point where the position of the pixel on the processing contour line coincides with the position of the key points output by the lane line network detection model is set as the fitting point of the second lane line information.

[0042] In one possible implementation, the second acquisition module is used to group the contours of the lane lines and determine whether the contours in the target group are merged into other groups based on the degree of overlap between the contours in the target group and the contours in other groups in the Y direction, and the distance between the fitted lines of the contours in the target group and the contours in other groups.

[0043] In one possible implementation, the first acquisition module is used to perform world coordinate transformation on the key points based on the key points output by the lane line network detection model and the coordinates of the first vanishing point, wherein the first vanishing point is the aggregation point of lane lines on the image output by the lane line network detection model.

[0044] The key points after world coordinate transformation are fitted to obtain the information of the first lane line.

[0045] Thirdly, embodiments of the present invention provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method as described in the first aspect or any possible implementation of the first aspect.

[0046] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method as described in the first aspect or any possible implementation thereof.

[0047] This invention provides a lane line detection method, device, and storage medium. First, the acquired lane line scene image is input into a pre-trained lane line network detection model, and the result is fitted to obtain first lane line information. Then, the acquired lane line scene image is input into a pre-trained lane line semantic segmentation model, and the result is extracted and fitted to obtain second lane line information. Finally, the first and second lane line information are fused to determine the lane line information corresponding to the lane line scene image.

[0048] This invention employs two different types of lane detection models to detect lane lines in the acquired lane scene map and fuses the outputs of the two models. This allows for more accurate lane line determination and avoids the problem of lane line loss or inaccurate prediction caused by inaccurate post-processing results of a single lane detection model, thereby improving the stability and safety of autonomous driving. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart illustrating the implementation of the lane line detection method provided in this embodiment of the invention.

[0051] Figure 2 This is a block diagram of the lane line detection method provided in an embodiment of the present invention;

[0052] Figure 3 This is a block diagram of contour extraction provided in an embodiment of the present invention;

[0053] Figure 4 This is a block diagram of contour merging provided in an embodiment of the present invention;

[0054] Figure 5 This is a schematic diagram of the lane line detection device provided in an embodiment of the present invention;

[0055] Figure 6 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0056] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0057] To make the objectives, technical solutions, and advantages of the present invention clearer, specific embodiments will be described below in conjunction with the accompanying drawings.

[0058] As described in the background section, most current methods use a single lane detection model to detect lane lines. However, inaccurate predictions can occur during the detection process, such as missing lane lines or lane lines sticking together. This reduces the accuracy of lane detection results, making it impossible to effectively locate vehicles and ensure driving safety.

[0059] To address the problems of the prior art, embodiments of the present invention provide a lane line detection method, device, and storage medium. The lane line detection method provided by the embodiments of the present invention will be described first.

[0060] This invention is applicable to both passenger cars and commercial vehicles. See [link / reference] Figure 1 and Figure 2 The flowchart illustrating the implementation of the method provided in the embodiment of the present invention is described in detail below:

[0061] Step S110: Input the obtained lane line scene map into the pre-trained lane line network detection model, and perform fitting processing on the obtained results to obtain the first lane line information.

[0062] Lane line scene map is a road scene map that includes lane lines. Specifically, it can be obtained by taking pictures of the road ahead of the vehicle through cameras or radar devices installed on the vehicle while the vehicle is in motion.

[0063] The lane line network detection model is used to output the key points of lane lines in the lane line scene map. This lane line network detection model is trained using a large number of lane line scene maps with labeled key points.

[0064] In some embodiments, the first lane line information can be obtained by fitting key points detected by the lane line network detection model. The specific steps are as follows:

[0065] Step S1101: Input the obtained lane line scene map into the pre-trained lane line network detection model and output the key points of all lane lines in the lane line scene map.

[0066] Step S1102: Determine the pitch angle based on the coordinates of the key points and vanishing points output by the lane line network detection model.

[0067] Because vehicles experience pitch changes during driving, the pitch angle needs to be determined before fitting lane lines.

[0068] The process of determining the pitch angle is as follows:

[0069] First, the key points in the lane line scene map output by the lane line network detection model are fitted using the least squares method to obtain the fitting equation of the lane line in the lane line scene map.

[0070] Then, based on the fitted equation of the lane lines in the lane line scene image, the coordinates of the first vanishing point are determined. The vanishing point refers to the intersection point of two lane lines in the same direction on the image output by the lane line network detection model.

[0071] Finally, the pitch angle of the camera device that captured the lane line scene can be obtained based on the coordinates of the first vanishing point.

[0072] Step S1103: Based on the coordinates of the first vanishing point, perform world coordinate transformation on the key points on all lane lines.

[0073] Step S1104: Fit the key points after world coordinate transformation to obtain the first lane line information.

[0074] Therefore, based on the above steps S1101-S1104, lane line information can be determined according to the key points of all lane lines in the lane line scene map output by the lane line network detection model.

[0075] Step S120: Input the obtained lane line scene map into the pre-trained lane line semantic segmentation model, and extract and fit the results to obtain the second lane line information.

[0076] Lane line semantic segmentation models are used to classify all pixels in an input image. For example, pixels can be pre-classified into two categories to determine whether they belong to lane lines or the background. White can be used to represent lane lines, and brown can be used to represent the background.

[0077] In some embodiments, lane line information in a lane line scene map can be determined using a lane line semantic segmentation model based on the following steps:

[0078] Step S1201: Input the lane line scene map into the lane line semantic segmentation model to obtain the lane line semantic label map.

[0079] The lane line semantic label map includes the classification labels of all pixels in the lane line scene map, that is, the lane lines and the background are classified with different labels.

[0080] Step S1202: Traverse all pixels on the lane line semantic label map, and extract the outline of the lane line in the lane line scene map based on the classification labels of all pixels and the positional relationship between all pixels.

[0081] like Figure 3 As shown, the specific traversal process is as follows:

[0082] Starting from the bottom left corner of the semantic label map, all pixels are traversed from left to right and bottom to top. First, the pixel's label category is used to determine if it is a lane line. Then, the positional relationship between the current pixel and existing contours is used to determine if it belongs to the same lane line. If it is adjacent to an existing contour, the pixel belongs to that contour and forms a lane line with it. If it is not adjacent to an existing contour, a new contour is created, meaning the pixel is in a different lane. Finally, by traversing all points, the contour groups are determined, and these groups correspond to different lane lines.

[0083] Step S1203: Merge and fit the contours of the lane lines to obtain the second lane line information.

[0084] To obtain the second lane line information, multiple points need to be selected from the lane line contour for fitting to obtain the lane line equation. The specific steps are as follows:

[0085] Step S12031: Based on the lane line contour and the key points output by the lane line network model, determine the fitting points used to fit the second lane line information.

[0086] Since there are many points on the lane line contour, multiple points need to be extracted from the contour points for fitting. In order to ensure the accuracy of the fitting, the reliability of the selected fitting points must be guaranteed.

[0087] In some embodiments, the lane line contours can first be merged to obtain processed contour lines. Then, based on the pixels on the processed contour lines and the key points output by the lane line network model, the points where the pixels on the processed contour lines coincide with the key points output by the lane line network model are set as the fitting points for the second lane line information.

[0088] In this embodiment, when the road is damaged or obstructed, and the lane line semantic segmentation model is used to detect the obtained lane line scene map, during contour extraction in step S1202, some contours may deviate from other normal contours, requiring merging of these deviated contours. The lane line contours can be grouped, and based on the overlap in the Y direction between the contour lines in the target group and the contour lines in other groups, it is determined whether the contour lines in the target group should be merged into other groups. The target group is the group that deviates from the contours.

[0089] like Figure 4 As shown, the specific merging process is as follows:

[0090] The deviated contours are grouped into one group, and the contours of each other lane line are grouped into another group. The groups of deviated contours are then traversed and processed.

[0091] If the maximum and minimum values ​​of the longitudinal coordinates of the existing grouped contours are Y, then... 11 and Y 12 The maximum and minimum values ​​of the longitudinal coordinates of the deviated contour are Y and Y, respectively. 21 and Y 22 The method for calculating the overlap can be based on Max(Y). 21 ,Y 11 )-Min(Y 22 ,Y 12 ) and Y 11 –Y 12 +Y 21 –Y 22 The size determines the degree of overlap. For example, Max(Y) 21 ,Y 11 )-Min(Y 22 ,Y 12 (less than Y) 11 –Y 12 +Y 21 –Y 22 This indicates that the deviated contour overlaps with the existing grouped contours in the Y direction, and they cannot be on the same lane line. Therefore, it is necessary to add a new lane line contour group. If Max(Y) 21 ,Y 11 )-Min(Y 22 ,Y12 ) greater than Y 11 –Y 12 +Y 21 –Y 22 If the deviation is not coincident with the existing grouped contours in the Y direction, it means that the deviation contour belongs to one of the existing grouped contours.

[0092] Then, the distance from a point on the deviated contour to an existing group contour is used to determine which existing group contour the deviated contour belongs to.

[0093] Finally, when the distance from a point on the deviated contour to an existing grouped contour is less than a preset distance, the deviated contour belongs to that group.

[0094] Step S12032: Based on the fitted points of the second lane line information and the vanishing points on the lane line semantic label map, perform world coordinate transformation on the fitted points.

[0095] By fitting the points of the second lane line information in the lane line scene map, the fitting equation can be obtained, and the vanishing points on the lane line semantic label map can be obtained through the fitting equation.

[0096] Once the vanishing point is obtained on the lane line semantic label map, the pitch angle can be obtained based on the coordinates of the vanishing point, thereby performing world coordinate transformation on the fitting point of the second lane line information.

[0097] Step S12033: Perform fitting processing on the fitted points after world coordinate transformation to obtain the second lane line information.

[0098] Based on the above steps, the results of the lane line semantic segmentation model detection can be obtained, and the determined second lane line information can be obtained.

[0099] Step S130: Perform fusion processing on the first lane line information and the second lane line information to determine the lane line information corresponding to the lane line scene map.

[0100] When the road is obstructed or damaged, there may be a discrepancy between the information of the first lane line and the information of the second lane line, and the abnormal lane lines need to be processed.

[0101] In some embodiments, abnormal lane lines can be determined based on the distance of each lane line from the camera capturing the lane line scene image. The specific steps are as follows:

[0102] Step S1301: Sort all lane lines based on the distances of all lane lines in the first lane line information and the second lane line information from the camera capturing the lane line scene image in the world coordinate system.

[0103] Step S1302: When the distance between two adjacent lane lines is less than the first preset distance, determine the lane lines that need to be deleted based on the confidence levels of the two adjacent lane lines.

[0104] The confidence level of lane lines can be calculated based on the points on each lane line.

[0105] In this embodiment, the confidence level of each lane line can be determined based on the confidence level of the number of fitting points for each lane line and / or the confidence level of the fitting residuals at those points.

[0106] Specifically, to improve accuracy, the confidence level of each lane line can be evaluated by using a weighted average of the confidence level of the number of fitting points and the confidence level of the fitting residuals of each point as the overall confidence level.

[0107] For example, when calculating the confidence score of points on each lane line, a score can be assigned based on the number of points selected during the fitting of each lane line equation, with a value ranging from 0 to 100. For instance, 0 points correspond to 0 points, and the maximum number of points corresponds to 100 points. Through linear fitting, the confidence score equation for the number of points can be obtained, thus the confidence score of the number of points on each lane line can be obtained based on the number of points selected during the fitting of each lane line equation.

[0108] The confidence score of the fitting residuals for each point includes the confidence score of the residuals when fitting points in the lane line scene map and the confidence score of the fitting residuals when fitting points in world coordinates. The confidence scores of the residuals when fitting points in the lane line scene map and the fitting residuals when fitting points in world coordinates are both 0-100 points. A score of 100 points is given for a fitting residual of 0, a score of 0 points is given for a fitting residual of 1000, and a score greater than 1000 is also 0. After performing a linear fit on the fitting residuals of all points, the confidence scores of the residuals when fitting points in the lane line scene map and the fitting residuals when fitting points in world coordinates for each lane line can be obtained.

[0109] The final overall confidence level is a weighted sum of the confidence level of the number of points for each lane line, the confidence level of the residuals when fitting points in the lane line scene map, and the confidence level of the fitting residuals when fitting points in world coordinates.

[0110] Finally, based on the overall confidence scores of the two adjacent lane lines, the lane line with the lower overall confidence score can be identified as the lane line that needs to be deleted.

[0111] Step S1303: Delete the lane lines that need to be deleted from the first lane line information and the second lane line information to obtain the lane line information corresponding to the lane line scene map.

[0112] According to step S1302, the lane line that needs to be deleted is determined. After deleting the lane line from the first lane line information and the second lane line information, the lane line information corresponding to the lane line scene map is finally obtained.

[0113] The lane line detection method provided by this invention firstly inputs the acquired lane line scene image into a pre-trained lane line network detection model, and performs fitting processing on the obtained result to obtain first lane line information. Then, the acquired lane line scene image is input into a pre-trained lane line semantic segmentation model, and the obtained result is extracted and fitted to obtain second lane line information. Finally, the first lane line information and the second lane line information are fused to determine the lane line information corresponding to the lane line scene image.

[0114] This invention employs two different types of lane detection models to detect lane lines in the acquired lane scene map and fuses the outputs of the two models. This allows for more accurate lane line determination and avoids the problem of lane line loss or inaccurate prediction caused by inaccurate post-processing results of a single lane detection model, thereby improving the stability and safety of autonomous driving.

[0115] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0116] Based on the lane line detection method provided in the above embodiments, the present invention also provides a specific implementation of a lane line detection device applied to the lane line detection method. Please refer to the following embodiments.

[0117] like Figure 5 As shown, a lane line detection device 500 is provided, the device comprising:

[0118] The first acquisition module 510 is used to input the acquired lane line scene map into a pre-trained lane line network detection model and perform fitting processing on the obtained result to obtain the first lane line information. The lane line network detection model is used to output the key points of the lane lines in the lane line scene map.

[0119] The second acquisition module 520 is used to input the acquired lane line scene map into a pre-trained lane line semantic segmentation model, and extract and fit the obtained results to obtain the second lane line information. The lane line semantic segmentation model is used to classify all pixels in the lane line scene map.

[0120] The determination module 530 is used to fuse the first lane line information and the second lane line information to determine the lane line information corresponding to the lane line scene map.

[0121] In one possible implementation, the determining module 530 is used to sort all lane lines based on the distances of all lane lines in the first lane line information and the second lane line information from the camera that captured the lane line scene map in the world coordinate system.

[0122] When the distance between two adjacent lane lines is less than a first preset distance, the lane lines that need to be deleted are determined based on the confidence levels of the two adjacent lane lines.

[0123] The lane lines that need to be deleted are deleted from the first lane line information and the second lane line information to obtain the lane line information corresponding to the lane line scene map.

[0124] In one possible implementation, the confidence of each of the two adjacent lane lines includes the confidence of the number of points at which each lane line was fitted and / or the confidence of the fitting residuals at those points.

[0125] In one possible implementation, the confidence level of the fitting residuals for points includes the confidence level of the fitting residuals when fitting points in the lane line scene map and / or the confidence level of the fitting residuals when fitting points in world coordinates.

[0126] In one possible implementation, the second acquisition module 520 is used to input the lane line scene map into the lane line semantic segmentation model, and extract the contour of the obtained lane line semantic label map to obtain the contour of the lane line.

[0127] Based on the lane line contour and the key points output by the lane line network detection model, the fitting points used to fit the second lane line information are determined.

[0128] The fitting points of the second lane line information after world coordinate transformation are fitted to obtain the second lane line information.

[0129] In one possible implementation, the second acquisition module 520 is used to merge the contours of the lane lines to obtain the processed contour lines.

[0130] Based on the pixels on the processing contour line and the key points output by the lane line network detection model, the point where the position of the pixel on the processing contour line coincides with the position of the key points output by the lane line network detection model is set as the fitting point of the second lane line information.

[0131] In one possible implementation, the second acquisition module 520 is used to group the contours of the lane lines and determine whether the contours in the target group are merged into other groups based on the degree of overlap between the contours in the target group and the contours in other groups in the Y direction, and the distance between the fitted lines of the contours in the target group and the contours in other groups.

[0132] In one possible implementation, the first acquisition module 510 is used to perform world coordinate transformation on the key points based on the key points output by the lane line network detection model and the coordinates of the first vanishing point, wherein the first vanishing point is the aggregation point of lane lines on the image output by the lane line network detection model.

[0133] The key points after world coordinate transformation are fitted to obtain the information of the first lane line. Figure 6 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Figure 6 As shown, the electronic device 6 in this embodiment includes: a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and executable on the processor 60. When the processor 60 executes the computer program 62, it implements the steps in the various lane detection method embodiments described above, for example... Figure 1 Steps 110 to 130 are shown. Alternatively, when the processor 60 executes the computer program 62, it implements the functions of each module in the above-described device embodiments, for example... Figure 5 The functions of modules 510 to 530 are shown.

[0134] For example, the computer program 62 can be divided into one or more modules, which are stored in the memory 61 and executed by the processor 60 to complete the present invention. The one or more modules can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 62 in the electronic device 6. For example, the computer program 62 can be divided into... Figure 5 Modules 510 to 530 are shown.

[0135] The electronic device 6 may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will understand that... Figure 6 This is merely an example of electronic device 6 and does not constitute a limitation on electronic device 6. It may include more or fewer components than shown, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0136] The processor 60 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0137] The memory 61 can be an internal storage unit of the electronic device 6, such as a hard disk or memory. The memory 61 can also be an external storage device of the electronic device 6, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 61 can include both internal and external storage units of the electronic device 6. The memory 61 is used to store the computer program and other programs and data required by the electronic device. The memory 61 can also be used to temporarily store data that has been output or will be output.

[0138] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0139] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0140] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0141] In the embodiments provided by this invention, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0142] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0143] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0144] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various lane detection method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0145] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A lane line detection method characterized by, The method comprises the following steps: inputting the obtained lane line scene graph into a pre-trained lane line network detection model, and performing fitting processing on the obtained result to obtain first lane line information, wherein the lane line network detection model is used to output key points of lane lines in the lane line scene graph; inputting the lane line scene graph into a lane line semantic segmentation model, and performing contour extraction on the obtained lane line semantic label graph to obtain a contour of the lane line; wherein the lane line semantic segmentation model is used to classify all pixel points in the lane line scene graph; determining fitting points for fitting second lane line information based on the contour of the lane line and the key points output by the lane line network detection model; performing fitting processing on the fitting points of the second lane line information after world coordinate conversion to obtain the second lane line information; sorting all lane lines based on distances of all lane lines in the first lane line information and the second lane line information from a camera for shooting the lane line scene graph in a world coordinate system; when a distance between two adjacent lane lines is less than a first preset distance, determining a lane line to be deleted based on respective confidences of the two adjacent lane lines; deleting the lane line to be deleted from the first lane line information and the second lane line information to obtain lane line information corresponding to the lane line scene graph.

2. The lane line detection method of claim 1, wherein, The respective confidences of the two adjacent lane lines comprise a confidence of a number of fitting points of each lane line and / or a confidence of fitting residuals of points.

3. The lane line detection method of claim 2, wherein, The confidence of the fitting residuals of the points comprises a confidence of fitting residuals of points in the lane line scene graph when fitting and / or a confidence of fitting residuals of points in the world coordinate system when fitting.

4. The lane line detection method of claim 1, wherein, The method further comprises the following steps before determining the fitting points for fitting the second lane line information based on the contour of the lane line and the key points output by the lane line network detection model: performing merging processing on the contour of the lane line to obtain a processed contour line; based on pixel points on the processed contour line and the key points output by the lane line network detection model, setting points at which the pixel points on the processed contour line coincide with positions of the key points output by the lane line network detection model as fitting points of the second lane line information.

5. The lane line detection method of claim 4, wherein, The method further comprises the following steps before determining the fitting points for fitting the second lane line information based on the contour of the lane line and the key points output by the lane line network detection model: grouping the contour of the lane line, and determining whether the contour line in a target group is merged into contour lines in other groups based on a coincidence degree of the contour line in the target group with contour lines in other groups in the Y direction and a distance between a fitting line of the contour line in the target group and fitting lines of the contour lines in other groups.

6. The lane line detection method of claim 1, wherein, The method of inputting the obtained lane line scene graph into a pre-trained lane line network detection model and performing fitting processing on the obtained result to obtain first lane line information comprises the following steps: Based on the key points output by the lane line network detection model and the coordinates of the first vanishing point, the key points are subjected to world coordinate conversion, wherein the first vanishing point is an aggregation point of lane lines on an image output by the lane line network detection model. The key points subjected to the world coordinate conversion are subjected to fitting processing to obtain the first lane line information.

7. An electronic device, comprising: A computer device comprising a memory for storing a computer program and a processor for invoking and running the computer program stored in the memory to execute the method according to any one of claims 1 to 6.

8. A computer-readable storage medium storing a computer program, the computer-readable storage medium comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 7. The computer program is executed by the processor to implement the steps of the method according to any one of claims 1 to 6.

Images