Lane line annotation, detection model determination, lane line detection method and related devices

Abstract

The present application relates to a lane line annotation, detection model determination, lane line detection method and related devices. By using a generative adversarial network model to perform lane line detection on a lane line scene graph to be detected, a corresponding lane line semantic graph is generated. Based on the position information of each pixel point in the lane line semantic graph, the lane lines in the lane line scene graph to be detected are determined. Accordingly, all lane lines in the lane line scene graph can be detected simultaneously, and the lane lines in the occluded area can be generated. In the lane line semantic label graph used when training the generative adversarial network model, the drawn lane line annotation lines are thicker at the proximal end and thinner at the distal end. Accordingly, the adhesion of the annotated lane lines at the distal end can be reduced, and the accuracy of lane line detection can be improved.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and in particular, to a lane line annotation, detection model determination, lane line detection method, and related devices. Background Art

[0002] With the development of intelligent driving technology, lane line detection has become a fundamental link in automotive assisted driving and driverless driving. Accurately detecting and recognizing lane lines is an important prerequisite for functions such as lane departure warning, lane keeping, and lane change. In current lane line detection methods based on deep learning, the detected lane lines are prone to adhesion at the far end. Lane line adhesion will cause inaccurate curve fitting, resulting in a low accuracy of the final lane line detection result. Summary of the Invention

[0003] Based on this, in view of the above technical problems, it is necessary to provide a lane line annotation, detection model determination, lane line detection method, and related devices that can improve the accuracy of lane line detection.

[0004] A lane line annotation method, the method includes:

[0005] Obtaining the position information of the annotation points on each lane line based on the lane line scene graph;

[0006] Determining the position information of the points to be drawn corresponding to each lane line according to the position information of the annotation points on each lane line;

[0007] Determining the thickness information of the annotation lines corresponding to each lane line at each point to be drawn based on the position information of the points to be drawn corresponding to each lane line;

[0008] Drawing the annotation lines corresponding to each lane line according to the position information of the points to be drawn corresponding to each lane line and the thickness information of the annotation lines at each point to be drawn, to obtain the lane line semantic label graph corresponding to the lane line scene graph.

[0009] A lane line annotation device, the device includes:

[0010] An annotation point information acquisition module, configured to obtain the position information of the annotation points on each lane line based on the lane line scene graph;

[0011] A point to be drawn information determination module, configured to determine the position information of the points to be drawn corresponding to each lane line according to the position information of the annotation points on each lane line;

[0012] An annotation line information determination module, configured to determine the thickness information of the annotation lines corresponding to each lane line at each point to be drawn based on the position information of the points to be drawn corresponding to each lane line;

[0013] The annotation line drawing module is used to draw the annotation lines corresponding to each lane line according to the position information of the points to be drawn corresponding to each lane line and the thickness information of the annotation lines at each point to be drawn, so as to obtain the lane line semantic label map corresponding to the lane line scene map.

[0014] A computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the following steps are implemented:

[0015] Obtain the position information of the annotation points on each lane line based on the lane line scene map;

[0016] Determine the position information of the points to be drawn corresponding to each lane line according to the position information of the annotation points on each lane line;

[0017] Determine the thickness information of the annotation lines corresponding to each lane line at each point to be drawn based on the position information of the points to be drawn corresponding to each lane line;

[0018] Draw the annotation lines corresponding to each lane line according to the position information of the points to be drawn corresponding to each lane line and the thickness information of the annotation lines at each point to be drawn, so as to obtain the lane line semantic label map corresponding to the lane line scene map.

[0019] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the following steps are implemented:

[0020] Obtain the position information of the annotation points on each lane line based on the lane line scene map;

[0021] Determine the position information of the points to be drawn corresponding to each lane line according to the position information of the annotation points on each lane line;

[0022] Determine the thickness information of the annotation lines corresponding to each lane line at each point to be drawn based on the position information of the points to be drawn corresponding to each lane line;

[0023] Draw the annotation lines corresponding to each lane line according to the position information of the points to be drawn corresponding to each lane line and the thickness information of the annotation lines at each point to be drawn, so as to obtain the lane line semantic label map corresponding to the lane line scene map.

[0024] The above lane line annotation method, device, computer device, and storage medium obtain the position information of the points to be drawn corresponding to the lane line based on the position information of the annotation points on the lane line in the lane line scene graph, determine the thickness information of the annotation line corresponding to the lane line at each point to be drawn through the position information of the points to be drawn corresponding to the lane line, so that the thickness of the line at different positions of the points to be drawn is different. The lane line annotation line drawn according to the position information of the points to be drawn corresponding to the lane line and the thickness information of the annotation line at each point to be drawn can achieve a thick proximal end and a thin distal end, thereby reducing the adhesion of the lane line annotated in the lane line semantic label graph at the distal end and improving the subsequent lane line detection accuracy.

[0025] A method for determining a lane line detection model, the method comprising:

[0026] Obtain a sample lane line scene graph;

[0027] Use the above lane line annotation method to perform lane line annotation on the sample lane line scene graph to obtain a lane line semantic label graph corresponding to the sample lane line scene graph;

[0028] Based on the sample lane line scene graph and the lane line semantic label graph, train a generative adversarial network model to be trained to obtain a trained generative adversarial network model;

[0029] Determine a lane line detection model according to the generator in the trained generative adversarial network model.

[0030] A device for determining a lane line detection model, the device comprising:

[0031] A sample acquisition module for obtaining a sample lane line scene graph;

[0032] A lane line annotation module for using the above lane line annotation method to perform lane line annotation on the sample lane line scene graph to obtain a lane line semantic label graph corresponding to the sample lane line scene graph;

[0033] A model training module for training a generative adversarial network model to be trained based on the sample lane line scene graph and the lane line semantic label graph to obtain a trained generative adversarial network model;

[0034] A model determination module for determining a lane line detection model according to the generator in the trained generative adversarial network model.

[0035] A computer device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0036] Obtain a sample lane line scene graph;

[0037] Use the above lane line annotation method to annotate the lane lines in the sample lane line scene graph, and obtain the lane line semantic label graph corresponding to the sample lane line scene graph;

[0038] Based on the sample lane line scene graph and the lane line semantic label graph, train the generative adversarial network model to be trained, and obtain the trained generative adversarial network model;

[0039] Determine the lane line detection model according to the generator in the trained generative adversarial network model.

[0040] A computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0041] Obtain a sample lane line scene graph;

[0042] Use the above lane line annotation method to annotate the lane lines in the sample lane line scene graph, and obtain the lane line semantic label graph corresponding to the sample lane line scene graph;

[0043] Based on the sample lane line scene graph and the lane line semantic label graph, train the generative adversarial network model to be trained, and obtain the trained generative adversarial network model;

[0044] Determine the lane line detection model according to the generator in the trained generative adversarial network model.

[0045] The method, device, computer device, and storage medium for determining the above lane line detection model are different from the general generative adversarial network that uses the semantic graph as input to generate a natural scene graph. By reversely using the generative adversarial network and taking the real lane line scene graph as input for semantic segmentation to generate the lane line semantic graph, it is beneficial to remove the complex background, and can generate the lane lines in the occluded area, with better robustness and stronger adaptability.

[0046] A lane line detection method, the method includes:

[0047] Obtain the lane line scene graph to be detected;

[0048] Use the lane line detection model determined by the above method for determining the lane line detection model to detect the lane lines in the lane line scene graph to be detected, and obtain the lane line semantic graph, where the lane line semantic graph includes the position information of each pixel point;

[0049] Based on the position information of each pixel point in the lane line semantic graph, determine the lane lines in the lane line scene graph to be detected.

[0050] A lane line detection device, the device comprising:

[0051] A to-be-detected image acquisition module, configured to acquire a to-be-detected lane line scene image;

[0052] A lane line detection module, configured to perform lane line detection on the to-be-detected lane line scene image by using a lane line detection model determined by the above-mentioned lane line detection model determination method, to obtain a lane line semantic map, where the lane line semantic map includes position information of each pixel point;

[0053] A lane line determination module, configured to determine the lane lines in the to-be-detected lane line scene image based on the position information of each pixel point in the lane line semantic map.

[0054] A computer device, comprising a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0055] Acquire a to-be-detected lane line scene image;

[0056] Perform lane line detection on the to-be-detected lane line scene image by using a lane line detection model determined by the above-mentioned lane line detection model determination method, to obtain a lane line semantic map, where the lane line semantic map includes position information of each pixel point;

[0057] Determine the lane lines in the to-be-detected lane line scene image based on the position information of each pixel point in the lane line semantic map.

[0058] A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0059] Acquire a to-be-detected lane line scene image;

[0060] Perform lane line detection on the to-be-detected lane line scene image by using a lane line detection model determined by the above-mentioned lane line detection model determination method, to obtain a lane line semantic map, where the lane line semantic map includes position information of each pixel point;

[0061] Determine the lane lines in the to-be-detected lane line scene image based on the position information of each pixel point in the lane line semantic map.

[0062] The above lane line detection method, device, computer equipment and storage medium use the generator in the generative adversarial network model to generate the lane line semantic map corresponding to the lane line scene map to achieve end-to-end detection, eliminating the steps of preprocessing and calculation of the lane line scene map, with a longer detection distance, fewer manual tuning parameters and better robustness. Compared with the lane line detection method based on the semantic segmentation neural network of the probability graph, which can only detect a fixed number of lane lines and cannot generate the lane lines in the occlusion area, using the generative adversarial network for lane line detection can simultaneously detect all the lane lines in the lane line scene map and can generate the lane lines in the occlusion area, thereby improving the accuracy of lane line detection and being able to adapt to most complex road scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 It is a schematic flowchart of the lane line annotation method in an embodiment;

[0064] Figure 2 It is a schematic diagram of the lane line scene map in an embodiment;

[0065] Figure 3 It is a schematic diagram of the lane line semantic label map in an embodiment;

[0066] Figure 4 It is a schematic flowchart of the method for determining the lane line detection model in an embodiment;

[0067] Figure 5 It is a schematic flowchart of the lane line detection method in an embodiment;

[0068] Figure 6 It is a schematic diagram of the lane line contour in an embodiment;

[0069] Figure 7 It is a schematic diagram of the lane line contour in an embodiment;

[0070] Figure 8 It is a schematic diagram of the lane line contour in an embodiment;

[0071] Figure 9 It is a structural block diagram of the lane line annotation device in an embodiment;

[0072] Figure 10 It is a structural block diagram of the device for determining the lane line detection model in an embodiment;

[0073] Figure 11 It is a structural block diagram of the lane line detection device in an embodiment;

[0074] Figure 12 It is an internal structure diagram of the computer equipment in an embodiment;

[0075] Figure 13 The internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0076] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0077] The lane line annotation method, the method for determining a lane line detection model, and the lane line detection method provided by the present application can be applied to a vehicle intelligent driving system. The vehicle intelligent driving system includes a vehicle controller and a collection device. The collection device can be installed on the vehicle and collect road pictures or videos as the vehicle travels. The vehicle controller can obtain a lane line scene map from the road pictures or videos collected by the collection device, and perform lane line annotation on the lane line scene map to obtain a lane line semantic label map; it can further train a lane line detection model based on the lane line scene map and the lane line semantic label map; it can further use the trained lane line detection model to perform lane line detection.

[0078] In one embodiment, as Figure 1 shown, a lane line annotation method is provided. Taking the application of this method to a vehicle controller as an example, it includes the following steps S102 to S108.

[0079] S102, obtain the position information of the annotation points on each lane line based on the lane line scene map.

[0080] The lane line scene map represents a road scene map containing lane lines, and can be specifically obtained by a camera installed on the vehicle taking pictures of the road ahead during the vehicle's travel. Figure 2 shows a schematic diagram of a lane line scene map in an embodiment. The lane line scene map includes four lane lines, which are solid line, dashed line, dashed line, and solid line from left to right. Each lane line Figure 2 shows the characteristic of being thicker at the proximal end and thinner at the distal end in, that is, the lane line is thicker near the camera and thinner far from the camera.

[0081] The annotation point represents a point selected on the lane line in the lane line scene map. At least two annotation points are selected for each lane line. The position information of the annotation point can specifically be the coordinate value of the annotation point in the coordinate system established based on the lane line scene map. In one embodiment, taking Figure 2The lower left corner point of the lane line scene diagram shown is the coordinate origin O. Taking the vertical direction as the direction of the first coordinate axis (represented by the Y-axis) (vertically upward is the positive direction) and the horizontal direction as the direction of the second coordinate axis (represented by the X-axis) (horizontally to the right is the positive direction), a coordinate system is established, and the position information of the points is represented by (y, x). Among them, y represents the Y-axis coordinate value of the marked point, and x represents the X-axis coordinate value of the marked point.

[0082] S104. Determine the position information of the points to be drawn corresponding to each lane line according to the position information of the marked points on each lane line.

[0083] The points to be drawn represent the points required to draw the lane lines in the lane line scene diagram, which can be understood as the points on the marked lines to be drawn corresponding to the lane lines. The position information of the points to be drawn can specifically be the coordinate values of the points to be drawn in the coordinate system established based on the lane line scene diagram. It should be noted that the marked points on the lane line can be directly used as the points to be drawn corresponding to the lane line, or linear interpolation can be performed based on the adjacent marked points on the lane line, and the marked points and the interpolated points are used together as the points to be drawn corresponding to the lane line.

[0084] S106. Determine the thickness information of the marked lines corresponding to each lane line at each point to be drawn based on the position information of the points to be drawn corresponding to each lane line.

[0085] Referring to the coordinate system in the foregoing embodiment, the position information of the points to be drawn can be represented by (y i , x i ), where i represents the i-th point to be drawn. Specifically, the thickness of the marked line at the point to be drawn can be determined according to the size of the Y-axis coordinate value of the point to be drawn (i.e., y i ), so that the thickness of the line at the points to be drawn with different Y-axis coordinate values is different. In one embodiment, the thickness at each point to be drawn is inversely related to the size of the Y-axis coordinate value, that is, the smaller the Y-axis coordinate value of the point to be drawn, the thicker the marked line at the corresponding position.

[0086] S108. Draw the marked lines corresponding to each lane line according to the position information of the points to be drawn corresponding to each lane line and the thickness information of the marked lines at each point to be drawn, and obtain the lane line semantic label map corresponding to the lane line scene diagram.

[0087] The lane line semantic label map represents a picture obtained by annotating the lane lines in the lane line scene diagram. Figure 3 The schematic diagram of the lane line semantic label map in one embodiment is shown. This lane line semantic label map is Figure 2 the lane line semantic label map corresponding to the lane line scene diagram shown, which includes four marked lines, corresponding respectively to Figure 2 the four lane lines in the lane line scene diagram shown. Each marked line is atFigure 3 shows the characteristic of being thicker at the proximal end and thinner at the distal end, which is consistent with that of each lane line in Figure 2 showing the characteristic of being thicker at the proximal end and thinner at the distal end.

[0088] In the above lane line annotation method, the position information of the points to be drawn corresponding to the lane line is obtained based on the position information of the annotation points on the lane line in the lane line scene graph. The thickness information of the annotation line corresponding to the lane line at each point to be drawn is determined through the position information of the points to be drawn corresponding to the lane line, so that the thickness of the line at different positions of the points to be drawn is different. The lane line annotation line drawn according to the position information of the points to be drawn corresponding to the lane line and the thickness information of the annotation line at each point to be drawn can achieve a thick proximal end and a thin distal end, thereby reducing the adhesion of the lane lines annotated in the lane line semantic label map at the distal end and improving the accuracy of subsequent lane line detection.

[0089] In one embodiment, the step of determining the position information of the points to be drawn corresponding to each lane line according to the position information of the annotation points on each lane line may specifically include the following steps: linear interpolation is performed according to the position information of adjacent annotation points among the annotation points on each lane line to obtain the position information of the interpolation points between adjacent annotation points; based on the position information of the annotation points and the interpolation points on each lane line, the position information of the points to be drawn corresponding to each lane line is determined.

[0090] The annotation points on the lane line can be manually annotated. To reduce the workload of manual annotation, there can be a distance between adjacent annotation points, and then interpolation points are inserted between all adjacent annotation points through linear interpolation. Finally, all annotation points and all interpolation points are used as the points to be drawn.

[0091] For the annotation points on the same lane line, any two adjacent annotation points can determine a straight line. Specifically, a linear equation can be fitted based on the position information of the two adjacent annotation points, and the position information of the interpolation points between the two adjacent annotation points is calculated according to the linear equation.

[0092] Specifically, for example, the position information of two adjacent annotation points (P1, P2) is (y1, x1) and (y2, x2) respectively. Then the linear equation fitted from the position information of P1 and P2 is: x = ky + b, where k = (x2 - x1) / (y2 - y1), b = x1 - ky1. The middle value is selected between y1 and y2 as the Y-axis coordinate value of the interpolation point. Then, taking the Y-axis coordinate value of the interpolation point as a known variable, substituting it into the above linear equation, the X-axis coordinate value of the interpolation point is calculated, thereby obtaining the position information of the interpolation point. It can be understood that one or more middle values can be selected between y1 and y2, so the number of interpolation points between P1 and P2 can be one or more.

[0093] In this embodiment, the position information of adjacent marked points on the lane line is used for linear interpolation to obtain the position information of the interpolation points between adjacent marked points. Then, based on the position information of all marked points and all interpolation points, the position information of the points to be drawn corresponding to the lane line is determined. Accordingly, the workload of manual marking can be reduced, the points to be drawn required for drawing the marked line of the lane line can be quickly obtained, the marking efficiency can be improved, and at the same time, manual marking errors can be reduced and the marking accuracy can be improved.

[0094] In one embodiment, the position information includes the first coordinate value in the direction of the first coordinate axis in the coordinate system established based on the lane line scene graph, and the direction of the first coordinate axis represents the direction corresponding to the lane line extension direction; the step of determining the thickness information of the marked line corresponding to each lane line at each point to be drawn based on the position information of the points to be drawn corresponding to each lane line may specifically be: based on the magnitude of the first coordinate value of each point to be drawn corresponding to each lane line, determine the thickness of the marked line corresponding to each lane line at each point to be drawn, so that the thickness of the marked line decreases along the corresponding lane line extension direction.

[0095] Referring to the coordinate system in the foregoing embodiment, the direction of the first coordinate axis is the Y-axis direction, the first coordinate value is the Y-axis coordinate value, and the lane line extension direction specifically represents the extension direction of the lane line from the proximal end to the distal end. The Y-axis direction corresponds to the lane line extension direction, and it can be understood that as the lane line extends from the proximal end to the distal end, the corresponding Y-axis coordinate value gradually increases.

[0096] Specifically, the first coordinate value of the points to be drawn corresponding to a lane line is represented by y i , and the determination method of the thickness of the marked line corresponding to this lane line at each point to be drawn is as follows: ε i = αy i + β, where ε i represents the thickness of the marked line at the position of the point to be drawn corresponding to y i , and α and β represent adjustment factors, which can be set according to the actual situation. According to the coordinate system in the foregoing embodiment, α is a negative value, so that ε i decreases as y i increases, making the thickness of the marked line decrease along the corresponding lane line extension direction.

[0097] In this embodiment, by the magnitude of the first coordinate value of each point to be drawn corresponding to each lane line, the thickness of the marked line corresponding to each lane line at each point to be drawn is determined, so that the thickness of the marked line decreases along the corresponding lane line extension direction, thereby reducing the adhesion of the marked lane lines at the distal end and improving the subsequent lane line detection accuracy.

[0098] In one embodiment, the following steps may further be included: obtaining the category information of the marked points on each lane line based on the lane line scene graph; obtaining the category information of the points to be drawn corresponding to each lane line according to the category information of the marked points on each lane line. The step of drawing the marked lines corresponding to each lane line based on the position information of the points to be drawn corresponding to each lane line and the thickness information of the marked lines at each point to be drawn, and obtaining the lane line semantic label graph corresponding to the lane line scene graph may specifically be: drawing the marked lines corresponding to each lane line according to the position information, category information of the points to be drawn corresponding to each lane line, and the thickness information of the marked lines at each point to be drawn, and obtaining the lane line semantic label graph corresponding to the lane line scene graph.

[0099] The category information is used to indicate the lane line category to which the marked point belongs. The category information may specifically be color information, that is, different colors are used to indicate different lane line categories. For example, Figure 2 as shown, the lane line categories include solid lines and dashed lines. The solid line category may be indicated by a first color (such as red), and the dashed line category may be indicated by a second color (such as green). Thus, when drawing the marked lines corresponding to each lane line, in addition to controlling the thickness of the drawn line according to the position information of the points to be drawn, the color of the drawn line may also be controlled according to the lane line category. For example, Figure 3 as shown, the colors of the four marked lines are red, green, green, and red from left to right.

[0100] In this embodiment, the corresponding lane line category is indicated by the category information of the marked points. When performing lane line detection subsequently, not only the position of the lane line can be detected, but also the lane line category can be identified, making the lane line detection result more comprehensive.

[0101] In one embodiment, as Figure 4 shown, a method for determining a lane line detection model is provided. Taking the application of this method to a vehicle controller as an example, it includes the following steps S402 to step S408.

[0102] S402, obtaining a sample lane line scene graph.

[0103] The sample lane line scene graph represents a road scene graph containing lane lines, and can specifically be obtained by taking a picture of the road ahead during the vehicle's driving by a camera installed on the vehicle. The sample lane line scene graph is used as a training set for training a generative adversarial network model.

[0104] S404, obtaining the lane line semantic label graph corresponding to the sample lane line scene graph.

[0105] The lane line annotation method in any of the above embodiments may be adopted to obtain the lane line semantic label graph corresponding to the sample lane line scene graph.

[0106] S406. Based on the sample lane line scene graph and the lane line semantic label graph, train the generative adversarial network model to be trained to obtain the trained generative adversarial network model.

[0107] The generative adversarial network model includes a generator and a discriminator. The generator is used to generate a lane line semantic graph from the input sample lane line scene graph, and the discriminator aims to distinguish the lane line semantic label graph from the generated lane line semantic graph. The training objective of the generative adversarial network model is to minimize the difference between the lane line semantic graph and the lane line semantic label graph. The generator and the discriminator are respectively trained adversarially based on the loss function, and finally the optimal parameters of the network model are obtained.

[0108] S408. Determine the lane line detection model according to the generator in the trained generative adversarial network model.

[0109] The generator in the trained generative adversarial network model can be used as the lane line detection model. By inputting the lane line scene picture to be detected into the trained generator, the corresponding lane line semantic graph can be generated.

[0110] In the above method for determining the lane line detection model, different from the general generative adversarial network that uses the semantic graph as the input to generate the natural scene graph, by using the generative adversarial network reversely, the real lane line scene graph is used as the input for semantic segmentation to generate the lane line semantic graph, which is beneficial to removing the complex background and can generate the lane lines in the occluded area, with better robustness and stronger adaptability.

[0111] In one embodiment, as Figure 5 shown, a lane line detection method is provided. Taking the application of this method to a vehicle controller as an example, it includes the following steps S502 to S506.

[0112] S502. Obtain the lane line scene picture to be detected.

[0113] The lane line scene picture to be detected represents the road scene picture containing the lane line to be detected, and can be specifically obtained by taking pictures of the road ahead during the vehicle driving through the camera installed on the vehicle.

[0114] S504. Perform lane line detection on the lane line scene picture to be detected to obtain the lane line semantic graph, and the lane line semantic graph includes the position information of each pixel point.

[0115] The lane line detection model can be used to perform lane line detection on the lane line scene picture to be detected to obtain the lane line semantic graph. The lane line detection model can specifically be the generator in the trained generative adversarial network model. For the method for determining the lane line detection model, reference can be made to the above embodiments and will not be elaborated here.

[0116] S506. Determine the lane lines in the lane line scene graph to be detected based on the position information of each pixel point in the lane line semantic graph.

[0117] The pixel point represents a point included in the detected lane line. The position information of the pixel point can specifically be the coordinate value of the pixel point in the coordinate system established based on the lane line semantic graph. In one embodiment, taking the bottom left point of the lane line semantic graph as the coordinate origin, the vertical direction as the direction of the first coordinate axis (represented by the Y-axis) (vertically upward is the positive direction), and the horizontal direction as the direction of the second coordinate axis (represented by the X-axis) (horizontally to the right is the positive direction), a coordinate system is established. The position information of the pixel point is represented by (y, x), where y represents the Y-axis coordinate value of the pixel point and x represents the X-axis coordinate value of the pixel point.

[0118] In the above lane line detection method, inputting the lane line scene graph to be detected into the generator of the generative adversarial network model to generate the corresponding lane line semantic graph can achieve end-to-end detection, eliminating steps such as preprocessing and calculation of the lane line scene graph. The detection distance is farther, the number of manual parameter adjustments is less, and the robustness is better. Compared with the lane line detection method based on the probability graph semantic segmentation neural network that can only detect a fixed number of lane lines and cannot generate the lane lines in the occlusion area, using the generative adversarial network for lane line detection can simultaneously detect all the lane lines in the lane line scene graph and can generate the lane lines in the occlusion area, thereby improving the accuracy of lane line detection and being able to adapt to most complex road scenarios.

[0119] In one embodiment, the step of determining the lane lines in the lane line scene graph to be detected based on the position information of each pixel point in the lane line semantic graph may specifically include the following steps: obtaining the lane line contours of each connected region based on the position information of each pixel point in the lane line semantic graph; for each lane line contour, judging whether the lane line contour is an adhered lane line contour according to the position information of the contour points of the lane line contour; when the lane line contour is an adhered lane line contour, segmenting the adhered lane line contour according to the position information of the contour points of the adhered lane line contour to obtain the segmented lane line contour; determining the target lane line contour according to the non-adhered lane line contours and the segmented lane line contours in the lane line contour, and determining the lane lines in the lane line scene graph to be detected based on the contour points of each target lane line contour.

[0120] Among them, the contour point position information is specifically the first coordinate extreme position information of the contour points. Based on the first coordinate extreme position information of the contour points, it is determined whether the lane line contour is an adhered lane line contour, and the adhered lane line contour is segmented. The target lane line contour represents the contour finally used for curve fitting to determine the lane line. In addition, among the lane line contours obtained based on the lane line semantic map or the lane line contours obtained after segmentation, there may also be a situation where the same lane line is disconnected, that is, there may be multiple lane line contours corresponding to the same lane line. Therefore, the corresponding lane line contours need to be merged. The merging of the lane line contours will be described in detail in the following embodiments.

[0121] After detecting the lane line semantic map, the lane line semantic map can also be preprocessed first, which specifically includes the following steps: performing a closing operation on the lane line semantic map to fill the holes in the lane line semantic map, facilitating the subsequent search for a complete closed lane line contour; performing an opening operation on the lane line semantic map after the closing operation to reduce the adhesion of two different lane lines caused by the previous closing operation; performing binarization on the lane line semantic map after the opening operation to filter out some noise pixel points.

[0122] After preprocessing the lane line semantic map, based on the position distribution of the pixel points in the preprocessed lane line semantic map, the closed contours of each connected region are obtained, regarded as the initial lane line contours of each lane. The perimeter of each closed contour is calculated, and the closed contours with a perimeter less than the perimeter threshold are removed to filter out the noise lane lines, obtaining the lane line contours after filtering out the noise, and then performing subsequent segmentation and merging processing.

[0123] In one embodiment, the contour point position information includes the first coordinate value in the direction of the first coordinate axis in the coordinate system established based on the lane line semantic map, and the direction of the first coordinate axis represents the direction corresponding to the lane line extension direction; the step of determining whether the lane line contour is an adhered lane line contour according to the contour point position information of the lane line contour may specifically include the following steps: taking any contour point of the lane line contour as the starting point, and sequentially obtaining the first coordinate values of each contour point of the lane line contour according to the contour direction of the lane line contour; obtaining the first coordinate maximum value and the first coordinate minimum value according to the sequentially obtained first coordinate values of each contour point; determining whether the lane line contour is an adhered lane line contour according to the number of the first coordinate maximum values and the number of the first coordinate minimum values.

[0124] Specifically, when at least one of the number of first coordinate maxima and the number of first coordinate minima is greater than 1, the lane line contour is determined to be an overlapping lane line contour. The overlapping lane line contour can be understood as a contour containing at least two contours corresponding to different lane lines. When both the number of first coordinate maxima and the number of first coordinate minima are 1, the lane line contour is determined to be a non-overlapping lane line contour. The non-overlapping lane line contour can be understood as a contour corresponding to the same lane line.

[0125] The first coordinate axis direction is the Y-axis direction, and the first coordinate value is the Y-axis coordinate value. Taking any contour point of the lane line contour as the starting point, in accordance with the contour direction of the lane line contour, the Y-axis coordinate values of each contour point of the lane line contour are obtained in sequence. Based on the Y-axis coordinate values of each contour point obtained in sequence, the Y-axis coordinate maximum value and the Y-axis coordinate minimum value are obtained. The contour points corresponding to the Y-axis coordinate maximum value and the Y-axis coordinate minimum value are respectively called the maximum value point and the minimum value point. The method for finding the extreme value points can be specifically as follows: Taking any contour point of the lane line contour as the starting point, in accordance with the counterclockwise direction sequence of the lane line contour, each contour point is stored in sequence to obtain a contour point set. It can be understood that the last contour point stored in the contour point set is the right adjacent point of the first contour point stored. If the difference between the Y-axis coordinate values of a certain contour point and its left and right adjacent N points is greater than a threshold, then this contour point is a minimum value point. If the difference between the Y-axis coordinate values of a certain contour point and its left and right adjacent N points is greater than a threshold, then this contour point is a maximum value point. Among them, N is a positive integer and can be set according to actual requirements. For example, it can be set to 2 or 3. The threshold is a positive number and can be set according to actual requirements, and no limitation is made here.

[0126] It should be noted that among the extreme value points found, there may be a situation where the number of maximum value points and minimum value points does not exactly match. If multiple extreme value points of the same nature (maximum value points or minimum value points) that are relatively close (relatively close means that the contour point storage indices are relatively close) are found, then the one with a higher ranking is taken, and the redundant misdetected extreme value points are filtered out. The probability of the above-mentioned situation of non-matching quantities is very small. The following takes the case where the maximum value and the minimum value correspond one by one as an example for explanation.

[0127] Figure 6 and Figure 7 respectively show the schematic diagrams of the lane line contours in an embodiment. As can be seen from the figure, Figure 6 there are two Y-axis coordinate maximum values (corresponding to the contour points YE_max1 and YE_max2 respectively) and two Y-axis coordinate minimum values (corresponding to the contour points YE_min1 and YE_min2 respectively), that is, Figure 6 the lane line contour shown is an overlapping lane line contour, containing the contours corresponding to two different lane lines. Figure 7There are three maximum values of the Y-axis coordinates (corresponding to the contour points YE_max1, YE_max2, and YE_max3 respectively) and three minimum values of the Y-axis coordinates (corresponding to the contour points YE_min1, YE_min2, and YE_min3 respectively), that is Figure 7 The lane line contour shown is an overlapping lane line contour, which contains the contours corresponding to three different lane lines.

[0128] When the detected lane lines are overlapping, it will affect the accuracy of subsequent curve fitting, and it may determine multiple overlapping lane lines as the same lane line. Therefore, it is necessary to segment the overlapping lane lines.

[0129] In one embodiment, when the lane line contour is an overlapping lane line contour, the step of segmenting the overlapping lane line contour according to the position information of the contour points of the lane line contour to obtain the segmented lane line contour may specifically include the following steps: taking the first coordinate maximum value point corresponding to any first coordinate maximum value as the starting point, and sorting the first coordinate maximum value points according to the contour direction of the overlapping lane line contour; obtaining the segmented lane line contour based on the contour points between the first coordinate maximum value points with adjacent serial numbers.

[0130] Specifically, store each contour point in sequence according to the counterclockwise direction of the lane line contour to obtain a contour point set, and take the first coordinate maximum value point closest to the first contour point stored in the contour point set as the starting point, and sort the first coordinate maximum value points according to the counterclockwise direction of the lane line contour. Figure 7 Taking the overlapping lane line contour shown as an example, assuming that YE_max1 is used as the starting point and the Y-axis maximum value points are sorted in the counterclockwise direction along the contour, the serial numbers of YE_max1, YE_max2, and YE_max3 are 1, 2, and 3 in sequence. Taking YE_max1, YE_max2, and YE_max3 as the segmentation points, the overlapping lane line contour is divided into 3 segments to obtain 3 segmented lane line contours. The contour points of the first segmented lane line contour include the contour points between YE_max1 and YE_max2, the contour points of the second segmented lane line contour include the contour points between YE_max2 and YE_max3, and the contour points of the third segmented lane line contour include the contour points between YE_max3 and YE_max1.

[0131] In the above embodiment, the overlapping lane line contour is segmented based on the first coordinate maximum value points. Subsequently, curve fitting can be performed on the contour points of each obtained segmented lane line contour respectively, avoiding the influence of lane line overlap on curve fitting and improving the accuracy of the lane line after fitting.

[0132] When a situation where the detected lane lines are disconnected in the same lane line is detected, it will also affect the accuracy of subsequent curve fitting. It may determine multiple disconnected lane lines belonging to the same lane line as multiple lane lines. Therefore, it is necessary to merge the disconnected lane lines.

[0133] In one embodiment, the step of determining the target lane line profile according to the non-adhesive lane line profile and the segmented lane line profile in the lane line profile may specifically include the following steps: For any two lane line profiles in the segmented profile, determine whether the two lane line profiles correspond to the same lane line according to the position information of the profile points of the two lane line profiles. The segmented profile includes the non-adhesive lane line profile and the segmented lane line profile; when the two lane line profiles correspond to the same lane line, merge the profile points of the two lane line profiles to obtain a merged lane line profile; determine the target lane line profile according to the lane line profiles corresponding to different lane lines and the merged lane line profile in the segmented profile.

[0134] The step of determining whether the two lane line profiles correspond to the same lane line according to the position information of the profile points of the two lane line profiles may specifically include the following steps: Obtain the position information of the first lowest point and the first highest point of the first lane line profile, and obtain the position information of the second lowest point and the second highest point of the second lane line profile. The lowest point and the highest point are determined based on the first coordinate value of each profile point, and the first coordinate value of the first highest point is greater than or equal to the first coordinate value of the second highest point; respectively perform linear fitting on the profile points of the first lane line profile and the second lane line profile to obtain a first fitting line and a second fitting line, and respectively calculate the slopes of the first fitting line and the second fitting line to obtain a first slope and a second slope; determine whether the two lane line profiles correspond to the same lane line according to the slope difference between the first slope and the second slope, the first distance between the first lane line profile and the second lane line profile, and the second distance between the first lowest point and the second highest point.

[0135] Among them, the first lowest point refers to the lowest point of the first lane line profile, that is, the profile point corresponding to the minimum Y-axis coordinate value in the first lane line profile. The first highest point refers to the highest point of the first lane line profile, that is, the profile point corresponding to the maximum Y-axis coordinate value in the first lane line profile. The second lowest point refers to the lowest point of the second lane line profile, that is, the profile point corresponding to the minimum Y-axis coordinate value in the second lane line profile. The second highest point refers to the highest point of the second lane line profile, that is, the profile point corresponding to the maximum Y-axis coordinate value in the second lane line profile. The Y-axis coordinate value of the first highest point is greater than or equal to the Y-axis coordinate value of the second highest point. The first distance refers to the distance between the first lane line profile and the second lane line profile, and the second distance represents the distance between the lowest point of the first lane line profile and the highest point of the second lane line profile.

[0136] For example, Figure 8The schematic diagram of the lane line profile in an embodiment is shown. The first lane line profile eline is located above the second lane line profile line. The points ps1 and pe1 represent the highest point and the lowest point of the first lane line profile eline respectively, and the points ps2 and pe2 represent the highest point and the lowest point of the second lane line profile line respectively.

[0137] The distance between the first lane line profile eline and the second lane line profile line can be represented by the distance from a vertex of one profile to the fitting line of the other profile. Specifically, it can be the distance from the lowest point pe1 of the first lane line profile eline to the fitting line of the second lane line profile line, or the distance from the highest point ps1 of the first lane line profile eline to the fitting line of the second lane line profile line, or the distance from the lowest point pe2 of the second lane line profile line to the fitting line of the first lane line profile eline, or the distance from the highest point ps2 of the second lane line profile line to the fitting line of the first lane line profile eline.

[0138] In one embodiment, when the slope difference is less than the first threshold, and the first distance is less than the second threshold, and the second distance is less than the third threshold, it is determined that the two lane line profiles correspond to the same lane line. That is, when the slopes of the two lane line profiles are close and the first distance and the second distance are small, it is considered that the two lane line profiles belong to the same lane line, and the profile points of the two lane line profiles can be merged to obtain the merged lane line profile. Among them, the first threshold, the second threshold, and the third threshold can all be set according to the actual situation.

[0139] It should be noted that in addition to using the above slope difference, the first distance, and the second distance as the parameters for judging whether the two lane line profiles correspond to the same lane line, other parameters can also be used for judgment. For example, as Figure 8 shown, the parameters can also include: the absolute value difference of the Y-axis coordinate values (denoted by Ly) between the lowest point pe1 of the first lane line profile eline and the lowest point pe2 of the second lane line profile line, the absolute value difference of the X-axis coordinate values (denoted by Lx) between the lowest point pe1 of the first lane line profile eline and the lowest point pe2 of the second lane line profile line, the absolute value difference of the Y-axis coordinate values (denoted by Hy) between the highest point ps1 of the first lane line profile eline and the highest point ps2 of the second lane line profile line, and the absolute value difference of the X-axis coordinate values (denoted by Hx) between the highest point ps1 of the first lane line profile eline and the highest point ps2 of the second lane line profile line. When Ly and Hy are large and Lx and Hx are small, it is considered that the first lane line profile eline and the second lane line profile line are very likely to correspond to the same lane line.

[0140] In the above embodiments, by merging multiple lane line contours belonging to the same lane line, the disconnected lane line contours can be effectively merged, improving the accuracy of subsequent curve fitting.

[0141] After performing the above-mentioned splitting and merging processes on the lane line contours that need to be split or merged in the detected lane line contours, the final target lane line contours are obtained. The contour points of each target lane line contour can be subjected to cubic curve fitting to obtain curve fitting parameters, and the fitted lane lines can be displayed as the final lane line detection results.

[0142] It should be noted that if there are no contours that need to be split or merged in the detected lane line contours, the detected lane line contours are the target lane line contours; if there are adhesive lane line contours in the detected lane line contours, the adhesive lane line contours are split to obtain split lane line contours, and the non-adhesive lane line contours in the detected lane line contours and the split lane line contours obtained by splitting are used as the split contours. If there are no contours that need to be merged in the split contours, the split contours are the target lane line contours; if there are contours that need to be merged in the split contours, the contours that need to be merged are merged to obtain merged lane line contours, and the lane line contours that do not need to be merged in the split contours and the merged lane line contours obtained by merging are used as the target lane line contours.

[0143] In one embodiment, the lane line semantic map further includes the category information of each pixel point, and the category information is used to indicate the lane line category to which the pixel point belongs; after obtaining the target lane line contours, the lane line category corresponding to each target lane line contour can also be determined according to the category information of each pixel point in each target lane line contour.

[0144] Specifically, for each target lane line contour, the number of pixel points corresponding to each category information in the target lane line contour is counted, and the lane line category indicated by the category information with the largest corresponding number of pixel points is determined as the lane line category corresponding to the target lane line contour.

[0145] For example, the category information is the color information in the lane line semantic map. The lane line categories include solid lines and dashed lines. The solid line category is indicated by red, and the dashed line category is indicated by green. Specifically, it is possible to determine whether a pixel point belongs to a solid line or a dashed line according to the RGB value of the pixel point. If the difference between the value of the R channel and the value of the G channel of the pixel point is greater than a preset threshold, it is considered that the pixel point corresponds to the solid line category. If the difference between the value of the G channel and the value of the R channel of the pixel point is greater than a preset threshold, it is considered that the pixel point corresponds to the dashed line category. For each target lane line contour, count the number of pixel points corresponding to the solid line category and the dashed line category respectively. If the number of pixel points corresponding to the solid line category is greater than the number of pixel points corresponding to the dashed line category, it is determined that the lane line category corresponding to the target lane line contour is a solid line. If the number of pixel points corresponding to the dashed line category is greater than the number of pixel points corresponding to the solid line category, it is determined that the lane line category corresponding to the target lane line contour is a dashed line.

[0146] In the above embodiment, the lane line category corresponding to the target lane line contour is determined by the category information of each pixel point included in the target lane line contour. Thus, not only can the position of the lane line in the lane line scene map be detected, but also the lane line category can be recognized, making the lane line detection result more comprehensive.

[0147] It should be understood that although Figure 1 、 4 、the steps in the flowcharts of 5 are shown in sequence according to the indication of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, Figure 1 、 4 、at least a part of the steps in 5 may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0148] In one embodiment, as Figure 9 shown, a lane line annotation device 900 is provided, including: an annotation point information acquisition module 910, a to-be-drawn point information determination module 920, an annotation line information determination module 930, and an annotation line drawing module 940, where:

[0149] The annotation point information acquisition module 910 is configured to acquire the position information of the annotation points on each lane line based on the lane line scene map.

[0150] The to-be-drawn point information determination module 920 is configured to determine the position information of the to-be-drawn points corresponding to each lane line according to the position information of the annotation points on each lane line.

[0151] The marked line information determination module 930 is configured to determine the thickness information of the marked lines corresponding to each lane line at each point to be drawn based on the position information of the points to be drawn corresponding to each lane line.

[0152] The marked line drawing module 940 is configured to draw the marked lines corresponding to each lane line according to the position information of the points to be drawn corresponding to each lane line and the thickness information of the marked lines at each point to be drawn, so as to obtain the lane line semantic label map corresponding to the lane line scene map.

[0153] In one embodiment, the point-to-be-drawn information determination module 920 is specifically configured to: perform linear interpolation according to the position information of adjacent marked points among the marked points on each lane line to obtain the position information of the interpolation points between adjacent marked points; and determine the position information of the points to be drawn corresponding to each lane line based on the position information of the marked points and the interpolation points on each lane line.

[0154] In one embodiment, the position information includes the first coordinate value in the direction of the first coordinate axis in the coordinate system established based on the lane line scene map, and the direction of the first coordinate axis represents the direction corresponding to the lane line extension direction; the marked line information determination module 930 is specifically configured to: determine the thickness of the marked lines corresponding to each lane line at each point to be drawn based on the magnitude of the first coordinate value of each point to be drawn corresponding to each lane line, so that the thickness of the marked lines decreases along the corresponding lane line extension direction.

[0155] In one embodiment, the marked point information acquisition module 910 is further configured to obtain the category information of the marked points on each lane line based on the lane line scene map; the point-to-be-drawn information determination module 920 is further configured to obtain the category information of the points to be drawn corresponding to each lane line according to the category information of the marked points on each lane line; the marked line drawing module 940 is specifically configured to draw the marked lines corresponding to each lane line according to the position information, category information of the points to be drawn corresponding to each lane line, and the thickness information of the marked lines at each point to be drawn, so as to obtain the lane line semantic label map corresponding to the lane line scene map.

[0156] In one embodiment, as Figure 10 shown, a determination device 1000 for a lane line detection model is provided, including: a sample acquisition module 1010, a lane line marking module 1020, a model training module 1030, and a model determination module 1040, where:

[0157] The sample acquisition module 1010 is configured to acquire a sample lane line scene map.

[0158] The lane line annotation module 1020 is used to perform lane line annotation on the sample lane line scene graph by using the lane line annotation method in any of the foregoing embodiments, so as to obtain the lane line semantic label graph corresponding to the sample lane line scene graph.

[0159] The model training module 1030 is used to train the to-be-trained generative adversarial network model based on the sample lane line scene graph and the lane line semantic label graph, so as to obtain the trained generative adversarial network model.

[0160] The model determination module 1040 is used to determine the lane line detection model according to the generator in the trained generative adversarial network model.

[0161] In one embodiment, as Figure 11 shown, a lane line detection device 1100 is provided, including: a to-be-detected picture acquisition module 1110, a lane line detection module 1120, and a lane line determination module 1130, where:

[0162] The to-be-detected picture acquisition module 1110 is used to acquire the to-be-detected lane line scene graph.

[0163] The lane line detection module 1120 is used to perform lane line detection on the to-be-detected lane line scene graph by using the lane line detection model determined by the method for determining the lane line detection model in any of the foregoing embodiments, so as to obtain the lane line semantic graph, and the lane line semantic graph includes the position information of each pixel point.

[0164] The lane line determination module 1130 is used to determine the lane lines in the to-be-detected lane line scene graph based on the position information of each pixel point in the lane line semantic graph.

[0165] In one embodiment, the lane line determination module 1130 includes: a contour acquisition unit, a first judgment unit, a segmentation unit, and a determination unit. The contour acquisition unit is used to obtain the lane line contours of each connected region based on the position information of each pixel point in the lane line semantic graph; the first judgment unit is used to judge whether the lane line contour is an adhered lane line contour for each lane line contour according to the contour point position information of the lane line contour; the segmentation unit is used to segment the adhered lane line contour according to the contour point position information of the adhered lane line contour when the lane line contour is an adhered lane line contour, so as to obtain the segmented lane line contour; the determination unit is used to determine the target lane line contour according to the non-adhered lane line contour and the segmented lane line contour in the lane line contour, and determine the lane lines in the to-be-detected lane line scene graph based on the contour points of each target lane line contour.

[0166] In one embodiment, the profile point position information includes a first coordinate value in the direction of a first coordinate axis in a coordinate system established based on a lane line semantic map, and the direction of the first coordinate axis represents a direction corresponding to the extension direction of the lane line; the first determination unit is specifically configured to: take any profile point of the lane line profile as a starting point, and sequentially obtain the first coordinate values of the profile points of the lane line profile according to the profile direction of the lane line profile; obtain a first coordinate maximum value and a first coordinate minimum value according to the sequentially obtained first coordinate values of the profile points; and determine whether the lane line profile is an overlapping lane line profile according to the number of first coordinate maximum values and the number of first coordinate minimum values.

[0167] In one embodiment, the first determination unit is further configured to determine that the lane line profile is an overlapping lane line profile when at least one of the number of first coordinate maximum values and the number of first coordinate minimum values is greater than 1.

[0168] In one embodiment, the segmentation unit is specifically configured to: take the first coordinate maximum value point corresponding to any first coordinate maximum value as a starting point, and sort the first coordinate maximum value points according to the profile direction of the overlapping lane line profile; and obtain a segmented lane line profile based on the profile points between the first coordinate maximum value points with adjacent serial numbers.

[0169] In one embodiment, the determination unit is specifically configured to: for any two lane line profiles in the segmented profile, determine whether the two lane line profiles correspond to the same lane line according to the profile point position information of the two lane line profiles, and the segmented profile includes non-overlapping lane line profiles and segmented lane line profiles; when the two lane line profiles correspond to the same lane line, merge the profile points of the two lane line profiles to obtain a merged lane line profile; and determine the target lane line profile according to the lane line profiles corresponding to different lanes and the merged lane line profile in the segmented profile.

[0170] In one embodiment, the second determination unit is specifically configured to: obtain the position information of the first lowest point and the first highest point of the first lane line profile, and obtain the position information of the second lowest point and the second highest point of the second lane line profile, where the lowest point and the highest point are determined based on the first coordinate values of the profile points, and the first coordinate value of the first highest point is greater than or equal to the first coordinate value of the second highest point; respectively perform linear fitting on the profile points of the first lane line profile and the second lane line profile to obtain a first fitting line and a second fitting line, respectively calculate the slopes of the first fitting line and the second fitting line to obtain a first slope and a second slope; and determine whether the two lane line profiles correspond to the same lane line according to the slope difference between the first slope and the second slope, the first distance between the first lane line profile and the second lane line profile, and the second distance between the first lowest point and the second highest point.

[0171] In one embodiment, the first distance includes any one of the following: the distance from the first lowest point to the second fitting line; the distance from the first highest point to the second fitting line; the distance from the second lowest point to the first fitting line; the distance from the second highest point to the first fitting line. The second determination unit is further configured to: when the slope difference is less than the first threshold, the first distance is less than the second threshold, and the second distance is less than the third threshold, determine that the two lane line profiles correspond to the same lane line.

[0172] In one embodiment, the lane line semantic map further includes category information of each pixel point, and the category information is used to indicate the lane line category to which the pixel point belongs. The lane line determination module 1130 is further configured to: determine the lane line category corresponding to each target lane line profile according to the category information of each pixel point in each target lane line profile.

[0173] In one embodiment, the lane line determination module 1130 is specifically configured to: for each target lane line profile, count the number of pixel points corresponding to each category information in the target lane line profile, and determine the lane line category indicated by the category information with the largest corresponding number of pixel points as the lane line category corresponding to the target lane line profile.

[0174] For the specific limitations on lane line annotation, determination of the lane line detection model, and the lane line detection device, reference may be made to the limitations on lane line annotation, determination of the lane line detection model, and the lane line detection method in the foregoing text, which will not be elaborated herein. Each module in the above-mentioned lane line annotation, determination of the lane line detection model, and the lane line detection device can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

[0175] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as Figure 12 shown. The computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a lane line annotation, determination of the lane line detection model, and a lane line detection method.

[0176] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structural diagram may be as shown in Figure 13 . The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner. The wireless manner can be implemented through WIFI, a carrier network, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a lane line annotation, determination of a lane line detection model, and a lane line detection method. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device may be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0177] Those skilled in the art can understand that Figure 12 or Figure 13 the structure shown in is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0178] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the steps in the above-mentioned method embodiments are implemented.

[0179] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by the processor, the steps in the above-mentioned method embodiments are implemented.

[0180] In one embodiment, a computer program product or a computer program is provided. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the steps in the above-mentioned method embodiments.

[0181] It should be understood that the terms "first", "second", etc. in the above embodiments are only for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features.

[0182] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0183] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0184] The above-described embodiments only represent several implementation manners of the present application. Their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A lane line annotation method, characterized in that, The method includes: Obtaining the position information of the labeled points on each lane line based on the lane line scene graph; Determining the position information of the points to be drawn corresponding to each lane line according to the position information of the labeled points on each lane line; Determining the thickness information of the labeled lines corresponding to each lane line at each point to be drawn based on the position information of the points to be drawn corresponding to each lane line; Drawing the labeled lines corresponding to each lane line according to the position information of the points to be drawn corresponding to each lane line and the thickness information of the labeled lines at each point to be drawn, to obtain the lane line semantic label graph corresponding to the lane line scene graph, where determining the position information of the points to be drawn corresponding to each lane line according to the position information of the labeled points on each lane line includes: Performing linear interpolation according to the position information of adjacent labeled points among the labeled points on each lane line to obtain the position information of the interpolation points between the adjacent labeled points; Determining the position information of the points to be drawn corresponding to each lane line based on the position information of the labeled points and the interpolation points on each lane line, where the position information includes the first coordinate value in the direction of the first coordinate axis in the coordinate system established based on the lane line scene graph, and the direction of the first coordinate axis represents the direction corresponding to the lane line extension direction; Determining the thickness information of the labeled lines corresponding to each lane line at each point to be drawn based on the position information of the points to be drawn corresponding to each lane line includes: Determining the thickness of the labeled lines corresponding to each lane line at each point to be drawn based on the magnitude of the first coordinate value of each point to be drawn corresponding to each lane line, such that the thickness of the labeled lines decreases along the corresponding lane line extension direction.

2. The method according to claim 1, wherein It further includes: Obtaining the category information of the labeled points on each lane line based on the lane line scene graph; obtaining the category information of the points to be drawn corresponding to each lane line according to the category information of the labeled points on each lane line; Drawing the labeled lines corresponding to each lane line according to the position information of the points to be drawn corresponding to each lane line and the thickness information of the labeled lines at each point to be drawn, to obtain the lane line semantic label graph corresponding to the lane line scene graph, includes: Drawing the labeled lines corresponding to each lane line according to the position information, category information of the points to be drawn corresponding to each lane line and the thickness information of the labeled lines at each point to be drawn, to obtain the lane line semantic label graph corresponding to the lane line scene graph.

3. A method for determining a lane line detection model, characterized in that, The method includes: Obtaining a sample lane line scene graph; Performing lane line annotation on the sample lane line scene graph using the method according to any one of claims 1 to 2 to obtain the lane line semantic label graph corresponding to the sample lane line scene graph; Training the to-be-trained generative adversarial network model based on the sample lane line scene graph and the lane line semantic label graph to obtain the trained generative adversarial network model; Determining a lane line detection model according to the generator in the trained generative adversarial network model.

4. A lane line detection method, characterized in that, The method includes: Obtaining a to-be-detected lane line scene graph; Using the lane line detection model determined by the method described in claim 3, perform lane line detection on the to-be-detected lane line scene graph to obtain a lane line semantic graph, where the lane line semantic graph includes the position information of each pixel point; Based on the position information of each pixel point in the lane line semantic graph, determine the lane lines in the to-be-detected lane line scene graph. Among them, the determining the lane lines in the to-be-detected lane line scene graph based on the position information of each pixel point in the lane line semantic graph includes: Based on the position information of each pixel point in the lane line semantic graph, obtain the lane line contours of each connected region; for each lane line contour, according to the contour point position information of the lane line contour, determine whether the lane line contour is an adhered lane line contour; when the lane line contour is an adhered lane line contour, perform segmentation on the adhered lane line contour according to the contour point position information of the adhered lane line contour to obtain segmented lane line contours; according to the non-adhered lane line contours and the segmented lane line contours in the lane line contour, determine the target lane line contours, and based on the contour points of each target lane line contour, determine the lane lines in the to-be-detected lane line scene graph.

5. The method according to claim 4, characterized in that, The contour point position information includes the first coordinate value in the direction of the first coordinate axis in the coordinate system established based on the lane line semantic graph, and the direction of the first coordinate axis represents the direction corresponding to the lane line extension direction; According to the contour point position information of the lane line contour, determining whether the lane line contour is an adhered lane line contour includes: Taking any contour point of the lane line contour as the starting point, and in accordance with the contour direction of the lane line contour, sequentially obtain the first coordinate values of each contour point of the lane line contour; According to the sequentially obtained first coordinate values of each contour point, obtain the first coordinate maximum value and the first coordinate minimum value; According to the number of the first coordinate maximum values and the number of the first coordinate minimum values, determine whether the lane line contour is an adhered lane line contour.

6. The method according to claim 5, wherein When at least one of the number of the first coordinate maximum values and the number of the first coordinate minimum values is greater than 1, determine that the lane line contour is an adhered lane line contour.

7. The method according to claim 6, characterized in that Performing segmentation on the adhered lane line contour according to the contour point position information of the adhered lane line contour to obtain segmented lane line contours includes: Taking the first coordinate maximum value point corresponding to any first coordinate maximum value as the starting point, and in accordance with the contour direction of the adhered lane line contour, sort each first coordinate maximum value point; Based on the contour points between the first coordinate maximum value points with adjacent serial numbers, obtain the segmented lane line contours.

8. The method according to claim 4, wherein According to the non-adhered lane line contours and the segmented lane line contours in the lane line contour, determining the target lane line contours includes: For any two lane line contours in the segmented contour, according to the contour point position information of the two lane line contours, determine whether the two lane line contours correspond to the same lane line, and the segmented contour includes the non-adhered lane line contours and the segmented lane line contours; When the two lane line contours correspond to the same lane line, merge the contour points of the two lane line contours to obtain a merged lane line contour; Determine the target lane line contour according to the lane line contours corresponding to different lane lines in the segmented contours and the merged lane line contour.

9. The method according to claim 8, wherein Judging whether the two lane line contours correspond to the same lane line according to the position information of the contour points of the two lane line contours includes: Obtain the position information of the first lowest point and the first highest point of the first lane line contour, and obtain the position information of the second lowest point and the second highest point of the second lane line contour. The lowest point and the highest point are determined based on the first coordinate values of each contour point, and the first coordinate value of the first highest point is greater than or equal to the first coordinate value of the second highest point; Perform linear fitting on the contour points of the first lane line contour and the second lane line contour respectively to obtain a first fitting line and a second fitting line, and calculate the slopes of the first fitting line and the second fitting line respectively to obtain a first slope and a second slope; Judge whether the two lane line contours correspond to the same lane line according to the slope difference between the first slope and the second slope, the first distance between the first lane line contour and the second lane line contour, and the second distance between the first lowest point and the second highest point.

10. The method according to claim 9, characterized in that, The first distance includes any one of the following: the distance from the first lowest point to the second fitting line; the distance from the first highest point to the second fitting line; the distance from the second lowest point to the first fitting line; the distance from the second highest point to the first fitting line; When the slope difference is less than the first threshold, the first distance is less than the second threshold, and the second distance is less than the third threshold, it is determined that the two lane line contours correspond to the same lane line.

11. The method according to any one of claims 5 to 10, characterized in that The lane line semantic map further includes the category information of each pixel point, and the category information is used to indicate the lane line category to which the pixel point belongs; The method further includes: determining the lane line category corresponding to each target lane line contour according to the category information of each pixel point in each target lane line contour.

12. The method according to claim 11, wherein Determining the lane line category corresponding to each target lane line contour according to the category information of each pixel point in each target lane line contour includes: For each target lane line contour, count the number of pixel points corresponding to each category information in the target lane line contour, and determine the lane line category indicated by the category information with the largest number of corresponding pixel points as the lane line category corresponding to the target lane line contour.

13. A lane marking device, characterized in that, The device includes: A marked point information acquisition module, configured to acquire the position information of the marked points on each lane line based on the lane line scene map; A to-be-drawn point information determination module, configured to determine the position information of the to-be-drawn points corresponding to each lane line according to the position information of the marked points on each lane line; A marked line information determination module, configured to determine the thickness information of the marked lines corresponding to each lane line at each to-be-drawn point based on the position information of the to-be-drawn points corresponding to each lane line; An annotation line drawing module, configured to draw annotation lines corresponding to each of the lane lines according to the position information of the points to be drawn corresponding to each of the lane lines and the thickness information of the annotation lines at each of the points to be drawn, so as to obtain a lane line semantic label map corresponding to the lane line scene map, wherein the point information determination module is specifically configured to: Perform linear interpolation according to the position information of adjacent annotation points among the annotation points on each of the lane lines to obtain the position information of the interpolation points between the adjacent annotation points; based on the position information of the annotation points and the interpolation points on each of the lane lines, determine the position information of the points to be drawn corresponding to each of the lane lines, wherein the position information includes a first coordinate value in the direction of a first coordinate axis in a coordinate system established based on the lane line scene map, and the direction of the first coordinate axis represents a direction corresponding to the lane line extension direction, and the annotation line information determination module is specifically configured to: Based on the magnitudes of the first coordinate values of the points to be drawn corresponding to each of the lane lines, determine the thickness magnitudes of the annotation lines corresponding to each of the lane lines at each of the points to be drawn, so that the thickness magnitudes of the annotation lines decrease along the corresponding lane line extension direction.

14. An apparatus for determining a lane line detection model, characterized in that The apparatus includes: A sample acquisition module, configured to acquire a sample lane line scene map; A lane line annotation module, configured to perform lane line annotation on the sample lane line scene map by using the method according to any one of claims 1 to 2 to obtain a lane line semantic label map corresponding to the sample lane line scene map; A model training module, configured to train a to-be-trained generative adversarial network model based on the sample lane line scene map and the lane line semantic label map to obtain a trained generative adversarial network model; A model determination module, configured to determine a lane line detection model according to a generator in the trained generative adversarial network model.

15. A lane line detection device, characterized in that, The apparatus includes: A to-be-detected image acquisition module, configured to acquire a to-be-detected lane line scene map; A lane line detection module, configured to perform lane line detection on the to-be-detected lane line scene map by using the lane line detection model determined by the method according to claim 3 to obtain a lane line semantic map, and the lane line semantic map includes the position information of each pixel point; A lane line determination module, configured to determine the lane lines in the to-be-detected lane line scene map based on the position information of each pixel point in the lane line semantic map, wherein the lane line determination module includes a contour acquisition unit, a first judgment unit, a segmentation unit, and a determination unit; The contour acquisition unit is configured to obtain the lane line contours of each connected region based on the position information of each pixel point in the lane line semantic map; the first judgment unit is configured to, for each lane line contour, judge whether the lane line contour is an adhered lane line contour according to the contour point position information of the lane line contour; the segmentation unit is configured to, when the lane line contour is an adhered lane line contour, segment the adhered lane line contour according to the contour point position information of the adhered lane line contour to obtain segmented lane line contours; the determination unit is configured to determine the target lane line contours according to the non-adhered lane line contours and the segmented lane line contours in the lane line contours, and determine the lane lines in the lane line scene graph to be detected based on the contour points of each of the target lane line contours.

16. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 12 are implemented.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 12 are implemented.

Images