Lane line identification method and device

Through the lane line recognition method based on relative starting points, the lane line detection model and clustering algorithm are used to solve the problem of difficult instance distinction in lane line recognition in complex topological structures, and a more accurate and stable lane line recognition effect is achieved.

CN118968446APending Publication Date: 2024-11-15GAC AION NEW ENERGY AUTOMOBILE CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410974211.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-19
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

When the existing lane line detection method deals with lane lines of complex topological structures, it is difficult to effectively distinguish different lane line instances, especially in scenarios such as intersections and multi-lane merging, the example distinction effect is poor.

Method used

The lane line recognition method based on relative starting point coordinates is adopted, and the lane instance clustering algorithm for dynamically adjusting the starting point is obtained by obtaining the foreground points and relative starting point coordinates, and performing clustering processing to determine the lane The coordinate values ​​of line instances are trained using backbone network, FPN network and detection head for feature extraction and detection, and are trained with Focal-loss and Push-PullLoss loss functions.

Benefits of technology

The accurate division and identification of lane lines is achieved, the stability and accuracy of lane lines identification in complex topological structures is improved, the problem of close starting points of lane lines is overcome, and it is suitable for difficult scenarios such as double lines and dense lane lines.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118968446A_ABST
    Figure CN118968446A_ABST
Patent Text Reader

Abstract

The invention provides a lane line identification method and device. The method comprises the following steps: collecting a road scene picture containing a lane line; the road scene picture comprises annotation information; performing lane line identification on the road scene picture by adopting a pre-constructed lane line detection model to obtain a lane line foreground point coordinate and a relative starting point coordinate; obtaining a pre-constructed lane instance clustering algorithm decoder based on the dynamic adjustment starting point; and processing the foreground point coordinate and the relative starting point coordinate of the lane line through a lane instance clustering algorithm decoder to obtain a lane line coordinate value containing a lane line instance. According to the method and the device, lane line instances in scenes of double lines, dense lane lines and the like can be expressed through relative starting point coordinates, so that the trouble of difficulty in classification of closer starting points is overcome, and accurate detection of the lane line instances is further realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a lane line recognition method and device. Background Art

[0002] In the rapid development of intelligent driving technology, lane detection is a key link, and its purpose is to accurately identify lane lines in images, provide accurate path information for autonomous vehicles, and ensure the safety and stability of vehicle driving. However, the lane detection task is extremely challenging due to its complexity. Especially when dealing with lane lines with complex topological structures, traditional methods often have difficulty in effectively distinguishing different lane line instances. In existing solutions, when facing lane lines with complex topological structures, such as intersections and multi-lane merging, existing methods still face the problem of poor instance differentiation. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a lane line recognition method and device, which can better represent the division between different lane lines based on the relative starting point coordinates, thereby realizing effective recognition of lane lines, and making it more suitable for lane line recognition scenarios with complex topological structures.

[0004] The first aspect of the present application provides a lane line recognition method, comprising:

[0005] Collecting a road scene picture including lane lines; the road scene picture includes annotation information;

[0006] Using a pre-built lane line detection model to perform lane line recognition on the road scene image, and obtaining the lane line foreground point coordinates and relative starting point coordinates; the relative starting point coordinates include the relative starting point H coordinates and the relative starting point W coordinates;

[0007] Get the pre-built decoder for the lane instance clustering algorithm based on dynamically adjusted starting points;

[0008] The lane line foreground point coordinates and the relative starting point coordinates are processed by the lane line instance clustering algorithm decoder to obtain a lane line coordinate value containing a lane line instance.

[0009] Furthermore, the lane line detection model includes a backbone network, an FPN network and a detection head;

[0010] The detection head includes a lane feature detection head and a relative starting point detection head;

[0011] The loss function of the lane feature detection head is the Focal-loss loss function;

[0012] The loss function of the relative starting point detection head is a Push-PullLloss loss function.

[0013] Furthermore, the use of a pre-built lane line detection model to perform lane line recognition on the road scene image to obtain the lane line foreground point coordinates and relative starting point coordinates includes:

[0014] Extracting features from the road scene image through the backbone network to obtain target features;

[0015] Performing feature fusion on the target features through the FPN network to obtain fused features;

[0016] The lane line feature detection head is used to detect the fused feature to obtain the lane line foreground point coordinates with the lane line feature; and the relative starting point detection head is used to detect the fused feature to obtain the relative starting point coordinates.

[0017] Further, the detecting the fusion feature by the relative starting point detection head to obtain the relative starting point coordinates includes:

[0018] Perform lane line modeling based on the annotation information to obtain a true value map of the foreground point;

[0019] Perform mathematical calculation based on the foreground point truth map to obtain a starting point truth map; the starting point truth map is used in the training process of the relative starting point detection head;

[0020] The relative starting point detection head performs relative starting point detection on the fusion feature to obtain relative starting point coordinates; the relative starting point coordinates include relative starting point H coordinates and relative starting point W coordinates corresponding to the lane line foreground point.

[0021] Furthermore, the lane line foreground point coordinates and the relative starting point coordinates are processed by the lane line instance clustering algorithm decoder to obtain a lane line coordinate value containing a lane line instance, including:

[0022] Sort the lane line foreground scenic spot coordinates in descending order according to the lane line foreground scenic spot H coordinates to obtain a lane line foreground scenic spot coordinate sequence;

[0023] Classifying the relative starting point corresponding to the lane line foreground scenic spot coordinates that are sorted first in the lane line foreground scenic spot coordinate sequence into the relative starting point subset;

[0024] Determine whether a first distance between the relative starting point to be classified and the mean starting point of the relative starting point subset is less than a first preset threshold; wherein the mean starting point is determined by the relative starting point H coordinate and the relative starting point W coordinate;

[0025] If yes, classify the relative starting point to be classified into the relative starting point subset corresponding to the smallest first distance, and update the corresponding mean starting point;

[0026] If not, determining whether a second distance between the relative starting point to be classified and the edge starting point of the relative starting point subset is less than a second preset threshold;

[0027] If yes, classify the relative starting point to be classified into the relative starting point subset corresponding to the smallest second distance, and update the corresponding mean starting point;

[0028] If not, classifying the relative starting point to be classified into a new relative starting point subset;

[0029] After all relative starting points are classified, the lane line coordinate values ​​including the lane line instance are determined according to the correspondence between each relative starting point in the relative starting point subset and the lane line foreground point coordinates.

[0030] A second aspect of the present application provides a lane line recognition device, the lane line recognition device comprising:

[0031] A collection unit, used for collecting road scene images including lane lines;

[0032] An identification unit, used to perform lane line identification on the road scene image using a pre-built lane line detection model, and obtain lane line foreground point coordinates and relative starting point coordinates; the relative starting point coordinates include relative starting point H coordinates and relative starting point W coordinates;

[0033] An acquisition unit, used for acquiring a pre-built lane instance clustering algorithm decoder based on a dynamically adjusted starting point;

[0034] A processing unit is used to process the lane line foreground point coordinates and the relative starting point coordinates through the lane line instance clustering algorithm decoder to obtain a lane line coordinate value containing a lane line instance.

[0035] Furthermore, the lane line detection model includes a backbone network, an FPN network and a detection head; the detection head includes a lane line feature detection head and a relative starting point detection head, the loss function of the lane line feature detection head is a Focal-loss loss function, and the loss function of the relative starting point detection head is a Push-PullLloss loss function.

[0036] Furthermore, the identification unit includes:

[0037] An extraction subunit, used to extract features from the road scene image through the backbone network to obtain target features;

[0038] A fusion subunit, used for fusing the target features through the FPN network to obtain fused features;

[0039] The detection subunit is used to detect the fused feature through the lane line feature detection head to obtain the lane line foreground point coordinates with the lane line feature; and detect the fused feature through the relative starting point detection head to obtain the relative starting point coordinates.

[0040] Furthermore, the detection subunit is specifically used to perform lane line modeling based on the annotation information to obtain a true value map of the foreground point;

[0041] The detection subunit is specifically used to perform mathematical calculation based on the foreground point truth map to obtain a starting point truth map; the starting point truth map is used in the training process of the relative starting point detection head;

[0042] The detection subunit is specifically used to perform relative starting point detection on the fusion feature through the relative starting point detection head to obtain relative starting point coordinates; the relative starting point coordinates include relative starting point H coordinates and relative starting point W coordinates corresponding to the foreground point of the lane line.

[0043] Furthermore, the processing unit includes:

[0044] A sorting subunit, used to sort the lane line foreground scenic spot coordinates in descending order of the lane line foreground scenic spot H coordinates to obtain a lane line foreground scenic spot coordinate sequence;

[0045] A classification subunit, configured to classify the relative starting point corresponding to the lane line foreground scenic spot coordinates that are sorted first in the lane line foreground scenic spot coordinate sequence into a relative starting point subset;

[0046] A first judgment subunit is used to judge whether a first distance between the relative starting point to be classified and the mean starting point of the relative starting point subset is less than a first preset threshold; wherein the mean starting point is determined by the relative starting point H coordinate and the relative starting point W coordinate;

[0047] The classification subunit is further configured to classify the relative starting point to be classified into a relative starting point subset corresponding to the smallest first distance when the first distance is less than the first preset threshold, and update the corresponding mean starting point;

[0048] A second judging subunit is configured to judge whether a second distance between the relative starting point to be classified and the edge starting point of the relative starting point subset is less than a second preset threshold value when the first distance is not less than the first preset threshold value;

[0049] The classification subunit is further configured to classify the relative starting point to be classified into a relative starting point subset corresponding to the smallest second distance when the second distance is less than the second preset threshold, and update the corresponding mean starting point;

[0050] The classification subunit is further configured to classify the relative starting point to be classified into a new relative starting point subset when the second distance is not less than the second preset threshold;

[0051] The determination subunit is used to determine the lane line coordinate value containing the lane line instance according to the correspondence between each relative starting point in the relative starting point subset and the lane line foreground point coordinate after completing the classification of all relative starting points.

[0052] A third aspect of the present application provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform any lane line recognition method described in any one of the first aspects of the present application.

[0053] A fourth aspect of the present application provides a computer-readable storage medium storing computer program instructions. When the computer program instructions are read and executed by a processor, the lane line recognition method described in any one of the first aspects of the present application is executed.

[0054] The beneficial effects of the present application are as follows: the method and device can accurately divide the lane line foreground points based on the relative starting points, so that the accurately divided lane line foreground points form a more accurate lane line recognition result. At the same time, the method and device can also cluster all relative starting points based on the accurate relative starting points corresponding to the nearby foreground points, so as to improve the clustering stability of the relative starting points, thereby achieving a more stable lane line recognition effect.

[0055] In addition, in scenarios where lane line recognition is difficult, such as double lines or dense lane lines, the method and device can show a larger starting point spacing relative to the starting point, thereby making the division of the lane line starting points clearer, thereby overcoming the problem of close lane line starting points and effectively solving the lane line detection problem in difficult scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 A schematic diagram of a process flow of a lane line recognition method provided in an embodiment of the present application;

[0058] Figure 2 A schematic diagram of a flow chart of another lane line recognition method provided in an embodiment of the present application;

[0059] Figure 3 A schematic diagram of an example flow of a lane line recognition method provided in an embodiment of the present application;

[0060] Figure 4 A schematic diagram of the structure of a lane line recognition device provided in an embodiment of the present application;

[0061] Figure 5 A schematic diagram of the structure of another lane recognition device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0063] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0064] Example 1

[0065] Please see Figure 1 , Figure 1 A schematic flow chart of a lane line recognition method provided in this embodiment. The lane line recognition method includes:

[0066] S101: Collect road scene images including lane lines.

[0067] In this embodiment, the road scene picture includes annotation information.

[0068] S102: Use a pre-built lane line detection model to perform lane line recognition on the road scene image to obtain the coordinates of the lane line foreground point and the relative starting point coordinates.

[0069] In this embodiment, the relative starting point coordinates include the relative starting point H coordinates and the relative starting point W coordinates.

[0070] In this embodiment, the lane line detection model includes a backbone network, an FPN network and a detection head;

[0071] The detection head includes a lane feature detection head and a relative starting point detection head;

[0072] The loss function of the lane feature detection head is the Focal-loss loss function;

[0073] The loss function of the detection head relative to the starting point is the Push-PullLloss loss function.

[0074] S103: Obtain a pre-built lane instance clustering algorithm decoder based on dynamically adjusted starting points.

[0075] S104. Processing the lane line foreground point coordinates and the relative starting point coordinates through a lane line instance clustering algorithm decoder to obtain a lane line coordinate value including a lane line instance.

[0076] In this embodiment, the lane line instance can be understood as a unique code of the lane line. For example, assuming there are three lane lines, the first lane line can correspond to No. 1, the second lane line can correspond to No. 2, and the third lane line can correspond to No. 3;

[0077] In this embodiment, the execution subject of the method may be a computing device such as a computer or a server, and no limitation is made in this embodiment.

[0078] It can be seen that the lane line recognition method described in this embodiment can identify the lane line foreground point and the relative starting point corresponding to the lane previous point based on the collected road scene image, and then complete the clustering of all relative starting points based on the positional relationship between the relative starting points, and map the clustering results back to the lane line foreground point, so that the lane line foreground point can complete more accurate instance division based on the clustering results of the relative starting points, thereby realizing effective recognition of the lane line, and making it more suitable for lane line recognition scenarios with complex topological structures.

[0079] Example 2

[0080] Please see Figure 2 , Figure 2 A schematic flow chart of a lane line recognition method provided in this embodiment. The lane line recognition method includes:

[0081] S201: Collect road scene images including lane lines.

[0082] In this embodiment, the road scene picture includes annotation information.

[0083] In this embodiment, the method can annotate lane line pixels on the collected road scene images, thereby obtaining annotation information.

[0084] S202: extracting features of the road scene image through the backbone network to obtain target features.

[0085] In this embodiment, the method can use MobileNet as the backbone network of the network structure.

[0086] S203, performing feature fusion on the target features through the FPN network to obtain fused features.

[0087] In this embodiment, the method may use an FPN network for feature fusion.

[0088] S204: Detect the fused features using a lane line feature detection head to obtain the coordinates of the lane line foreground point with lane line features.

[0089] In this embodiment, the lane line feature detection head is used to predict whether a pixel point in a road scene image belongs to a foreground point with a lane line feature or a background point without a lane line feature, and obtain its coordinate value.

[0090] In this embodiment, the prediction of lane line foreground points is supervised using Focal loss. In order to highlight the foreground features of the lane lines, a specific area where lane lines are concentrated in the image can be selected as the network input.

[0091] S205: Lane line modeling is performed based on the annotation information to obtain a true value map of the foreground point.

[0092] In this embodiment, the process of performing lane line modeling in this method may be:

[0093] The lane line set is represented as L = {l1, l2, ..., l N}; where N is the number of lane lines in a single image;

[0094] Each lane line is represented as: i =[(x i1 ,y i1 ), (x i2 ,y i2 ),…,(x ij ,y ij ),…,(x iM ,y iM )]; where M is the number of points in a single lane line, i is the lane line index (or lane line instance), i∈[1,N], j is the jth lane line foreground point in a single lane line, j∈[1,M]; l i represents the i-th lane line, (x ij ,y ij ) The coordinates of the j-th lane line foreground point of the ith lane line.

[0095] S206, performing mathematical calculation based on the foreground point truth map to obtain a starting point truth map; the starting point truth map is used in the training process of the relative starting point detection head.

[0096] In this embodiment, the method can construct two starting point truth maps according to the lane line foreground point truth map. Specifically, one starting point truth map is to fill in the H coordinate of the starting point of the lane line to which the foreground point belongs at the corresponding position of the lane line foreground point, and the other starting point truth map is to fill in the W coordinate of the starting point of the lane line to which the foreground point belongs at the corresponding position of the lane line foreground point.

[0097] In this embodiment, the method uses PullLoss supervision to make the variance value of the starting points corresponding to the foreground points belonging to the same lane line as small as possible; at the same time, PushLoss supervision is used to make the distance between the starting points corresponding to the foreground points not belonging to the same lane line as large as possible.

[0098] S207, performing relative starting point detection on the fusion feature through a relative starting point detection head to obtain relative starting point coordinates.

[0099] In this embodiment, the relative starting point coordinates include the relative starting point H coordinates and the relative starting point W coordinates corresponding to the lane line foreground point.

[0100] In this embodiment, there are more than one lane line prospect coordinates and relative starting point coordinates, wherein multiple relative starting point coordinates will also constitute a relative starting point feature map.

[0101] For example, there are 3 independent lane lines, each lane line has 10 foreground points, and each foreground point has a corresponding relative starting point. At this time, there are a total of 30 foreground points and 30 relative starting points. Since each independent lane line corresponds to 10 relative starting points, the corresponding 10 relative starting points can constitute a starting point feature map corresponding to the lane line, which is used to represent the starting point feature of the lane line.

[0102] In practice, the number of independent lane lines is usually unknown, but the coordinates of all lane line foreground points are known. Therefore, the purpose of this method is to determine all relative starting points through the lane line foreground points, and determine multiple lane line foreground points belonging to the same independent lane line based on the relationship between the relative starting points, so as to accurately identify the corresponding independent lane line based on the multiple lane line foreground points, and at the same time know the number of independent lane lines. Based on the above example, the method can know that the number of lane lines is 3 at this time, and the features of each lane line are clearly visible.

[0103] S208. Obtain a pre-built lane instance clustering algorithm decoder based on dynamically adjusted starting points.

[0104] S209 , sorting the coordinates of the scenic spots in front of the lane line in descending order of the H coordinates of the scenic spots in front of the lane line to obtain a coordinate sequence of the scenic spots in front of the lane line.

[0105] In this embodiment, since the relative starting point predicted based on the nearby lane line foreground point is more stable, this method chooses to sort the lane line foreground points from large to small according to the H coordinate, so as to determine the nearby lane line foreground point and the corresponding relative starting point.

[0106] Assume that there are 3 independent lane lines in the road scene image, and each independent lane line has 10 lane line foreground points. Take 1 independent lane line as an example. The 10 lane line foreground points in the independent lane line are arranged from near to far. Therefore, the lane line foreground point closest to the vehicle can be easily determined by the size of the H coordinate. Due to the factors related to the near and far lights, the lane line foreground point closest to the vehicle can usually predict a relatively accurate relative starting point. Therefore, this method can determine the first accurate relative starting point at this time, thereby ensuring the accuracy of subsequent dynamic clustering.

[0107] S210: Classify the relative starting point corresponding to the lane line foreground scenic spot coordinates that are ranked first in the lane line foreground scenic spot coordinate sequence into a relative starting point subset.

[0108] In this embodiment, the coordinates of the lane line foreground point that is ranked first are the coordinates of the lane line foreground point that is closest to the vehicle.

[0109] In this embodiment, one lane line foreground point is usually obtained. However, in some cases, multiple lane line foreground points may be obtained, but the multiple lane line foreground points are required to belong to the same lane line.

[0110] Among them, obtaining a lane line foreground point is the most stable and easiest to implement. To obtain multiple lane line foreground points, it is necessary to determine in advance whether the multiple lane line foreground points belong to the same lane line (when obtaining a lane line foreground point, it must belong to the same lane line). Therefore, compared with obtaining multiple lane line foreground points, it has the above-mentioned technical effect.

[0111] The reason why the case where there can be multiple lane line foreground points is proposed is that the multiple relative starting points determined by the multiple lane line foreground points in the same lane line can be corrected with each other, thereby adding a prerequisite for the implementation of the method to eliminate the error that exists at a single relative starting point with a very small probability.

[0112] It should be understood that the identification of multiple lane line foreground points of the same lane line is, to some extent, a result of multiple implementations of step S212. That is, the method includes flexible adjustments between steps to achieve corresponding technical effects.

[0113] S211, determining whether a first distance between the relative starting point to be classified and the mean starting point of the relative starting point subset is less than a first preset threshold, if so, executing step S212; if not, executing step S213.

[0114] In this embodiment, the mean starting point is determined by the relative starting point H coordinate and the relative starting point W coordinate.

[0115] In this embodiment, when step S211 is performed for the first time, the number of relative starting point subsets is 1. When step S211 is performed multiple times, the number of relative starting point subsets may be greater than or equal to 1.

[0116] In this embodiment, when the number of relative starting point subsets is greater than 1, step S211, when making a judgment, calculates the first distance for different relative starting point subsets respectively, and obtains the results of whether multiple first distances are less than the first preset threshold (the multiple results need to be selected by comparing multiple first distances in subsequent steps).

[0117] For example, when the relative starting point subset includes two relative starting point coordinates, the two relative starting point coordinates are (W1, H1) and (W2, H2). At this time, the mean starting point is ((W1+W2) / 2, (H1+H2) / 2). The mean starting point is usually virtual, but in some cases it can also be real (for example, there happens to be a relative starting point at this coordinate).

[0118] S212, classify the relative starting point to be classified into the relative starting point subset corresponding to the smallest first distance, and update the corresponding mean starting point, and then execute step S216.

[0119] In this embodiment, if there is only one first distance, the smallest one is itself; if there are multiple first distances, the smallest one is the smallest one among them.

[0120] In this embodiment, the updating method of the mean starting point is similar to the calculation method, which will not be described in detail here.

[0121] S213, determining whether the second distance between the relative starting point to be classified and the edge starting point of the relative starting point subset is less than a second preset threshold, if so, executing step S214; if not, executing step S215.

[0122] In this embodiment, since some relative starting points should belong to a relative starting point subset, but their first distance is greater than the first preset threshold, these relative starting points are not effectively classified. Therefore, step S213 proposes a further relative starting point classification judgment step to effectively classify the above relative starting points, thereby ensuring the classification effect of the relative starting points.

[0123] In this embodiment, the edge starting point is a relatively edge relative starting point in the relative starting point subset, and the number of the relative starting point may be one or more.

[0124] In this embodiment, the first preset threshold and the second preset threshold may be the same or different.

[0125] S214, classify the relative starting point to be classified into the relative starting point subset corresponding to the smallest second distance, update the corresponding mean starting point, and then execute step S216.

[0126] S215, classify the relative starting points to be classified into a new relative starting point subset, and then execute step S216.

[0127] In this embodiment, when the method still determines that the relative starting point should not be classified into the existing relative starting point subset after double judgment, a new relative starting point subset is created.

[0128] In this embodiment, a relative starting point subset corresponds to a lane line.

[0129] S216: Determine whether the classification of all relative starting points has been completed. If so, execute step S217; if not, execute step S211.

[0130] S217: Determine the lane line coordinate value including the lane line instance according to the correspondence between the coordinates of each relative starting point in the relative starting point subset and the lane line foreground point.

[0131] In this embodiment, the lane line instance may be a unique lane line code, such as a number.

[0132] Please see Figure 3 , Figure 3 An example flow chart of a lane line recognition method is shown, wherein the example flow chart corresponds to steps S201 to S217.

[0133] In this embodiment, the execution subject of the method may be a computing device such as a computer or a server, and no limitation is made in this embodiment.

[0134] It can be seen that the lane line recognition method described in this embodiment can identify the lane line foreground point and the relative starting point corresponding to the lane previous point based on the collected road scene image, and then complete the clustering of all relative starting points based on the positional relationship between the relative starting points, and map the clustering results back to the lane line foreground point, so that the lane line foreground point can complete more accurate instance division based on the clustering results of the relative starting points, thereby realizing effective recognition of the lane line, and making it more suitable for lane line recognition scenarios with complex topological structures.

[0135] Example 3

[0136] Please see Figure 4 , Figure 4 FIG. 1 is a schematic diagram of the structure of a lane line recognition device provided in this embodiment. Figure 4 As shown, the lane line recognition device includes:

[0137] A collection unit 310 is used to collect road scene images including lane lines;

[0138] The recognition unit 320 is used to perform lane line recognition on the road scene image using a pre-built lane line detection model to obtain the lane line foreground point coordinates and the relative starting point coordinates; the relative starting point coordinates include the relative starting point H coordinates and the relative starting point W coordinates;

[0139] An acquisition unit 330, configured to acquire a pre-built lane instance clustering algorithm decoder based on a dynamically adjusted starting point;

[0140] The processing unit 340 is used to process the lane line foreground point coordinates and the relative starting point coordinates through a lane line instance clustering algorithm decoder to obtain a lane line coordinate value containing a lane line instance.

[0141] In this embodiment, the explanation of the lane line recognition device can refer to the description in Example 1 or Example 2, and will not be repeated in this embodiment.

[0142] It can be seen that the lane line recognition device described in the present embodiment can identify the lane line foreground point and the relative starting point corresponding to the lane previous point based on the collected road scene picture, and then complete the clustering of all relative starting points based on the positional relationship between the relative starting points, and map the clustering results back to the lane line foreground point, so that the lane line foreground point can complete more accurate instance division based on the clustering results of the relative starting points, thereby realizing effective recognition of the lane line, and making it more suitable for lane line recognition scenarios with complex topological structures.

[0143] Example 4

[0144] Please see Figure 5 , Figure 5 FIG. 1 is a schematic diagram of the structure of a lane line recognition device provided in this embodiment. Figure 5 As shown, the lane line recognition device includes:

[0145] A collection unit 310 is used to collect road scene images including lane lines;

[0146] The recognition unit 320 is used to perform lane line recognition on the road scene image using a pre-built lane line detection model to obtain the lane line foreground point coordinates and the relative starting point coordinates; the relative starting point coordinates include the relative starting point H coordinates and the relative starting point W coordinates;

[0147] An acquisition unit 330, configured to acquire a pre-built lane instance clustering algorithm decoder based on a dynamically adjusted starting point;

[0148] The processing unit 340 is used to process the lane line foreground point coordinates and the relative starting point coordinates through a lane line instance clustering algorithm decoder to obtain a lane line coordinate value containing a lane line instance.

[0149] In this embodiment, the lane line detection model includes a backbone network, an FPN network and a detection head; the detection head includes a lane line feature detection head and a relative starting point detection head. The loss function of the lane line feature detection head is a Focal-loss loss function, and the loss function of the relative starting point detection head is a Push-PullLloss loss function.

[0150] As an optional implementation, the identification unit 320 includes:

[0151] The extraction subunit 321 is used to extract features from the road scene image through the backbone network to obtain target features;

[0152] The fusion subunit 322 is used to fuse the target features through the FPN network to obtain fused features;

[0153] The detection subunit 323 is used to detect the fused features through a lane line feature detection head to obtain the coordinates of the lane line foreground point with the lane line features; and to detect the fused features through a relative starting point detection head to obtain the relative starting point coordinates.

[0154] As an optional implementation, the detection subunit 323 is specifically used to perform lane line modeling based on the annotation information to obtain a true value map of the foreground point;

[0155] The detection subunit 323 is specifically used to perform mathematical calculation based on the foreground point truth map to obtain the starting point truth map; the starting point truth map is used in the training process of the relative starting point detection head;

[0156] The detection subunit 323 is specifically used to perform relative starting point detection on the fusion feature through a relative starting point detection head to obtain relative starting point coordinates; the relative starting point coordinates include a relative starting point H coordinate and a relative starting point W coordinate corresponding to the foreground point of the lane line.

[0157] As an optional implementation, the processing unit 340 includes:

[0158] The sorting subunit 341 is used to sort the lane line foreground scenic spot coordinates in descending order of the lane line foreground scenic spot H coordinates to obtain a lane line foreground scenic spot coordinate sequence;

[0159] The classification subunit 342 is used to classify the relative starting point corresponding to the lane line foreground scenic spot coordinates that are sorted first in the lane line foreground scenic spot coordinate sequence into the relative starting point subset;

[0160] A first judging subunit 343 is used to judge whether a first distance between the relative starting point to be classified and the mean starting point of the relative starting point subset is less than a first preset threshold; wherein the mean starting point is determined by the relative starting point H coordinate and the relative starting point W coordinate;

[0161] The classification subunit 342 is further configured to classify the relative starting point to be classified into a relative starting point subset corresponding to the smallest first distance when the first distance is less than a first preset threshold, and update the corresponding mean starting point;

[0162] A second judging subunit 344 is used to judge whether a second distance between the relative starting point to be classified and the edge starting point of the relative starting point subset is less than a second preset threshold value when the first distance is not less than the first preset threshold value;

[0163] The classification subunit 342 is further configured to classify the relative starting point to be classified into a relative starting point subset corresponding to the smallest second distance when the second distance is less than a second preset threshold, and update the corresponding mean starting point;

[0164] The classification subunit 342 is further configured to classify the relative starting point to be classified into a new relative starting point subset when the second distance is not less than a second preset threshold;

[0165] The determination subunit 345 is used to determine the lane line coordinate value containing the lane line instance according to the correspondence between each relative starting point and the lane line foreground point coordinate in the relative starting point subset after completing the classification of all relative starting points.

[0166] In this embodiment, the explanation of the lane line recognition device can refer to the description in Example 1 or Example 2, and will not be repeated in this embodiment.

[0167] It can be seen that the lane line recognition device described in the present embodiment can identify the lane line foreground point and the relative starting point corresponding to the lane previous point based on the collected road scene picture, and then complete the clustering of all relative starting points based on the positional relationship between the relative starting points, and map the clustering results back to the lane line foreground point, so that the lane line foreground point can complete more accurate instance division based on the clustering results of the relative starting points, thereby realizing effective recognition of the lane line, and making it more suitable for lane line recognition scenarios with complex topological structures.

[0168] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the lane line recognition method in embodiment 1 or embodiment 2 of the present application.

[0169] An embodiment of the present application provides a computer-readable storage medium storing computer program instructions. When the computer program instructions are read and executed by a processor, the lane line recognition method in Embodiment 1 or Embodiment 2 of the present application is executed.

[0170] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0171] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0172] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0173] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0174] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0175] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A lane line recognition method, characterized in that: include: Collecting a road scene picture including lane lines; the road scene picture includes annotation information; Using a pre-built lane line detection model to perform lane line recognition on the road scene image, and obtaining the lane line foreground point coordinates and relative starting point coordinates; the relative starting point coordinates include the relative starting point H coordinates and the relative starting point W coordinates; Get the pre-built decoder for the lane instance clustering algorithm based on dynamically adjusted starting points; The lane line foreground point coordinates and the relative starting point coordinates are processed by the lane line instance clustering algorithm decoder to obtain a lane line coordinate value containing a lane line instance.

2. The lane line recognition method according to claim 1, characterized in that: The lane line detection model includes a backbone network, an FPN network and a detection head; The detection head includes a lane feature detection head and a relative starting point detection head; The loss function of the lane feature detection head is the Focal-loss loss function; The loss function of the relative starting point detection head is a Push-PullLloss loss function.

3. The lane line recognition method according to claim 2, characterized in that: The lane line detection model pre-built is used to perform lane line recognition on the road scene image to obtain the foreground point coordinates of the lane line and the relative starting point coordinates, including: Extracting features from the road scene image through the backbone network to obtain target features; Performing feature fusion on the target features through the FPN network to obtain fused features; The lane line feature detection head is used to detect the fused feature to obtain the lane line foreground point coordinates with the lane line feature; and the relative starting point detection head is used to detect the fused feature to obtain the relative starting point coordinates.

4. The lane line recognition method according to claim 3, characterized in that: The detecting the fusion feature by the relative starting point detection head to obtain the relative starting point coordinates includes: Perform lane line modeling based on the annotation information to obtain a true value map of the foreground point; Perform mathematical calculation based on the foreground point truth map to obtain a starting point truth map; the starting point truth map is used in the training process of the relative starting point detection head; The relative starting point detection head performs relative starting point detection on the fusion feature to obtain relative starting point coordinates; the relative starting point coordinates include relative starting point H coordinates and relative starting point W coordinates corresponding to the lane line foreground point.

5. The lane line recognition method according to claim 1, characterized in that: The lane line foreground point coordinates and the relative starting point coordinates are processed by the lane line instance clustering algorithm decoder to obtain a lane line coordinate value containing a lane line instance, including: Sort the lane line foreground scenic spot coordinates in descending order according to the lane line foreground scenic spot H coordinates to obtain a lane line foreground scenic spot coordinate sequence; Classifying the relative starting point corresponding to the lane line foreground scenic spot coordinates that are sorted first in the lane line foreground scenic spot coordinate sequence into the relative starting point subset; Determine whether a first distance between the relative starting point to be classified and the mean starting point of the relative starting point subset is less than a first preset threshold; wherein the mean starting point is determined by the relative starting point H coordinate and the relative starting point W coordinate; If yes, classify the relative starting point to be classified into the relative starting point subset corresponding to the smallest first distance, and update the corresponding mean starting point; If not, determining whether a second distance between the relative starting point to be classified and the edge starting point of the relative starting point subset is less than a second preset threshold; If yes, classify the relative starting point to be classified into the relative starting point subset corresponding to the smallest second distance, and update the corresponding mean starting point; If not, classifying the relative starting point to be classified into a new relative starting point subset; After all relative starting points are classified, the lane line coordinate values ​​including the lane line instance are determined according to the correspondence between each relative starting point in the relative starting point subset and the lane line foreground point coordinates.

6. A lane line recognition device, characterized in that: The lane line recognition device comprises: A collection unit, used for collecting road scene images including lane lines; An identification unit, used to perform lane line identification on the road scene image using a pre-built lane line detection model, and obtain lane line foreground point coordinates and relative starting point coordinates; the relative starting point coordinates include relative starting point H coordinates and relative starting point W coordinates; An acquisition unit, used for acquiring a pre-built lane instance clustering algorithm decoder based on a dynamically adjusted starting point; A processing unit is used to process the lane line foreground point coordinates and the relative starting point coordinates through the lane line instance clustering algorithm decoder to obtain a lane line coordinate value containing a lane line instance.

7. The lane line recognition device according to claim 6, characterized in that: The lane line detection model includes a backbone network, an FPN network and a detection head; the detection head includes a lane line feature detection head and a relative starting point detection head, the loss function of the lane line feature detection head is a Focal-loss loss function, and the loss function of the relative starting point detection head is a Push-PullLloss loss function.

8. The lane line recognition device according to claim 7, characterized in that: The identification unit comprises: An extraction subunit, used to extract features from the road scene image through the backbone network to obtain target features; A fusion subunit, used for fusing the target features through the FPN network to obtain fused features; The detection subunit is used to detect the fused feature through the lane line feature detection head to obtain the lane line foreground point coordinates with the lane line feature; and detect the fused feature through the relative starting point detection head to obtain the relative starting point coordinates.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the lane line recognition method according to any one of claims 1 to 5.

10. A readable storage medium, characterized in that: The readable storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the lane line recognition method according to any one of claims 1 to 5 is executed.