A lane line detection method and apparatus

Abstract

The application discloses a lane line detection method and device, the method comprises the following steps: processing the first feature map based on lane line proposal initialization network to generate a preliminary lane line detection result; based on a lane line position refinement network and a second feature map, more detailed sampling point features are obtained, which are converted into control point features, and the control point feature correlation form is consistent with the curve nature, and a more accurate lane line detection result is generated; based on a spline curve with a group of control points, the position of the lane line is represented; based on a non-maximum suppression algorithm, the lane line is de-duplicated; based on inverse perspective transformation, the lane line is converted to a ground coordinate system; based on the closed area and the curve length, the curve regression loss function is calculated. The application can enhance the fitting ability of the lane line detection scheme, and improve the environmental adaptability under the condition of low calculation complexity.

Description

Technical Field

[0001] This invention relates to the field of lane line detection technology, specifically to a lane line detection method and apparatus. Background Technology

[0002] As autonomous driving technology and vehicle assistance systems gain increasing favor among automotive companies, a series of technological researches with autonomous driving as their background are receiving more and more attention from academia and industry. The key and difficult point of autonomous driving technology research lies in the analysis of the vehicle's surrounding environment, and computer vision can be used to solve this key and difficult point, including lane detection. Lane detection refers to the use of digital image processing and pattern recognition technology to effectively extract lane information from road images and fit lane lines, which is used to assist functions such as autonomous driving, automatic parking, and collision avoidance warning.

[0003] Lane detection technology, an indispensable part of the environmental perception technology system, is a crucial component for achieving autonomous and assisted driving. This technology accurately assists vehicles in positioning, acquiring their precise location information on the road in real time, ensuring that autonomous vehicles strictly adhere to traffic regulations during operation. Existing parametric curve-based lane detection schemes can directly output lane instance data and have fast inference speeds, but their detection accuracy is often relatively low, and their ability to fit lanes with complex shapes is weak. These models tend to output straight-line lanes, which constitute a large proportion of the dataset, during training, resulting in poor environmental adaptability and significantly hindering the application of this approach. Summary of the Invention

[0004] To address the aforementioned shortcomings, this invention proposes a lane line detection method and apparatus.

[0005] This invention provides a lane line detection method, comprising the following steps:

[0006] S1. Obtain the first image;

[0007] S2. Scale the first image according to the preset image size to generate the second image;

[0008] S3. Based on the backbone network, feature extraction is performed on the second image to generate a first feature map and a second feature map;

[0009] S4. The first feature map is processed by the lane line proposal initialization network to generate the first lane line control point location set and the corresponding first lane line confidence.

[0010] S5. Based on the definition of spline curve, process the first lane line control point location set to obtain the first lane line sampling point location set;

[0011] S6. Based on the lane line position refinement network, the second feature map and the first lane line sampling point position set are processed to obtain the second lane line control point position set and the corresponding second lane line confidence.

[0012] S7. Based on the spline curve definition, process the set of control point locations of the second lane line to obtain the set of sampling point locations of the second lane line.

[0013] S8. Based on the non-maximum suppression algorithm, process the second lane line sampling point location set and the second lane line confidence to generate the third lane line sampling point location set;

[0014] S9. The third lane line sampling point location set is processed based on inverse perspective transformation to generate the fourth lane line sampling point location set; the fourth lane line sampling point location set is defined in the top view perspective.

[0015] S10. Calculate the overall loss function of the network, implement the backpropagation algorithm, and update the network parameters.

[0016] Preferably, step S3 specifically includes the following steps:

[0017] S301. Input the second image into the backbone network to extract features and obtain three levels of output feature maps of different sizes, namely, level one, level two and level three feature maps;

[0018] S302. Use the first-level feature map as the first feature map;

[0019] S303. Use the third-level feature map as the second feature map.

[0020] Preferably, step S4 specifically includes the following steps:

[0021] S401. The first feature map is processed by a global feature extraction network for lane lines to generate global features for lane lines.

[0022] S402. Input the global features of the lane line into the first prediction head composed of a fully connected network to generate the first lane line control point location set of the lane line and the corresponding first lane line confidence.

[0023] Preferably, step S5 specifically includes the following steps:

[0024] S501. Use the set of control points of the first lane line as the control points of the spline curve to generate the first curve set;

[0025] S502, The position of the sampling points of the first curve set on the curve is controlled by an equidistant parameter u set;

[0026] S503. Calculate the corresponding basis function values ​​based on the set of parameters u;

[0027] S504. Combine the basis function values ​​to generate the first transformation matrix M;

[0028] S505. Based on the first set of lane control point locations and the first transformation matrix M, generate a first set of lane sampling point locations.

[0029] Preferably, step S6 specifically includes the following steps:

[0030] S601. Perform a matrix transformation on the first transformation matrix M to generate a second transformation matrix W; the matrix transformation is achieved by the following formula: W = (M T M) -1 M T ;

[0031] S602. Based on the second feature map and the first lane line sampling point location set, perform bilinear interpolation on the features at the corresponding pixels of the first sampling point location set in the second feature map to obtain the feature set F of the first sampling points. s ;

[0032] S603, Based on the feature set F of the first sampling point s The second transformation matrix W is used to generate the control point feature set F. C The specific implementation method is F C =WF s ;

[0033] S604, Set the control point feature set F C The second prediction head, consisting of a fully connected network, is input to generate the set of second control point locations for the lane lines, as well as the corresponding second lane line confidence scores.

[0034] Preferably, step S8 specifically includes the following steps:

[0035] S801. Calculate the distance set between the sampling point sets of the second lane line based on the chamfer distance, calculate the first curve distance D; and transform it to obtain the standardized second curve distance D. norm :

[0036]

[0037] Where the parameter r is a constant;

[0038] S802, Based on the nonmaximum suppression algorithm and the second curve distance D normThe second set of lane line sampling point locations is filtered to obtain the third set of lane line sampling point locations.

[0039] Preferably, step S9 specifically includes the following steps:

[0040] S901. Based on the intrinsic and extrinsic parameters of the image acquisition device, generate the inverse perspective transformation matrix H;

[0041] S902. Generate the fourth lane line sampling point location set based on the inverse perspective transformation matrix H and the third lane line sampling point location set.

[0042] Preferably, step S10 specifically includes the following steps:

[0043] S1001. Based on the second set of lane line sampling point locations and the manually labeled lane line true values, calculate the area S of the closed region between curves and calculate the length L of the lane line true values; the area S of the closed region between curves is decomposed into the area of ​​triangles for approximate calculation, and the length L of the lane line true values ​​is decomposed into the length of line segments for approximate calculation.

[0044] S1002. Calculate the area loss function based on the area S of the closed region between the curves and the length L of the true value of the lane line.

[0045] S1003. Based on the second set of lane line sampling point locations and the manually labeled lane line ground truth values, calculate the endpoint loss function L. point The endpoint loss function is calculated using the L2 loss function.

[0046] S1004. Based on the classification loss function, area loss function, and endpoint loss function, calculate the overall loss function of the network and use the calculation results for backpropagation and parameter update.

[0047] S1005. Calculate the matching loss using the same method as the loss function of the overall computational network, and assign labels to the true values ​​of lane lines and the set of second sampling points.

[0048] This invention provides a lane line detection device, comprising:

[0049] Image acquisition device, used to acquire the first image;

[0050] The backbone network processing module is used to extract features from the second image based on the backbone network to generate a first feature map and a second feature map.

[0051] The lane line proposal initialization module is used to process the first feature map based on the lane line proposal initialization network to generate a first lane line control point location set and a corresponding first lane line confidence score.

[0052] The lane line position refinement module is used to process the second feature map and the first lane line sampling point position set based on the lane line position refinement network to obtain the second lane line control point position set and the corresponding second lane line confidence.

[0053] The non-maximum suppression module is used to process the second lane line sampling point location set and the second lane line confidence based on the non-maximum suppression algorithm to generate the third lane line sampling point location set.

[0054] This invention provides an electronic device, comprising:

[0055] One or more processors;

[0056] Memory, used to store one or more programs;

[0057] When the one or more programs are executed by the one or more processors, the one or more processors implement the lane line detection method provided by the present invention.

[0058] The present invention provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the steps of the lane line detection method provided by the present invention.

[0059] Compared with the prior art, the beneficial effects of the present invention are:

[0060] This invention improves the flexibility of lane line representation, allowing lane lines to be represented by spline curves with locality for complex scenes. A lane line proposal initialization module is designed to make the initial position of lane line proposals more targeted to each image, enabling the use of fewer lane line proposals and reducing computational complexity. A lane line position refinement module is designed to collect features of sampling points on a high-resolution second feature map, converting these features into features of control points, further improving the local fit of lane lines. Simultaneously, for spline curves with locality, the features of neighboring control points can be correlated, while the features of distant control points are decoupled, thereby improving the model's fitting ability and fully utilizing the degrees of freedom of the parametric curves themselves to achieve fitting of more complex and diverse lane lines. Attached Figure Description

[0061] Figure 1 This is a schematic diagram of a lane line detection method provided in Embodiment 1 of the present invention;

[0062] Figure 2 This is a module structure diagram of a lane line detection device provided in Embodiment 2 of the present invention.

[0063] Figure 3This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. Detailed Implementation

[0064] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0065] The first embodiment of the present invention provides a lane line detection method, such as... Figure 1 The diagram shown is a schematic representation of a lane line detection method provided in Embodiment 1 of the present invention, which mainly includes the following steps:

[0066] S1. Obtain the first image;

[0067] Specifically, the first image is a two-dimensional image captured by an onboard camera.

[0068] S2. Scale the first image according to the preset image size to generate the second image;

[0069] Specifically, the preset image size is W0*H0, and the second image size is W0*H0, where W0 is the width of the preset image size and H0 is the height of the preset image size.

[0070] Here, the preset image size is the specified size of the input image for the backbone network used subsequently. For example, if the backbone network specifies an input image size of 800*320, then the width W of the preset image size is 800 and the height is 320. Since the size of the first image captured by the vehicle-mounted camera is not uniform, when using the backbone network, any first image needs to be scaled according to the preset image size to obtain a second image of size W0*H0.

[0071] S3. Based on the backbone network, feature extraction is performed on the second image to generate a first feature map and a second feature map;

[0072] Specifically, the second image is input into the backbone network for feature extraction to obtain three levels of output feature maps of different sizes, namely level one, level two and level three feature maps;

[0073] The backbone network is constructed using a three-level feature pyramid network with a residual network as the feature extraction network. The first-level feature map is used as the first feature map; the third-level feature map is used as the second feature map; the resolution of the first-level, second-level, and third-level feature maps increases sequentially, and the width / height of the first-level, second-level, and third-level feature maps are 1 / 32, 1 / 16, and 1 / 4 of the width / height of the second image, respectively.

[0074] S4. The first feature map is processed by the lane line proposal initialization network to generate the first lane line control point location set and the corresponding first lane line confidence.

[0075] Specifically, a global feature extraction network for lane lines is used to process the first feature map to generate global lane line features. The global feature extraction network processes the first feature map using spatial attention weighting. The size of the first feature map is represented as W1*H1. First, a first set of convolutions generates first spatial attention weights with a size of N*W1*H1, where N is the number of lane line proposals, which is set to 60 here. The sigmoid function is applied to the first spatial attention weights, and the sum of the weights in space is calculated. This sum is used as the denominator to normalize the first spatial attention weights, resulting in the second spatial attention weights.

[0076] The first feature map is processed by a second set of convolutions to generate a third feature map of size C*W1*H1, where C is the number of channels, which is set to 128 here. The second spatial attention weights and the third feature map are multiplied spatially and then summed in the spatial dimension to obtain the global features of the lane line, with a size of N*C. The global features are processed by two fully connected layers to obtain the set of control point locations for the first lane line and the corresponding confidence scores for the first lane line. The control point locations are normalized using the sigmoid function, and the confidence scores are calculated using the softmax function.

[0077] S5. Based on the definition of spline curve, process the first lane line control point location set to obtain the first lane line sampling point location set;

[0078] Specifically, the set of control points for the first lane is used as the control points for the spline curve, resulting in the first curve set. The positions of the sampling points of the first curve set on the curve are controlled by an equidistant parameter u set; here, the number of sampling points is 30, and N is used. s To represent this, the number of control points is 8, denoted by N. C Let represent the calculation of the corresponding basis function values ​​based on the parameter set u, and further combine them to generate the first transformation matrix M, with size N. s *N C Multiplying the first transformation matrix M by the set of sampling points for the first lane line yields the generated set of sampling points for the first lane line.

[0079] S6. Based on the lane line position refinement network, the second feature map and the first lane line sampling point position set are processed to obtain the second lane line control point position set and the corresponding second lane line confidence.

[0080] Specifically, because the first transformation matrix M contains the correspondence between curve control points and sampling points, the formula W = (M T M) -1 M T To calculate the second transformation matrix W, whose size is N C *N s .

[0081] Based on the second feature map and the first lane line sampling point location set, bilinear interpolation is performed on the features at the corresponding pixels of the first sampling point location set in the second feature map to obtain the feature set F of the first sampling points. s The size is N*N s *C. Obtaining the feature set F of the first sampling point. s Subsequently, F can also be further enhanced and updated through deformable attention mechanisms. s To obtain richer local features of the sampling points. This is achieved through formula F. C =WF s The feature set F of the first sampling point s Transform into a set of control point features F C Its size is N*N C *C. The control point feature set F C The input consists of a second prediction head composed of two fully connected layers, which generates a set of second control point locations for the lane lines and the corresponding second lane line confidence scores. The control point locations are normalized using the sigmoid function, and the confidence scores are calculated using the softmax function.

[0082] S7. Based on the spline curve definition, process the set of control point locations of the second lane line to obtain the set of sampling point locations of the second lane line.

[0083] Specifically, the first transformation matrix M is multiplied by the second lane control point location set to obtain the second lane sampling point location set of the lane line.

[0084] S8. Based on the non-maximum suppression algorithm, process the second lane line sampling point location set and the second lane line confidence to generate the third lane line sampling point location set;

[0085] Specifically, the first curve distance D is obtained by calculating the distance set between the second set of sampling points of the lane lines based on the chamfer distance. The parameter r is set to 9 to normalize the first curve distance D, thus obtaining the second curve distance D. norm The standardized formula used is:

[0086]

[0087] The fast nonmaximum suppression algorithm is adopted to make Dnorm In the alternative algorithm, the role of IoU is as follows: First, the first curve distance matrix is ​​calculated. Then, the first curve distance matrix is ​​upper triangulated to obtain the second curve distance matrix. The maximum value in the first dimension of the second curve distance matrix is ​​taken to generate the first selection vector. The first selection vector is binarized using an NMS threshold of 0.2 to obtain the second selection vector. The second selection vector is set to False for positions where the first selection vector is greater than the NMS threshold, and True for positions where the first selection vector is less than the NMS threshold. The length of the first selection vector is N, and the size of the second lane line sampling point location set is N*N. s *2, Use the first selection vector to select the second lane line sampling point location set in the first dimension. Keep the rows where the first selection vector position is True and discard the rows where the first selection vector position is False to obtain the fourth lane line sampling point location set.

[0088] S9. The third lane line sampling point location set is processed based on inverse perspective transformation to generate the fourth lane line sampling point location set; the fourth lane line sampling point location set is defined in the top view perspective.

[0089] Specifically, the inverse perspective transformation matrix H is calculated based on the camera's intrinsic and extrinsic parameters. Then, a fourth lane line sampling point location set is generated based on the inverse perspective transformation matrix H and the third lane line sampling point location set. This step transforms the 2D lane line points to a ground coordinate system, and the resulting fourth lane line sampling point location set can be used for downstream tasks in lane line detection.

[0090] S10. Calculate the overall loss function of the network, implement the backpropagation algorithm, and update the network parameters;

[0091] Specifically, based on the set of sampling points for the second lane line and the manually labeled true lane line values, the area S of the closed region between curves is calculated, and the length L of the true lane line values ​​is also calculated. A series of equal numbers of sampling points are taken on each curve (here, the number of sampling points is set to 100), dividing the closed region into multiple triangles. The difference in the sequence number of the sampling points corresponding to the vertices of each triangle does not exceed 1. The area S of the closed region is approximated by calculating the sum of the areas of the triangles. For sampling points on the same curve, adjacent sampling points are connected to form a series of shorter line segments. The lengths of these line segments are calculated and summed to obtain an approximate length L of the true lane line values. Based on the area S of the closed region between curves and the length L of the true lane line values, an area loss function is calculated. Based on the set of sampling points for the second lane line and the manually labeled true values ​​of the lane lines, calculate the endpoint loss function L. pointThe endpoint loss function is calculated using the L2 loss function. Based on the classification loss function L... cls Area loss function L area Endpoint loss function L point The overall loss function of the network is calculated, and the result is used for backpropagation and parameter updates. The matching loss is calculated using the same method as for the overall network loss function, and labels are assigned between the ground truth lane values ​​and the set of sampling points for the second lane. The label assignment is implemented using a one-to-many simOTA algorithm.

[0092] Figure 2 This is a module structure diagram of a lane line detection device provided in Embodiment 2 of the present invention. This device can be a terminal device or server implementing the aforementioned method embodiment, or it can be a device that enables the aforementioned terminal device or server to implement the aforementioned method embodiment. For example, the device can be a device or chip system of the aforementioned terminal device or server. Figure 2 As shown, the device includes: an image acquisition device, a backbone network processing module, a lane line proposal initialization module, a lane line position refinement module, and a non-maximum suppression module.

[0093] Image acquisition device 1 is used to acquire the first image;

[0094] Backbone network processing module 2 is used to extract features from the second image based on the backbone network, and generate a first feature map and a second feature map;

[0095] The lane line proposal initialization module 3 is used to process the first feature map based on the lane line proposal initialization network to generate a first lane line control point location set and a corresponding first lane line confidence score.

[0096] The lane line position refinement module 4 is used to process the second feature map and the first lane line control point position set based on the lane line position refinement network to obtain the second lane line control point position set and the corresponding second lane line confidence.

[0097] The non-maximum suppression module 5 is used to process the second lane line sampling point location set and the second lane line confidence based on the non-maximum suppression algorithm to generate the third lane line sampling point location set.

[0098] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0099] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0100] Accordingly, the present invention also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the lane line detection method as described above. Figure 3 The diagram shown is a structural schematic of an electronic device provided in Embodiment 3 of the present invention, except that... Figure 3 In addition to the processor, memory, DMA controller, disk, and non-volatile memory shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0101] Accordingly, the present invention also provides a computer-readable storage medium storing computer instructions thereon, which, when executed by a processor, implement the lane detection method described above. The computer-readable storage medium can be an internal storage unit of any data processing device as described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0102] While the invention has been described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the invention. Therefore, it should be understood that many modifications can be made to the exemplary embodiments, and other arrangements can be designed without departing from the spirit and scope of the invention as defined by the appended claims. It should be understood that different dependent claims and features described herein can be combined in ways different from those described in the original claims. It is also understood that features described in conjunction with individual embodiments can be used in other described embodiments.

Claims

1. A lane line detection method, characterized in that, Includes the following steps: S1. Obtain the first image; S2. Scale the first image according to the preset image size to generate the second image; S3. Based on the backbone network, feature extraction is performed on the second image to generate a first feature map and a second feature map; S4. The first feature map is processed by the lane line proposal initialization network to generate the first lane line control point location set and the corresponding first lane line confidence. S5. Based on the definition of spline curve, process the first lane line control point location set to obtain the first lane line sampling point location set; S6. Based on the lane line position refinement network, the second feature map and the first lane line sampling point position set are processed to obtain the second lane line control point position set and the corresponding second lane line confidence. S7. Based on the spline curve definition, process the set of control point locations of the second lane line to obtain the set of sampling point locations of the second lane line. S8. Based on the non-maximum suppression algorithm, process the second lane line sampling point location set and the second lane line confidence to generate the third lane line sampling point location set; S9. Process the set of sampling points for the third lane line based on inverse perspective transformation to generate the set of sampling points for the fourth lane line. The set of sampling points for the fourth lane line is defined in the top-view perspective; S10. Calculate the overall loss function of the network, implement the backpropagation algorithm, and update the network parameters; Step S5 specifically includes the following steps: S501. Use the set of control points of the first lane line as the control points of the spline curve to generate the first curve set; S502, the positions of the sampling points of the first curve set on the curve are obtained through equidistant parameters. Controlled by sets; S503, according to the parameters Set calculation of the corresponding basis function values; S504. Combine the basis function values ​​to generate the first transformation matrix. ; S505, Based on the first set of lane control point locations and the first transformation matrix Generate the set of sampling point locations for the first lane line; Step S6 specifically includes the following steps: S601, For the first transformation matrix Perform matrix transformations to generate the second transformation matrix. The matrix transformation is achieved through the following formula: ; S602. Based on the second feature map and the first lane line sampling point location set, perform bilinear interpolation on the features at the corresponding pixels of the first sampling point location set in the second feature map to obtain the feature set of the first sampling points. ; S603, Based on the feature set of the first sampling point and the second transformation matrix Generate a set of control point features The specific implementation method is as follows ; S604, Set the control point feature set The second prediction head, consisting of a fully connected network, is input to generate the set of second control point locations for the lane lines, as well as the corresponding second lane line confidence scores.

2. The lane line detection method according to claim 1, characterized in that, Step S3 specifically includes the following steps: S301. Input the second image into the backbone network to extract features and obtain three levels of output feature maps of different sizes, namely, level one, level two and level three feature maps; S302. Use the first-level feature map as the first feature map; S303. Use the third-level feature map as the second feature map.

3. The lane line detection method according to claim 1, characterized in that, Step S4 specifically includes the following steps: S401. The first feature map is processed by a global feature extraction network for lane lines to generate global features for lane lines. S402. Input the global features of the lane line into the first prediction head composed of a fully connected network to generate the first lane line control point location set of the lane line and the corresponding first lane line confidence.

4. The lane line detection method according to claim 1, characterized in that, Step S8 specifically includes the following steps: S801. Calculate the distance set between the sampling point sets of the second lane line based on the chamfer distance, and calculate the distance of the first curve. And transform to obtain the standardized second curve distance. : ； Where parameters It is a constant; S802, Based on the nonmaximum suppression algorithm and the distance of the second curve. The second set of lane line sampling point locations is filtered to obtain the third set of lane line sampling point locations.

5. The lane line detection method according to claim 4, characterized in that, Step S9 specifically includes the following steps: S901. Based on the intrinsic and extrinsic parameters of the image acquisition device, generate the inverse perspective transformation matrix. ; S902, Based on the inverse perspective transformation matrix The third lane line sampling point location set is used to generate the fourth lane line sampling point location set; Step S10 specifically includes the following steps: S1001. Calculate the area of ​​the closed region between curves based on the second set of lane line sampling point locations and the manually labeled lane line true values. And calculate the true length of the lane line. The area of ​​the closed region between the curves The length of the true value of the lane line is approximated by decomposing it into the area of ​​a triangle. Decompose it into line segment lengths for approximate calculation; S1002, Based on the area of ​​the closed region between the curves and the length of the true value of the lane line Calculate the area loss function ; S1003. Calculate the endpoint loss function based on the second set of lane line sampling point locations and the manually labeled true values ​​of the lane lines. The endpoint loss function is obtained through... The loss function is calculated; S1004. Based on the classification loss function, area loss function, and endpoint loss function, calculate the overall loss function of the network and use the calculation results for backpropagation and parameter update. S1005. Calculate the matching loss using the same method as the loss function of the overall computational network, and assign labels to the true values ​​of lane lines and the set of second sampling points.

6. An apparatus for implementing the lane line detection method according to any one of claims 1-5, characterized in that, include: Image acquisition device, used to acquire the first image; The backbone network processing module is used to extract features from the second image based on the backbone network to generate a first feature map and a second feature map. The lane line proposal initialization module is used to process the first feature map based on the lane line proposal initialization network to generate a first lane line control point location set and a corresponding first lane line confidence score. The lane line position refinement module is used to process the second feature map and the first lane line sampling point position set based on the lane line position refinement network to obtain the second lane line control point position set and the corresponding second lane line confidence. The non-maximum suppression module is used to process the second lane line sampling point location set and the second lane line confidence based on the non-maximum suppression algorithm to generate the third lane line sampling point location set.

7. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-5.

8. A computer-readable storage medium storing computer instructions thereon, characterized in that, When executed by the processor, this instruction implements the steps of the method as described in any one of claims 1-5.

Images