A lane line detection method based on lane segments

Abstract

The present invention proposes a lane line detection method based on lane segments. By utilizing the local and global structural information of lane points, a set of multiple lane reference points with local maximum responses is first extracted to provide a rough global lane structure. A lane line is then divided into several lane segments by the lane reference points. The offset between the lane reference point and other lane points is explicitly predicted. Then, based on the offset prediction, reliable adjacent lane points are associated to construct the lane segment corresponding to each lane reference point. Structural correlations between adjacent lane points are constructed within the lane segments of the local area. Finally, the complete lane line is reconstructed by smoothly connecting all lane segments. The present invention utilizes local lane details and global structural information to simultaneously achieve good local lane positioning and smooth lane structure prediction.

Description

Technical Field

[0001] The present invention relates to a lane line detection technology widely used in autonomous driving, and specifically to a lane line detection method based on lane segments. Background Art

[0002] Lane detection is a critical task in autonomous driving. To accomplish this task, lane lines are typically represented as a set of ordered points. Point-based lane detection methods first locate each individual lane point and reconstruct the complete lane line in a bottom-up manner. While point-based methods offer advantages in localizing lane points, they rarely consider the overall structure that lane points should satisfy and can therefore result in inaccurate lane line localization and non-smooth lane predictions.

[0003] Point-based lane detection methods fail to consider or struggle to construct lane point correlations at an appropriate scale. Consequently, when lane lines are obscured or lane cues are invisible due to extreme lighting conditions, these methods suffer from reduced lane positioning accuracy, making it difficult to obtain a reliable overall lane structure and leading to reduced lane detection accuracy. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention proposes a lane line detection method based on lane segments, which utilizes the local and global structural information of lane points. First, a set of multiple lane reference points with local maximum responses are extracted, which can provide a rough global lane structure. A lane line is then divided into several lane segments by the lane reference points. The offset between the lane reference point and other lane points is explicitly predicted. Then, based on the offset prediction, reliable adjacent lane points are associated to construct the lane segment corresponding to each lane reference point. The structural correlation of adjacent lane points is constructed within the lane segment of the local area. Finally, the complete lane line is reconstructed by smoothly connecting all lane segments. The present invention utilizes local lane details and global structural information to simultaneously achieve good local lane positioning and smooth lane structure prediction.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A lane line detection method based on lane segments includes the following steps:

[0007] Step S1: The forward traffic image is subjected to feature extraction through the backbone network and the self-attention module, and a high-resolution lane feature map is reconstructed based on the feature pyramid network.

[0008] Step S2: The confidence prediction head and the offset prediction head predict the lane confidence map based on the lane feature map. and lane point horizontal offset map Among them, the lane confidence map Each channel output corresponds to the confidence of a lane line, lane point horizontal offset map Each channel output corresponds to the horizontal offset of other lane points relative to the given lane reference point;

[0009] Step S3: Use the maximum pooling operation to extract lane confidence maps A batch of lane points with local maximum responses are selected. When the horizontal offset relative to a given reference point is lower than a given screening threshold, it is selected as the final lane reference point. Therefore, the selected lane reference point has both high confidence and accurate horizontal offset prediction;

[0010] Step S4: The lane reference points obtained above divide a complete lane line into multiple lane segments. The corresponding horizontal offset prediction map is associated with other lane points in the local area according to the position index of the lane reference point to obtain the lane segment corresponding to each lane reference point.

[0011] Step S5: In the area where adjacent lane segments overlap, the final position of the lane point in the overlapping area is determined by weighted averaging, thereby smoothly connecting multiple lane segments to obtain the final lane line position.

[0012] Furthermore, in step S2, the lane confidence map and lane point horizontal offset map is defined as follows:

[0013] The goal of the lane detection task is to predict a set of lane lines L = {l1,l2,…,l K}, where l represents a single lane line, the subscript K is the total number of lanes, and the lane line is represented by a set of lane points distributed on evenly spaced lines, as shown below:

[0014]

[0015] y i =i·Δy,i=1,…,N k (1b)

[0016] Where Δy is the vertical distance between two adjacent lane points, Indicates that the kth lane is on the yth i The horizontal coordinate of the lane point on the line, N k is the maximum number of lane points that constitute a given single lane line;

[0017] In order to enhance the local and global correlation between lane points, a backbone network, a pyramid network and two prediction heads are set, namely the confidence prediction head and the offset prediction head; first, the lane confidence map is used to Select a set of reliable lane reference points to provide the overall structure of the lane; then, use the lane point horizontal offset map Construct the corresponding lane segments of the lane reference points to fill in the local lane details; for a given input image I, Among them H I 、W I is the input image height and width, R is the real number domain; the final output of the two prediction heads is the lane confidence map R + is the field of positive real numbers, and the lane point horizontal offset map r is the output step size, N k is the maximum number of lane points that constitute a single lane; lane confidence map Describes the probability that each pixel is a lane point. The K output channels are associated with K lane instances respectively. The y coordinates of the lane points defined in formula (1a) and formula (1b) are fixed. If a lane reference point in the i-th row of the k-th lane is determined Then all other lane points are relative to the lane reference point The horizontal offset of the lane point is obtained through the horizontal offset diagram The i-th channel index is obtained.

[0018] Furthermore, the selection of lane reference points in step S3 includes:

[0019] Use the predicted lane confidence map to find lane reference points in lane areas with significant responses;

[0020] By using a kernel size of K ref ×K ref The maximum pooling layer selects a group of points and adjusts the global distribution of lane reference points and lane segments by adjusting the size of the maximum pooling kernel. The lane reference points maintain the consistency of the lane confidence map and the lane point horizontal offset map.

[0021] Taking into account the extreme imbalance of positive and negative samples, the focal loss function is used to constrain the learning of the confidence regression task as follows:

[0022]

[0023] in, is the confidence prediction loss, L CE (·) is the focal loss function, and G conf Represent the confidence prediction and true value of the lane point respectively, N is the number of output pixels, and xy represents the coordinates of the lane point, that is,

[0024] Furthermore, the step S4 includes:

[0025] Taking any lane point as the lane reference point, the offset head predicts the horizontal offset of all other lane points relative to the lane reference point, constructs the global relationship between lane points, and uses local lane clues to accurately determine the position of lane points in the area; the lane point is represented as (x, y), and its relative to the lane reference point (x r ,y r ) is predicted to have a horizontal offset of δx. If (x+δx,y) and (x r ,y r ) is close enough, that is, ||x r Lane points are associated with lane reference points if -x-δx||1≤δ, where ||·||1 is the 1-norm of the calculated distance, where δ is a given threshold; a complete lane is constructed using the weighted sum of lane segments;

[0026] The consistency between the lane confidence map and the lane point horizontal offset map should be considered in the loss calculation of the offset prediction:

[0027]

[0028] where L1(·) is the smoothed L1 loss, is the predicted lane confidence map, is the prediction confidence value at point (x,y), and Represents the predicted and true values ​​of the offset of the i-th lane point, N p is the number of lane points in the image, N k is the maximum number of lane points that constitute a single lane line; G oft are mapped to the desired output scale during training and do not need to have integer values;

[0029] For the input image The offset map belongs to the output space Therefore, a point in the offset map is associated with an r×r grid in the original input space, where r is the output stride. Lane points fall on predefined rows with fixed y coordinates. Horizontal adjustment is used to compensate for quantization errors, introducing an additional quantization loss function. To constrain the learning task of horizontal quantization error:

[0030]

[0031] where L1(·) is the smoothed L1 loss, and G qua It is the prediction and ground truth space of the lane position of each point in the output, and each lane point (x,y k ) The corresponding position truth value is calculated as:

[0032]

[0033] Among them, x is the horizontal coordinate of the lane point, y k is the vertical coordinate of the lane point, defined by formula (1);

[0034] The final total training loss for:

[0035]

[0036] Among them, α and β are weighting coefficients.

[0037] Furthermore, the step S5 includes:

[0038] According to the lane point confidence map output by the network and the offset map relative to the lane reference point, the lane segments and lanes are reconstructed. A set of lane reference points with local maximum confidence is obtained through the maximum pooling operation and the confidence threshold. Then, a lane is divided into several parts by a set of lane reference points. For each lane reference point, the corresponding lane line segment is constructed between the upper and lower bounds determined by its two adjacent lane reference points. Given a lane reference point, the lane is first divided into several parts according to the row index of the reference point. The corresponding offset map is extracted from the lane point; if the following conditions are met, the lane point is associated with the corresponding lane reference point and becomes part of the lane segment. The coordinates of the lane reference point are expressed as (x r ,y r ) The upper and lower adjacent lane reference points are respectively denoted as and Then the upper and lower boundaries of a lane segment are determined as and y1 and The starting and ending positions of the lane are defined by formula (1). A rectangular area with a width of δ is defined with the lane reference point as the center. The left and right boundaries of the area are B lt =max(x r -δ,0) and If the following conditions are met, then the lane point (x i ,y i ) is determined to be part of a lane segment:

[0039] B up <y i bot (6a)

[0040]

[0041] in, is the lane reference point (x r ,y r ​) related offset map; by explicitly predicting the horizontal offset of all lane points relative to the lane reference point, the global structural relationship of the lane is established; at this point, the corresponding lane segments have been obtained through local lane cues;

[0042] The lane representation based on lane segments is described as:

[0043]

[0044] in, is the number of lane segments that constitute the kth lane, and S represents the lane segment;

[0045] In the overlapping area of ​​two adjacent lane segments, the final lane point position is determined by taking a weighted average of the lane point positions of different lane segments; the complete lane line is smoothly reconstructed by connecting all lane segments.

[0046] Beneficial effects:

[0047] Point-based methods have advantages in local lane point positioning, but they rarely consider the structural correlation of lane points, making it difficult to obtain accurate lane point positions and reliable overall lane structures. The present invention proposes a lane line detection method based on lane segments. First, multiple lane reference points are extracted in areas with rich lane textures. Then a lane is divided into several lane segments by the lane reference points. The corresponding lane line segments are constructed based on the predicted offsets between the lane reference points and other lane points in the local range. The complete lane is reconstructed by smoothly connecting all lane segments. This process can well explore the multi-level structural relationship of the lane. The lane reference points provide a rough global lane structure, while constructing the local correlation of lane points within the lane segments. By making full use of local lane details and global structural information, the present invention can achieve good local lane positioning and smooth and reliable lane structure prediction in complex road conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a framework diagram of a lane line detection method based on lane segments of the present invention;

[0049] Figure 2 Flowchart for constructing lane points; (a) is the confidence map of the lane, (b) is a set of reference lane points with local maximum confidence obtained by max pooling and confidence thresholding, (c) is a detailed map constructed from a lane segment, (d) is a detailed map constructed from another lane segment, and (e) is a schematic diagram of lane lines. DETAILED DESCRIPTION

[0050] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only intended to illustrate the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0051] like Figure 1 As shown, a lane line detection method based on lane segments of the present invention includes the following steps:

[0052] Step S1: The forward traffic image is subjected to feature extraction through the backbone network and the self-attention module, and a high-resolution lane feature map is reconstructed based on the feature pyramid network.

[0053] Step S2: The confidence prediction head and the offset prediction head predict the lane confidence map based on the lane feature map. and lane point horizontal offset map Among them, the lane confidence map Each channel output corresponds to the confidence of a lane line, lane point horizontal offset map Each channel output corresponds to the horizontal offset of other lane points relative to the given lane reference point;

[0054] Step S3: Use the maximum pooling operation to extract lane confidence maps A batch of lane points with local maximum responses are selected. When the offset horizontal offset prediction value corresponding to the lane point is lower than a given screening threshold, it is selected as the final lane reference point. Therefore, the lane reference point has both high confidence and accurate offset prediction;

[0055] Step S4: The lane reference points obtained above divide a complete lane line into multiple lane segments. Based on the position index of the lane reference point, the corresponding horizontal offset prediction map can be associated with other lane points in the local area to obtain the lane segment corresponding to each lane reference point.

[0056] Step S5: In the area where adjacent lane segments overlap, the final position of the lane point is confirmed by weighted averaging, thereby smoothly connecting multiple lane segments to obtain the final lane line position.

[0057] Specifically, the lane confidence map in step S2 and lane point horizontal offset map is defined as follows:

[0058] The goal of the lane detection task is to predict a set of lanes L = {l1,l2,…,l K}, where l represents a single lane line and the subscript K is the total number of lanes. Lane lines are usually represented by a set of lane points distributed on evenly spaced lines, as shown below.

[0059]

[0060] y i =i·Δy,i=1,…,N k (1b)

[0061] Where Δy is the vertical distance between two adjacent lane points, Indicates that the kth lane is on the yth i The horizontal coordinate of the lane point on the line, N k is the maximum number of lane points that make up a lane.

[0062] In order to enhance the local and global correlation between lane points, the overall framework of the present invention mainly consists of three parts, including the backbone network, the pyramid network and two prediction heads, namely the confidence prediction head and the offset prediction head. First, a set of reliable lane reference points are selected through the lane confidence map, which provide the overall structure of the lane. Then, the lane point horizontal offset map is used to construct the corresponding lane fragments of the lane reference points to fill in the local lane details. For a given input image I, Among them H I 、W I The final output of the two prediction heads is the lane confidence map. R + is the field of positive real numbers, and the lane point horizontal offset map r is the output step size. N k Is the maximum number of lane points that constitute a single lane. Lane confidence map Describes the probability that each pixel is a lane point, and the K output channels are associated with K lane instances. It is worth noting that the y coordinates of the lane points defined in formula (1a) and formula (1b) are fixed. If a lane reference point in the i-th row of the k-th lane is determined Then all other lane points are relative to the lane reference point The horizontal offset of the lane point can be obtained by the horizontal offset map The i-th channel index is obtained.

[0063] The selection of lane reference points in step S3 includes:

[0064] The lane confidence map represents the probability of a pixel being a lane point. Generally speaking, when lane texture is prominent, the corresponding confidence response is large. Conversely, if the lane is invisible due to extreme lighting or occlusion, the response in the corresponding area may be weak. Therefore, the predicted confidence map should be fully utilized to find lane reference points in salient lane regions.

[0065] In fact, an ideal set of lane reference points should satisfy two important constraints. First, the lane reference points should be selected from the region where the lane cues are reliable and the local response is the strongest. Second, the distribution of lane reference points should reflect the global structure of the lane to some extent. A natural and effective strategy is to use a kernel size of K to select lane reference points. ref ×K ref The maximum pooling layer selects a set of points. By adjusting the size of the maximum pooling kernel, the global distribution of lane reference points and lane segments can be adjusted accordingly. In addition, the lane reference points need to maintain consistency between the lane confidence map and the lane point horizontal offset map, which means that they must not only have high confidence but also have small offset values.

[0066] Due to the inherently elongated structure of lanes, lanes occupy only a small fraction of the pixels in an image, approximately 3.1% on the CULane dataset, measured on a training set with a lane width of 30 pixels and an image resolution of 1640 x 590. Considering the extreme imbalance between positive and negative samples, a focal loss function is used to constrain the learning of the confidence regression task:

[0067]

[0068] in, is the confidence prediction loss, L CE (·) is the focal loss function, and G conf Represent the confidence prediction and true value of the lane point respectively. N is the number of output pixels, that is, xy represents the lane point coordinates.

[0069] The construction of lane segments in step S4 includes:

[0070] Taking any lane point as the lane reference point, the offset head predicts the horizontal offset of all other lane points relative to the lane reference point. In this way, the global relationship between lane points is explicitly constructed. Since lane clues usually appear around reliable lane reference points, the position of lane points in this area can be accurately determined using rich local lane clues. In the method of the present invention, a lane segment consists of multiple lane points that are close to the reference point and have accurate offset predictions. If multiple reliable lane segments are determined, the complete lane can be reconstructed from the local lane segments. The present invention represents a lane point as (x, y), which is relative to the lane reference point (x r ,y r ) is predicted to have a horizontal offset of δx. If (x+δx,y) and (x r ,y r ) is close enough, that is, ||x r -x-δx||1≤δ, ||·||1 is the 1-norm of the calculated distance, then the lane point is associated with the lane reference point, where δ is a given threshold. Figure 2 An intuitive explanation is given in . For the lane confidence map, a set of lane points with the largest local confidence are selected as lane reference points. Taking a lane reference point, the corresponding lane segment can be constructed between the upper and lower bounds determined by its two adjacent lane reference points. Given a lane reference point Figure 2 (c) and Figure 2 For the black points in (d), if the position of the lane point adjusted according to the predicted offset is within the rectangular area centered on the black lane reference point, then other lane points can be associated as part of the lane segment. A complete lane can be constructed as a weighted sum of lane segments.

[0071] In fact, a complete lane can be reconstructed with a lane point based on the corresponding offset map. However, only lane points with high confidence can be selected as lane reference points and are truly needed. Therefore, the consistency between the lane confidence map and the lane point horizontal offset map should be considered in the loss calculation of the offset prediction:

[0072]

[0073] where L1(·) is the smoothed L1 loss, is the predicted lane confidence map. is the predicted confidence value at point (x,y). and Represent the predicted and true values ​​of the offset of the i-th lane point respectively. N p is the number of lane points in the image. N k is the maximum number of lane points that constitute a single lane. Note that G oftis mapped to the desired output scale during training and does not need to have integer values. Therefore, G oft represents the exact position of the lane point without quantization error. As shown in Equation (3). The offset loss is modulated by the confidence. By introducing the confidence map, the offset loss can be mainly contributed by points with higher confidence, which enhances the robustness of lane segment construction.

[0074] In the method of the present invention, for the input image The offset map belongs to the output space Therefore, a point in the offset map is associated with an r×r grid in the original input space. For larger output stride r, the resulting quantization error will reduce the accuracy of lane point localization in the input space. In our method, lane points fall on predefined lines with fixed y coordinates. Therefore, small horizontal adjustments are required to compensate for the quantization error. An additional quantization loss function is introduced to constrain the learning task to the horizontal quantization error:

[0075]

[0076] where L1(·) is the smoothed L1 loss, and G qua It is the prediction and ground truth space of the lane position of each point in the output, and each lane point (x,y k ) can be calculated as x is the horizontal coordinate of the lane point, y is the horizontal coordinate of the lane point, k is the vertical coordinate of the lane point, defined by formula (1).

[0077] The horizontal quantization error prediction head consists of several convolutional layers, not in Figure 2 Displayed in.

[0078] The final total training loss for:

[0079]

[0080] Among them, α and β are weighting coefficients.

[0081] In step S5, lane construction includes:

[0082] According to the lane point confidence map and the offset map relative to the lane reference point output by the network, lane segments and lanes can be accurately and effectively reconstructed. The complete lane construction process is as follows Figure 2 As shown. Given Figure 2 The confidence map of the lane in (a) is obtained by the maximum pooling operation and the confidence threshold to obtain a set of lane reference points with local maximum confidence, such as Figure 2(b). A lane is then divided into several parts by a set of lane reference points. For each lane reference point, a corresponding lane segment can be constructed between the upper and lower bounds determined by its two adjacent lane reference points. Adjacent lane segments can partially overlap. The details of lane segment construction are as follows: Figure 2 (c) and Figure 2 As shown in (d), the final lane segment is as follows Figure 2 (e). Given a lane reference point (black dot), first, according to the row index of the lane reference point from As mentioned above, the horizontal offset map records the horizontal offset between the lane reference point and other lane points. If the following conditions are met, the lane point is associated with the corresponding lane reference point and becomes part of the lane segment. The coordinates of the lane reference point are expressed as (x r ,y r ) The upper and lower adjacent lane reference points are respectively denoted as and Then the upper and lower boundaries of a lane segment can be determined as and y1 and That is, the starting and ending positions of the lane, defined by formula (1). Figure 2 (c) and Figure 2 The black lane reference point in (d) is the center, defining a rectangular area with a width of δ. The left and right boundaries of the area are B lt =max(x r -δ,0) and If the following conditions are met, then the lane point (x i ,y i ) is determined to be part of the lane segment,

[0083] B up <y i bot (6a)

[0084]

[0085] in, is the lane reference point (x r ,y r ) related offset map. By explicitly predicting the horizontal offset of all lane points relative to the lane reference point, the global structural relationship of the lane can be established. Figure 2 (c) and Figure 2 ​As shown in (d), lane points that fall within the rectangular area after horizontal offset adjustment are successfully associated with the corresponding lane segments, while lane points that fall outside the rectangular area after horizontal offset adjustment are ignored due to inaccurate horizontal offset prediction. At this point, the corresponding lane segments have been obtained through local lane cues. Note that the lane representation based on lane segments can be described as:

[0086]

[0087] Where S represents the lane segment, is the number of lane segments that make up the kth lane. In the overlapping region of two adjacent lane segments, the final lane point position is determined by taking a weighted average of the lane point positions of different lane segments. Therefore, the complete lane line can be smoothly reconstructed by connecting all lane segments.

[0088] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A lane line detection method based on lane segments, characterized in that: The steps include: Step S1: The forward traffic image is subjected to feature extraction through the backbone network and the self-attention module, and a high-resolution lane feature map is reconstructed based on the feature pyramid network. Step S2: The confidence prediction head and the offset prediction head predict the lane confidence map based on the lane feature map. and lane point horizontal offset map ; Among them, the lane confidence map Each channel output corresponds to the confidence of a lane line, lane point horizontal offset map Each channel output corresponds to the horizontal offset of other lane points relative to the given lane reference point; Lane confidence map and lane point horizontal offset map is defined as follows: The goal of the lane detection task is to predict a set of lane lines , where l represents a single lane line, the subscript K is the total number of lanes, and the lane line is represented by a set of lane points distributed on evenly spaced lines, as shown below: （1a） （1b） in, is the vertical distance between two adjacent lane points, Indicates that the kth lane is in The horizontal coordinates of the lane points on the line, is the maximum number of lane points that constitute a given single lane line; In order to enhance the local and global correlation between lane points, a backbone network, a pyramid network and two prediction heads are set, namely the confidence prediction head and the offset prediction head; first, the lane confidence map is used to Select a set of reliable lane reference points to provide the overall structure of the lane; then, use the lane point horizontal offset map Construct the corresponding lane segments of the lane reference points to fill in the local lane details; for a given input image I, ,in 、 is the input image height and width, R is the real number domain; the final output of the two prediction heads is the lane confidence map , R + is the field of positive real numbers, and the lane point horizontal offset map , r is the output step size, is the maximum number of lane points that constitute a single lane; lane confidence map Describes the probability that each pixel is a lane point. The K output channels are associated with K lane instances respectively. The y coordinates of the lane points defined in formula (1a) and formula (1b) are fixed. If a lane reference point in the i-th row of the k-th lane is determined , then all other lane points are relative to the lane reference point The horizontal offset of the lane point is obtained through the horizontal offset diagram The i-th channel index is obtained; Step S3: Use the maximum pooling operation to extract lane confidence maps A batch of lane points with local maximum responses are selected. When the horizontal offset relative to a given reference point is lower than a given screening threshold, it is selected as the final lane reference point. Therefore, the selected lane reference point has both high confidence and accurate horizontal offset prediction; Step S4: The lane reference points obtained above divide a complete lane line into multiple lane segments. The corresponding horizontal offset prediction map is associated with other lane points in the local area according to the position index of the lane reference point to obtain the lane segment corresponding to each lane reference point. Step S5: In the area where adjacent lane segments overlap, the final position of the lane point in the overlapping area is determined by weighted averaging, thereby smoothly connecting multiple lane segments to obtain the final lane line position.

2. The lane line detection method based on lane segments according to claim 1, characterized in that: The selection of lane reference points in step S3 includes: Use the predicted lane confidence map to find lane reference points in lane areas with significant responses; The kernel size is The maximum pooling layer selects a group of points and adjusts the global distribution of lane reference points and lane segments by adjusting the size of the maximum pooling kernel. The lane reference points maintain the consistency of the lane confidence map and the lane point horizontal offset map. Taking into account the extreme imbalance of positive and negative samples, the focal loss function is used to constrain the learning of the confidence regression task as follows: （2） in, is the confidence prediction loss, is the focal loss function, and Represent the confidence prediction and true value of the lane point respectively, N is the number of output pixels, and xy represents the coordinates of the lane point, that is, .

3. The lane line detection method based on lane segments according to claim 2, characterized in that: The step S4 comprises: Taking any lane point as the lane reference point, the offset head predicts the horizontal offset of all other lane points relative to the lane reference point, constructs the global relationship between lane points, and uses local lane clues to accurately determine the position of lane points in the area; the lane point is represented as , which is relative to the lane reference point The predicted horizontal offset is ,if and The horizontal position of is close enough, that is , To calculate the 1-norm of the distance, the lane point is associated with the lane reference point, where is a given threshold; the complete lane is constructed using the weighted sum of lane segments; The consistency between the lane confidence map and the lane point horizontal offset map should be considered in the loss calculation of the offset prediction: （3） in, is the smoothed L1 loss, is the predicted lane confidence map, Yes The prediction confidence value at and Represent the predicted and true values ​​of the offset of the i-th lane point, is the number of lane points in the image, is the maximum number of lane points that constitute a single lane line; are mapped to the desired output scale during training and do not need to have integer values; For the input image , the offset map belongs to the output space , so a point in the offset map is different from a point in the original input space Grid related, r is the output stride, lane points fall on a fixed On the predefined lines of the coordinates, horizontal adjustment is used to compensate for the quantization error, introducing an additional quantization loss function To constrain the learning task of horizontal quantization error: （4） in, is the smoothed L1 loss, and is the prediction and ground truth space of the lane position of each point in the output, respectively for each lane point in the input space The corresponding position truth value is calculated as: Among them, x is the horizontal coordinate of the lane point, is the vertical coordinate of the lane point, defined by formula (1); The final total training loss for: （5） in, and is the weighting coefficient.

4. The lane line detection method based on lane segments according to claim 3, characterized in that: The step S5 comprises: According to the lane point confidence map output by the network and the offset map relative to the lane reference point, the lane segments and lanes are reconstructed. A set of lane reference points with local maximum confidence is obtained through the maximum pooling operation and the confidence threshold. Then, a lane is divided into several parts by a set of lane reference points. For each lane reference point, the corresponding lane line segment is constructed between the upper and lower bounds determined by its two adjacent lane reference points. Given a lane reference point, the lane is first divided into several parts according to the row index of the reference point. Extract the corresponding offset map from ; if the following conditions are met, the lane point is associated with the corresponding lane reference point and becomes part of the lane segment, and the coordinates of the lane reference point are expressed as The upper and lower adjacent lane reference points are respectively denoted as and , then the upper and lower boundaries of a lane segment are determined as and , and The starting and ending positions of the lane are defined by formula (1). With the lane reference point as the center, a width of The rectangular area with the left and right boundaries of the area is and , if the following conditions are met, then the lane point Determined to be part of a lane segment: （6a） （6b） in, Is the lane reference point The corresponding lane segments are obtained by using local lane cues. The lane representation based on lane segments is described as: （7） in, is the number of lane segments that make up the k-th lane, represents a lane segment; In the overlapping area of ​​two adjacent lane segments, the final lane point position is determined by taking a weighted average of the lane point positions of different lane segments; the complete lane line is smoothly reconstructed by connecting all lane segments.

Images