Lane line detection method and system for expected function safety, vehicle and medium

Abstract

The invention discloses a lane line detection method and system for expected function safety, a vehicle and a medium. The detection method comprises the following steps: constructing a preselected lane line by using two-dimensional coordinate points; constructing a backbone network based on a residual structure, and inputting the acquired road image into the backbone network to obtain a feature map; extracting features of the pre-selected lane line in the feature map according to the position of the pre-selected lane line; mapping the features of the preselected lane line into a first feature vector, and processing the first feature vector through a full connection layer to obtain a second feature vector of the lane line; establishing a relation between different lane lines through a self-attention mechanism, and obtaining a third feature of the whole preselected lane line relative to the current lane line; and obtaining a fourth feature according to the second feature and the third feature, and further obtaining the final positioning of each preselected lane line. According to the method, the problem of low accuracy of lane line detection under the condition of dark light or insufficient light can be solved, and the detection effect is further improved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving and expected functional safety, and in particular to a lane line detection method, system, vehicle and medium for expected functional safety. Background Art

[0002] For self-driving cars, in order to ensure the expected functional safety, lane line detection in all scenarios is the basic problem that must be solved first. Through lane line detection, the use of modern technologies such as adaptive cruise control, lane departure warning, and driving route planning to assist drivers is crucial to ensuring the expected functional safety of self-driving vehicles during driving. At the same time, whether the lane line can be correctly detected is also an important boundary condition for the operation of the L3 autonomous driving system. L3 autonomous driving is an autonomous driving mode that can execute some functional decisions under restricted conditions.

[0003] In the related technologies, lane line detection mostly uses a fusion of global image features and local lane line features. It first uses global features to roughly determine the position of the lane line, and then uses local features to refine the lane line. For some scenes, this method can detect and locate the lane line relatively well in simple scenes such as highways because the lane line is in good condition (good light, obvious lane lines, etc.). However, on ordinary roads in real life, lane lines often have complex situations such as unclear lane lines. The above detection method is difficult to meet the requirements of accurate detection and identification of lane lines, causing expected functional safety problems. Therefore, it is necessary to provide a detection method that can cope with various complex conditions existing on real roads, and can accurately detect lane lines under conditions such as extreme lighting and bad weather, such as night, snow, fog, rain and other poor lighting conditions, to reduce the expected functional safety risks.

[0004] The disclosure of the above background technology content is only used to assist in understanding the inventive concept and technical solution of the present invention. It does not necessarily belong to the prior art of this patent application, nor does it necessarily provide technical guidance. In the absence of clear evidence that the above content has been disclosed before the filing date of this patent application, the above background technology should not be used to evaluate the novelty and creativity of the present application. Summary of the invention

[0005] The purpose of the present invention is to provide a detection method for accurately detecting lane lines under insufficient light conditions with an eye on expected functional safety.

[0006] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0007] A lane line detection method for expected functional safety under insufficient light conditions includes the following steps:

[0008] Use 2D coordinate points to construct pre-selected lane lines;

[0009] Constructing a backbone network based on a residual structure, and inputting the collected road image into the backbone network to obtain a feature map;

[0010] Mapping the pre-selected lane line constructed using two-dimensional coordinate points onto the feature map, and pooling local feature information at corresponding positions in the feature map into a first feature vector for each pre-selected lane line;

[0011] The first feature vector is processed by the first process to obtain a second feature vector of the lane line;

[0012] Establishing connections between different lane lines through a self-attention mechanism to obtain a third feature of the entire pre-selected lane line relative to the current lane line;

[0013] A fourth feature is obtained based on the second feature and the third feature, thereby obtaining the final positioning of each pre-selected lane line.

[0014] Further, based on any one of the above technical solutions or a combination of multiple technical solutions, extracting the features of the pre-selected lane line in the feature map according to the position of the pre-selected lane line includes:

[0015] The first feature vector extracted according to the pre-selected lane line coordinates is: Where K is the number of pre-selected lane lines, and the mapping relationship is as follows:

[0016]

[0017] Among them, x j is the horizontal coordinate of the pre-selected lane line after mapping, y j =0,1,2,...,H-1, which represents the fixed vertical coordinate of the pre-selected lane line according to the height H of the feature map; θ is the angle between the pre-selected lane line and the x-axis in the constructed xy coordinate system, x start is the horizontal coordinate of the starting point of the pre-selected lane line, y start is the starting ordinate of the pre-selected lane line, and α is the global stride from the original image to the feature map.

[0018] Further, based on any one of the above-mentioned technical solutions or a combination of multiple technical solutions, the second feature vector of the lane line is obtained after the first feature vector is processed by the first process, including:

[0019] The first processing is to obtain a second feature vector of the lane line after the first feature vector is processed by a fully connected layer.

[0020] Further, based on any one of the above-mentioned technical solutions or a combination of multiple technical solutions, a connection between different lane lines is established through a self-attention mechanism to obtain a third feature of the entire pre-selected lane line relative to the current lane line, including:

[0021] Calculate the second feature through linear transformation to obtain query feature, value feature and key feature respectively;

[0022] The key features of other lane lines are obtained through the calculated query features, and then the second feature relationship between lane lines is established.

[0023] Further, based on any one of the above-mentioned technical solutions or a combination of multiple technical solutions, the connection between different lane lines is established through the self-attention mechanism to obtain the third feature of the entire pre-selected lane line relative to the current lane line, including:

[0024] The relative position relationship between the current lane line and other lane lines is established based on the horizontal coordinate of the starting point of the lane line, the length of the lane line, and the angle between the lane line and the coordinate axis;

[0025] The relative position relationship is encoded and embedded into a high-dimensional representation to obtain a feature dimension after the high-dimensional representation;

[0026] The relative position relationship feature is calculated according to the feature dimension, and then the total relationship weight is calculated according to the relative position relationship feature.

[0027] Further, based on any one of the above technical solutions or a combination of multiple technical solutions, the establishing of the connection between different lane lines through the self-attention mechanism to obtain the third feature of the entire pre-selected lane line relative to the current lane line also includes:

[0028] The third feature of the entire pre-selected lane line set relative to the current lane line is calculated based on the value feature and the total relationship weight.

[0029] Further, based on any one of the above technical solutions or a combination of multiple technical solutions, the fourth feature is obtained according to the second feature and the third feature to obtain the final positioning of each pre-selected lane line, and further includes:

[0030] The second feature is aggregated with the third feature to obtain a fourth feature, and a predicted lane line is output for each pre-selected lane line through a fully connected layer;

[0031] The output includes classification prediction and regression prediction. The classification prediction includes the probability that the predicted lane line is the real lane line, and the regression prediction is the offset between the pre-selected lane line and the predicted lane line in the horizontal distance.

[0032] Furthermore, any one of the above-mentioned technical solutions or a combination of multiple technical solutions further includes:

[0033] Use non-maximum suppression to remove invalid and repeated predicted lane lines.

[0034] Furthermore, any one of the above-mentioned technical solutions or a combination of multiple technical solutions further includes:

[0035] The second feature is an appearance feature, the third feature is a relationship feature, and the fourth feature is an aggregation feature.

[0036] Furthermore, any one of the above-mentioned technical solutions or a combination of multiple technical solutions further includes:

[0037] The loss function is calculated based on the focal loss between each predicted lane line and the real lane line and the Smooth L1 loss of the distance between the predicted lane line and the real lane line.

[0038] According to another aspect of the present invention, the present invention provides a lane line detection system under insufficient light conditions, comprising:

[0039] A lane line construction module is used to construct pre-selected lane lines using two-dimensional coordinate points;

[0040] A feature map generation module is used to construct a backbone network based on a residual structure, and input the collected road image into the backbone network to obtain a feature map;

[0041] A pre-selected lane line feature extraction module, used to extract the features of the pre-selected lane line in the feature map according to the position of the pre-selected lane line;

[0042] A second feature acquisition module, used for mapping the feature of the pre-selected lane line into a first feature vector, wherein the first feature vector is subjected to a first processing to obtain a second feature vector of the lane line;

[0043] A third feature acquisition module, used to establish a connection between different lane lines through a self-attention mechanism, and obtain a third feature of the entire pre-selected lane line relative to the current lane line;

[0044] The decision module is used to obtain a fourth feature based on the second feature and the third feature, and further obtain the final positioning of each pre-selected lane line.

[0045] According to another aspect of the present invention, the present invention provides a lane line detection system under insufficient light conditions for expected functional safety, including:

[0046] A memory storing executable program code;

[0047] a central processing unit coupled to the memory;

[0048] The central processing unit calls the executable program code stored in the memory to execute the aforementioned lane line detection method under insufficient light conditions for expected functional safety.

[0049] According to another aspect of the present invention, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the aforementioned lane line detection method under insufficient light conditions for expected functional safety.

[0050] According to another aspect of the present invention, the present invention provides a vehicle, the vehicle comprising the aforementioned lane line detection system for expected functional safety under insufficient light conditions.

[0051] The beneficial effects brought by the technical solution provided by the present invention are as follows:

[0052] a. The present invention provides a lane line detection method under low light conditions for expected functional safety. The self-attention mechanism is used to establish the relative relationship between lane lines, thereby improving the effect of lane line detection under low light conditions.

[0053] b. The present invention provides a lane line detection method under low-light conditions for expected functional safety. The image collected by the sensor is passed through the backbone network to obtain a feature map, and the features of the pre-selected lane lines are extracted from the feature map according to the positions of the pre-selected lane lines. Then, the lane line-to-lane line relative relationship module is used to enable each lane line to obtain the appearance feature relationship and relative position relationship with other lane lines. Finally, the final positioning of each pre-selected lane line is obtained and non-maximum suppression is used to remove invalid and repeated lane lines, so that the lane line detection effect under low-light conditions is further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0055] Figure 1 A schematic diagram of a lane line detection method under insufficient light conditions for expected functional safety provided by an exemplary embodiment of the present invention;

[0056] Figure 2 A model framework diagram of a lane line detection method under insufficient light conditions for expected functional safety provided by an exemplary embodiment of the present invention;

[0057] Figure 3 A relationship feature acquisition flow chart provided for an exemplary embodiment of the present invention;

[0058] Figure 4 A schematic diagram of a lane line detection system under insufficient light conditions for expected functional safety provided by an exemplary embodiment of the present invention;

[0059] Figure 5 A schematic diagram of a vehicle is provided for an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0060] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0061] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0062] In one embodiment of the present invention, Figure 1 The present invention provides a lane line detection method in insufficient light conditions for expected functional safety, including:

[0063] S1: Use 2D coordinate points to construct pre-selected lane lines;

[0064] S2: construct a backbone network based on a residual structure, and input the collected road image into the backbone network to obtain a feature map;

[0065] S3: Mapping the pre-selected lane line constructed using two-dimensional coordinate points onto the feature map, and pooling the local feature information at the corresponding position in the feature map into a first feature vector for each pre-selected lane line;

[0066] S4: obtaining a second feature vector of the lane line after first processing the first feature vector;

[0067] S5: Establishing connections between different lane lines through a self-attention mechanism to obtain a third feature of the entire pre-selected lane line relative to the current lane line;

[0068] S6: Obtain a fourth feature based on the second feature and the third feature, and then obtain the final positioning of each pre-selected lane line.

[0069] It should be noted that the road image can be collected by a camera located at any position in the vehicle. For example, when the vehicle is traveling in a straight line, the road image is collected by a camera located in front of the vehicle. When the vehicle turns, the road image is collected by a camera located on the left or right side of the vehicle. The present invention does not specifically limit this, as long as the collected road image contains lane lines. It is also possible to shoot a road video by a camera device on the vehicle while the vehicle is traveling, and then collect the road image corresponding to the current frame of the captured video. The collected road image reflects the road conditions of the vehicle's environment, including at least one lane line. It should be noted that the lane lines in the embodiments of the present invention include the vehicle's driving lane lines, such as various indicator lines on the road surface, and also include lane boundary lines, such as the curb of the road.

[0070] In one embodiment of the present invention, the model framework diagram of the method is as follows Figure 2 As shown in the figure, the backbone network generates a feature map from the input original image, and then the pre-selected lane line is projected onto the feature map to pool the pre-selected lane line features, and then a fully connected layer is used to obtain the appearance feature vector of the pre-selected lane line. Next, the relative relationship module between different lane lines is used to establish the appearance relationship features and relative position relationship features between other lane lines and the current lane line, and the overall relationship features are fused with the appearance feature vector of the current lane line. Finally, a fully connected layer is used to obtain the output result of the model.

[0071] In one embodiment of the present invention, S1: constructing a pre-selected lane line using two-dimensional coordinate points, which can be exemplarily expressed as:

[0072] Use the two-dimensional coordinate points to construct K pre-selected lane lines L = {(x1, y1), ···, (x N ,y N )},

[0073] Among them, the vertical coordinate of the pre-selected lane line is composed of equidistant points. The horizontal coordinate x of the pre-selected lane line i With the vertical coordinate y i The starting point coordinates O = (x start ,y start ), the angle with the x-axis is θ, the lane length is l, H orig is the original image height.

[0074] In one embodiment of the present invention, S2: constructing a backbone network based on a residual structure, inputting the collected road image into the backbone network to obtain a feature map, which can be exemplarily expressed as:

[0075] X=backbone(X orig )

[0076] Among them, the original image W orig is the original image width, and the obtained feature map H and W are the height and width of the downsampled feature map, and C is the number of channels.

[0077] In one embodiment of the present invention, S3: extracting the features of the pre-selected lane line in the feature map according to the position of the pre-selected lane line, including extracting the features of the pre-selected lane line in the feature map according to the position of the pre-selected lane line, which can be exemplarily expressed as:

[0078] The pre-selected lane line constructed in step S1 is mapped to the feature map, and the local feature information of the corresponding position in the feature map is pooled into the first feature vector of each pre-selected lane line. Where K is the number of pre-selected lane lines, and the mapping relationship is as follows:

[0079]

[0080] Among them, x j is the horizontal coordinate of the pre-selected lane line after mapping, y j =0,1,2,...,H-1, which represents the fixed vertical coordinate of the pre-selected lane line according to the height H of the feature map; θ is the angle between the pre-selected lane line and the x-axis in the constructed xy coordinate system, x start is the horizontal coordinate of the starting point of the pre-selected lane line, y start is the starting ordinate of the pre-selected lane line, and α is the global stride from the original image to the feature map.

[0081] In one embodiment of the present invention, S4: the first feature vector is processed by the first process to obtain a second feature vector of the lane line, which can be exemplarily expressed as:

[0082] The second characteristic is an appearance characteristic;

[0083] The first feature vector of the pre-selected lane line is passed through a fully connected layer to obtain the appearance feature vector of the lane line. That is, the second eigenvector:

[0084]

[0085] Among them, FC(·) represents the fully connected layer, and n represents the sequence number of the current pre-selected lane line.

[0086] In one embodiment of the present invention, Figure 3 As shown, S5: establish connections between different lane lines through a self-attention mechanism to obtain a third feature of the entire pre-selected lane line relative to the current lane line, wherein the third feature is a relationship feature: exemplarily including:

[0087] S51: The appearance features (Second eigenvector) Calculate query features by linear transformation Value Features and key features

[0088]

[0089] Where W q , W v and W k are learnable parameters.

[0090] S52: Use the query feature calculated in step S51 to obtain key features of other lane lines to establish appearance feature relationships between lane lines

[0091]

[0092] Among them, d k is the feature dimension after linear transformation of the appearance feature, m is the serial number of other lane lines different from the current pre-selected lane line, is the key feature of the mth lane line.

[0093] S53: Use the starting point horizontal coordinate of the lane line, the length of the lane line, and the angle between the lane line and the x-axis to establish the relative position relationship between the current lane line and other lane lines as

[0094]

[0095] is the position information of the current lane line obtained in step 1, including the horizontal and vertical coordinates of the starting point of the pre-selected lane line (x start ,y start ), lane length (l) and lane angle (θ), is the position information of other lane lines obtained in step 1.

[0096] S54: Use position encoding to embed the relative position relationship between lane lines into a high-dimensional representation:

[0097]

[0098] where i is the dimension, d pis the feature dimension after being embedded in the high-dimensional representation.

[0099] S55: Calculate relative position features

[0100]

[0101] Where W p Because the position relationship parameters are learnable, the ReLU activation function is used to introduce nonlinear factors, PE β For PE (β,2i) and PE (β,2i+1) Alternate storage.

[0102] S56: Calculate the total relationship weight

[0103]

[0104] Where k is the kth pre-selected lane line, which is the same as Calculated by Same as Calculated by

[0105] S57: The relationship feature of the entire pre-selected lane line set relative to the current lane line is calculated using the value feature calculated in S51 and the total relationship weight calculated in S56, that is, the third feature:

[0106]

[0107] In one embodiment of the present invention, S6: obtaining a fourth feature based on the second feature and the third feature, and then obtaining the final positioning of each pre-selected lane line, exemplarily includes: S61: aggregating the relationship feature and the appearance feature to obtain the fourth feature:

[0108]

[0109] S62: Use non-maximum suppression to remove invalid and repeated predicted lane lines;

[0110] The output includes classification prediction and regression prediction. The classification prediction includes the probability that the predicted lane line is the real lane line, and the regression prediction is the offset between the pre-selected lane line and the predicted lane line in the horizontal distance.

[0111] By setting the probability threshold and the offset threshold, the lane line is calculated in combination with the output probability and offset. For example, the probability is set to A%, and the offset threshold is set to B. When the output probability is greater than A% and the offset does not exceed B, the final positioning of the pre-selected lane line is determined.

[0112] In one embodiment of the present invention, the method further includes: S7: based on the focus loss between each predicted lane line and the real lane line and the Smooth L1 loss of the distance between the predicted lane line and the real lane line, calculating a loss function to verify whether the lane line is accurately predicted, which is exemplarily expressed as:

[0113] L total =λL cls +L reg

[0114] Where L cls is the focal loss between each predicted lane line and the real lane line, L reg is the Smooth L1 loss of the distance between the predicted lane line and the actual lane line, and λ is the hyperparameter of the balance loss. total If the value exceeds the preset threshold, the prediction is considered to have failed, otherwise it is considered to have passed.

[0115] In summary, the embodiments of the present invention provide a lane line detection method under insufficient light conditions for expected functional safety. The method adopts pre-set pre-selected lane lines, obtains a feature map through a backbone network through the image collected by the sensor, extracts the features of the pre-selected lane lines in the feature map according to the positions of the pre-selected lane lines, and then uses a relative relationship module between lane lines to enable each lane line to obtain an appearance feature relationship and a relative position relationship with other lane lines. Finally, the final positioning of each pre-selected lane line is obtained and non-maximum suppression is used to remove invalid and repeated lane lines. Compared with the prior art method of using the fusion of image global features and lane line local features, the method of using global features to roughly determine the position of the lane line and then using local features to refine the lane line can overcome the problem that under insufficient light conditions, the image quality is reduced, making the global features and local features not obvious, resulting in a decrease in detection effect, and improves the accuracy of lane line detection.

[0116] In one embodiment of the present invention, Figure 4 The present invention provides a lane marking system for expected functional safety in low light conditions, including:

[0117] A lane line construction module is used to construct pre-selected lane lines using two-dimensional coordinate points;

[0118] A feature map generation module is used to construct a backbone network based on a residual structure, and input the collected road image into the backbone network to obtain a feature map;

[0119] A pre-selected lane line feature extraction module, used to extract the features of the pre-selected lane line in the feature map according to the position of the pre-selected lane line;

[0120] A second feature acquisition module, used for mapping the feature of the pre-selected lane line into a first feature vector, wherein the first feature vector is subjected to a first processing to obtain a second feature vector of the lane line;

[0121] A third feature acquisition module, used to establish a connection between different lane lines through a self-attention mechanism, and obtain a third feature of the entire pre-selected lane line relative to the current lane line;

[0122] The decision module is used to obtain a fourth feature based on the second feature and the third feature, and further obtain the final positioning of each pre-selected lane line.

[0123] In summary, the embodiments of the present invention provide a lane line detection system for expected functional safety under low-light conditions. The system adopts pre-set pre-selected lane lines, obtains a feature map through a backbone network from the image collected by the sensor, extracts the features of the pre-selected lane lines in the feature map according to the positions of the pre-selected lane lines, and then uses a relative relationship module between lane lines to enable each lane line to obtain an appearance feature relationship and a relative position relationship with other lane lines. Finally, the final positioning of each pre-selected lane line is obtained and non-maximum suppression is used to remove invalid and repeated lane lines. Compared with the prior art method of fusing image global features with lane line local features, the system uses global features to roughly determine the position of the lane line and then uses local features to refine the lane line. This can overcome the problem that in low-light conditions, the image quality is reduced, making both global features and local features not obvious, resulting in a reduction in detection effect, and improves the accuracy of lane line detection.

[0124] The lane line detection system under insufficient light conditions provided in this embodiment and the lane line detection method under insufficient light conditions provided in the above-mentioned embodiment belong to the same concept. Here, by quoting the full text, the entire content of the lane line detection method embodiment under insufficient light conditions is introduced into the lane line detection system embodiment under insufficient light conditions of this network intrusion detection device.

[0125] In one embodiment of the present invention, the present invention also provides a lane line detection system under insufficient light conditions for expected functional safety, including: a memory storing executable program code; a central processing unit coupled to the memory; the central processing unit calls the executable program code stored in the memory to execute the above-mentioned lane line detection method under insufficient light conditions. The idea of ​​this detection system embodiment and the working process of a lane line detection method under insufficient light conditions in the above-mentioned embodiment are the same idea. The entire content of the above-mentioned lane line detection method embodiment under insufficient light conditions is incorporated into this lane line detection system embodiment under insufficient light conditions by quoting the full text, and no further description is given.

[0126] In one embodiment of the present invention, the present invention also provides a vehicle, such as Figure 5 As shown, it includes a processor 501, a memory 502, and a computer program 5021 stored in the memory and executable on the processor. When the processor 501 executes the computer program, the aforementioned lane line detection method under insufficient light conditions is implemented.

[0127] In one embodiment of the present invention, the embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned detection method is implemented.

[0128] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0129] The above is only a specific implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A lane line detection method for expected functional safety, characterized in that: The following steps are involved: Use 2D coordinate points to construct pre-selected lane lines; Constructing a backbone network based on a residual structure, and inputting the collected road image into the backbone network to obtain a feature map; Mapping the pre-selected lane line constructed using two-dimensional coordinate points onto the feature map, and pooling local feature information at corresponding positions in the feature map into a first feature vector for each pre-selected lane line; The first feature vector is processed by the first process to obtain a second feature vector of the lane line; Establishing connections between different lane lines through a self-attention mechanism to obtain a third feature of the entire pre-selected lane line relative to the current lane line; A fourth feature is obtained based on the second feature and the third feature, thereby obtaining the final positioning of each pre-selected lane line.

2. The lane line detection method under insufficient light conditions according to claim 1, characterized in that: Extracting the features of the pre-selected lane line in the feature map according to the position of the pre-selected lane line includes: The first feature vector extracted according to the pre-selected lane line coordinates is: Where K is the number of pre-selected lane lines, and the mapping relationship between pre-selected lane lines is as follows: Among them, x j is the horizontal coordinate of the pre-selected lane line after mapping, y j =0,1,2,...,H-1, which represents the fixed vertical coordinate of the pre-selected lane line according to the height H of the feature map; θ is the angle between the pre-selected lane line and the x-axis in the constructed xy coordinate system, x start is the horizontal coordinate of the starting point of the pre-selected lane line, y start is the starting ordinate of the pre-selected lane line, and α is the global stride from the original image to the feature map.

3. The lane line detection method under insufficient light conditions according to claim 1, characterized in that: The second feature vector of the lane line is obtained after the first feature vector is processed by the first process, including: The first processing is to obtain a second feature vector of the lane line after the first feature vector is processed by a fully connected layer.

4. The lane line detection method under insufficient light conditions according to claim 1, characterized in that: The connection between different lane lines is established through the self-attention mechanism to obtain the third feature of the entire pre-selected lane line relative to the current lane line, including: Calculate the second feature through linear transformation to obtain query feature, value feature and key feature respectively; The key features of other lane lines are obtained through the calculated query features, and then a second feature relationship between lane lines is established.

5. The lane line detection method under insufficient light conditions according to claim 4, characterized in that: The method of establishing a connection between different lane lines by a self-attention mechanism to obtain a third feature of the entire pre-selected lane line relative to the current lane line also includes: The relative position relationship between the current lane line and other lane lines is established based on the horizontal coordinate of the starting point of the lane line, the length of the lane line, and the angle between the lane line and the coordinate axis; The relative position relationship is encoded and embedded into a high-dimensional representation to obtain a feature dimension after the high-dimensional representation; The relative position relationship feature is calculated according to the feature dimension, and then the total relationship weight is calculated according to the relative position relationship feature.

6. The lane line detection method under insufficient light conditions according to claim 5, characterized in that: The method of establishing a connection between different lane lines by a self-attention mechanism and obtaining a third feature of the entire pre-selected lane line relative to the current lane line also includes: The third feature of the entire pre-selected lane line set relative to the current lane line is calculated based on the value feature and the total relationship weight.

7. The lane line detection method under insufficient light conditions according to claim 4, characterized in that: The obtaining of the fourth feature according to the second feature and the third feature, and then obtaining the final positioning of each pre-selected lane line, includes: The second feature is aggregated with the third feature to obtain a fourth feature, and a predicted lane line is output for each pre-selected lane line through a fully connected layer; The output includes classification prediction and regression prediction. The classification prediction includes the probability that the predicted lane line is the real lane line, and the regression prediction is the offset between the pre-selected lane line and the predicted lane line in the horizontal distance.

8. The lane line detection method under insufficient light conditions according to claim 7, characterized in that: Before outputting the predicted lane line for each pre-selected lane line, it also includes: Use non-maximum suppression to remove invalid and repeated predicted lane lines.

9. The lane line detection method under insufficient light conditions according to any one of claims 1 to 8, characterized in that: The second feature is an appearance feature, the third feature is a relationship feature, and the fourth feature is an aggregation feature.

10. The lane line detection method under insufficient light conditions according to any one of claims 1 to 8, characterized in that: Also includes: The loss function is calculated based on the focal loss between each predicted lane line and the real lane line and the Smooth L1 loss of the distance between the predicted lane line and the real lane line.

11. A lane line detection system under insufficient light conditions, characterized in that: include: A lane line construction module is used to construct pre-selected lane lines using two-dimensional coordinate points; A feature map generation module is used to construct a backbone network based on a residual structure, and input the collected road image into the backbone network to obtain a feature map; A pre-selected lane line feature extraction module, used to extract the features of the pre-selected lane line in the feature map according to the position of the pre-selected lane line; A second feature acquisition module, used for mapping the feature of the pre-selected lane line into a first feature vector, wherein the first feature vector is subjected to a first processing to obtain a second feature vector of the lane line; A third feature acquisition module, used to establish a connection between different lane lines through a self-attention mechanism, and obtain a third feature of the entire pre-selected lane line relative to the current lane line; The decision module is used to obtain a fourth feature based on the second feature and the third feature, and further obtain the final positioning of each pre-selected lane line.

12. A lane detection system for expected functional safety, characterized in that: Includes the following modules: A memory storing executable program code; a central processing unit coupled to the memory; The central processing unit calls the executable program code stored in the memory to execute the lane line detection method under insufficient light conditions as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program enables a computer to execute the lane line detection method under insufficient light conditions according to any one of claims 1 to 10.

14. A vehicle, characterized in that: comprising a processor, a memory, and a computer program stored in the memory and executable on the processor; Wherein, when the processor executes the computer program, the lane line detection method under insufficient light conditions as described in any one of claims 1 to 10 is implemented.

Images