A lane line and lane area detection method based on a road scene

Abstract

The application relates to a lane line and lane area detection method based on a road scene, which processes a vehicle front-view RGB image into a high-resolution image and a low-resolution image, inputs the high-resolution image and the low-resolution image into a detection network which is constructed and trained in advance, and obtains a lane line and lane detection result. The detection network comprises a lane line primary feature extraction module which extracts primary features of lane lines, a lane primary feature extraction module which extracts primary features of lanes, a feature interaction learning module which performs information interaction on the two primary features to generate interactive features, a lane line decoder and a lane decoder which respectively perform first decoding on the interactive features, input the features after the first decoding into the opposite decoder, and perform second decoding in combination with the interactive features, and an interaction constraint loss module which performs constraint on the decoding features after the second decoding of the two decoders. Compared with the prior art, the application can simultaneously and accurately detect lane lines and lanes.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a method for detecting lane lines and lane areas based on road scenarios. Background Technology

[0002] The autonomous driving perception system mainly consists of two parts: onboard sensor hardware and perception algorithm software. Onboard sensors primarily include cameras, LiDAR, millimeter-wave radar, and integrated positioning systems. The perception algorithm transforms the environmental data measured by the onboard sensors into semantic information, enabling the perception and understanding of the environment. The main tasks of the perception system include the detection and tracking of dynamic and static obstacles, road marking detection, localization and map building, and traffic sign and signal detection. Based on the semantic information obtained from the perception system, the downstream planning and control system of the autonomous vehicle can then use decision-making algorithms to perform the next step of path planning and control. The tasks of detecting dynamic and static obstacles, road markings, traffic signs, and signals in the perception process widely utilize object detection and semantic segmentation technologies.

[0003] The road marking task of an autonomous driving perception system mainly consists of the detection and segmentation of lane lines, road markings (turn arrows, stop lines, zebra crossings, speed reduction markings, no-stopping zones, and guide zones), and lanes (or drivable areas). These detected targets are also essential semantic elements guiding the driving of autonomous vehicles. Whether for advanced autonomous driving systems or general ADAS systems, high-precision, real-time lane and lane line perception systems are crucial for vehicles to make reasonable decision-making and control schemes during driving. At the same time, a good perception system can help reduce driver fatigue, provide early warnings before danger arrives, and avoid traffic accidents.

[0004] The lane and lane line detection tasks in autonomous driving perception systems primarily involve cameras and LiDAR. Among these, onboard cameras, with their low cost and rich information content, have become the most widely used sensors in autonomous driving. Specifically, lane line detection and lane segmentation are both performed based on images acquired by onboard cameras. Perception algorithms can extract lane line and lane features from the image information, estimate the accurate location of lane lines and lanes using this feature information, and convert it into the information needed by the autonomous driving planning and control system. This ensures the vehicle's safe driving, steering, and lane changing, thereby improving the safety and reliability of autonomous vehicles.

[0005] However, existing lane segmentation methods are almost non-existent, and datasets specifically designed for lane segmentation are extremely rare. Although there are many existing lane line detection methods, almost none of them consider lane information or factors that need to be applied to actual driving scene perception tasks. In real-world driving scene perception tasks, simply treating lane line detection as an independent task is not conducive to the perception model's overall understanding of the scene. Especially in complex road scenarios (such as missing lane lines, lane line occlusion, etc.), robustness of lane line detection alone is insufficient. Therefore, it is necessary to detect both lanes and lane lines. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art by providing a lane line and lane area detection method based on road scenarios. This invention can simultaneously and accurately detect lane lines and lanes.

[0007] The objective of this invention can be achieved through the following technical solutions:

[0008] This invention provides a method for detecting lane lines and lane regions based on road scenes. The method processes a vehicle's forward-looking RGB image containing a real road scene into high-resolution and low-resolution images with different resolutions. Both the high-resolution and low-resolution images are input into a pre-constructed and trained detection network to obtain the detection results of lane lines and lanes.

[0009] The detection network includes:

[0010] Lane line primary feature extraction module: used to extract primary features of lane lines, with high-resolution images as input;

[0011] Lane primary feature extraction module: used to extract primary features of lanes, with low-resolution image as input;

[0012] Feature Interaction Learning Module: Used to exchange information between the primary features of lane lines and the primary features of lanes to generate interactive features;

[0013] Lane line decoder: Used to perform the first decoding of the interaction features. The features after the first decoding are input into the lane decoder. The lane decoder receives the decoded features from the first decoding and combines them with the interaction features for the second decoding.

[0014] Lane decoder: Used to perform the first decoding of the interaction features. The features after the first decoding are input into the lane decoder. The lane decoder receives the decoded features from the first decoding of the lane decoder and combines them with the interaction features for the second decoding.

[0015] Interactive constraint loss module: used to constrain the decoding features of the lane line decoder and the second decoding of the lane decoder.

[0016] Preferably, both the lane line primary feature extraction module and the lane primary feature extraction module include a convolutional neural network, a context information extraction unit, and an auxiliary loss branch. The convolutional neural network is used to extract the initial primary features of the lane line / lane, and the context information extraction unit and the auxiliary loss branch are used to optimize the initial primary features of the lane line / lane to generate the primary features of the lane line / lane.

[0017] Preferably, the auxiliary loss branch includes multiple convolutional layers. The initial primary features of the lane lines / lanes are processed by multiple convolutional layers to obtain the preliminary segmentation results of the lane lines / lanes, and the cross-entropy loss is calculated in combination with the ground truth labels to supervise the convolutional neural network.

[0018] The context information extraction unit includes an attention calculation preprocessing branch and an attention calculation branch. The attention calculation preprocessing branch is used to calculate the weights of the initial segmentation results obtained by the auxiliary loss branch in its own spatial dimension direction through a normalized exponential function, and combine them with the initial primary features to obtain context information. The context information is then converted into key vectors and value vectors through a convolutional layer, and the initial primary features are converted into query vectors through another convolutional layer. The attention calculation branch takes the key vectors, value vectors and query vectors as inputs, and concatenates the output features with the initial primary features as the output, which is the primary feature.

[0019] Preferably, the feature interaction learning module includes a concatenation layer, a linear preprocessing layer, a global average pooling layer, and multiple linear operation layers connected in sequence;

[0020] The primary features of lane lines and lanes are concatenated by a stitching layer, and then preprocessed by a linear preprocessing layer to obtain the first fused feature F0. The first fused feature F0 is then subjected to nonlinear activation and global average pooling by a global average pooling layer to obtain the weighted feature F. gp Weighted features F gp After processing through two linear operation layers, the system is divided into a lane line feature branch and a lane feature branch, with their respective weights α1 and α2. α1 and α2 are then processed using a normalized exponential function to obtain the relative weights α′1 and α′2 of the two branches. After multiplying α′1 and α′2 by the features concatenated by the splicing layer, the results are summed to obtain the second fusion feature F1. This results in the attention-weighted fusion of the two branches. Finally, the second fusion feature F1 is processed through two linear operation layers to restore it to the same dimension as the first fusion feature F0, resulting in the third fusion feature F2. The residuals of the third fusion feature F2 and the first fusion feature F0 are then added to obtain the final fusion feature.

[0021] Preferably, the expression for the calculation process of the feature interaction learning module is as follows:

[0022] F0 = Linear(Concat(F) Lane F drivable ))

[0023] F gp =AvgPool(GELU(F0))

[0024] F = GELU(Linear(F) gp ))

[0025] W Lane W drivable =Softmax(Concat(Linear(F),Linear(F)))

[0026] F1 = W Lane ·F Lane +W drivable ·F drivable

[0027] F2=Dropout(Linear(Linear(LayerNorm(F1))))

[0028] Out=Linear(Dropout(GELU(LayerNorm(F2+F0))))

[0029] In the formula, F0 is the first fusion feature, F Lane As a primary feature of lane markings, F drivable For the primary features of the lane, Concat(*), represents the concatenation operation, Linear(*), represents the linear operation, and F... gp Here, F represents the weighted features, AvgPool(*) is the average pooling operation, F is the intermediate weight generated by the pooling features, and W is the weighted feature. Lane and W drivable F1 represents the weights of the lane line feature branch and the lane feature branch, respectively. F2 represents the second fusion feature, F1 represents the third fusion feature, and Out represents the final fusion feature.

[0030] Preferably, the lane line decoder includes a coarse upsampling branch and a fine upsampling branch. The coarse upsampling branch obtains a coarse upsampling output result through convolution combined with bilinear interpolation upsampling. The fine upsampling branch obtains a fine upsampling output result through deconvolution layers and non-bottleneck layers. The coarse upsampling output result and the fine upsampling output result are directly added together to obtain the lane line decoding features.

[0031] Preferably, the features decoded by the lane line decoder in the first decoding stage are input into the lane decoder through the decoding feature interaction module, and the features decoded by the lane decoder in the first decoding stage are input into the lane line decoder. The decoding feature interaction module includes a convolutional layer and a residual structure. The features decoded by the two decoders in the first decoding stage are respectively passed through the convolutional layer and the residual structure, so as to extract auxiliary information from the output of the two decoders to the other decoder. The auxiliary information is directly added to the original input of the two decoders to obtain fused information, which is used as the new input of the decoder for the second decoding to obtain the prediction result after the information interaction between the two branches in the decoding stage.

[0032] Preferably, the interaction constraint loss module includes:

[0033] Lane loss function: used to constrain the output of the lane line decoder;

[0034] Lane loss function: used to constrain the output of the lane decoder;

[0035] Interactive loss function: used to constrain the outputs of the lane line decoder and the lane decoder, thereby constraining the spatial geometric relationship between the two outputs.

[0036] Preferably, the interaction loss function is the intersection-over-union ratio (IoU) loss of lane lines and lane interactions, where IoU is the intersection-over-union ratio loss. iou The specific calculation expression is as follows:

[0037]

[0038] Among them, L pred and L gt D represents the predicted and ground truth pixel regions of the lane lines, respectively. pred and D gt These represent the predicted and ground truth pixel regions of the lane, respectively.

[0039] Preferably, the overall loss function of the detection network is calculated as follows:

[0040] Loss = w aux L aux +w iou L iou +w ce L ce +w rmi L rmi

[0041] Where L aux L represents the auxiliary loss. iou To compare the loss between intersection and union, L ce and L rmi These are the lane line loss function and the lane loss function, respectively.aux、 w iou w ce、 w rmi The weights represent the auxiliary loss, intersection-to-merge ratio loss, lane line loss, and lane loss, respectively.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] This invention provides a lane line and lane region detection method based on road scenes. Taking lane line and lane region detection—two fundamental perceptual information elements in road scenes—as its starting point, it proposes a novel interactive learning detection network. Within a multi-task framework, it introduces multi-scale input, interactive learning in the feature extraction stage, interactive learning in the decoding stage, and an interactive loss function incorporating lane line and lane geometric constraints. The proposed method can improve the detection accuracy of both lane lines and lanes while simultaneously achieving simultaneous detection. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of the detection network structure for a lane line and lane area detection method based on a road scene provided in this embodiment. Detailed Implementation

[0045] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0046] refer to Figure 1 As shown, this embodiment provides a lane line and lane area detection method based on road scenes. The method processes a vehicle forward-looking RGB image containing a real road scene into high-resolution images and low-resolution images with different resolutions. Both high-resolution and low-resolution images are input into a pre-constructed and trained detection network to obtain the detection results of lane lines and lanes.

[0047] The detection network includes:

[0048] Lane line primary feature extraction module: used to extract primary features of lane lines, with high-resolution images as input;

[0049] Lane primary feature extraction module: used to extract primary features of lanes, with low-resolution image as input;

[0050] Feature Interaction Learning Module: Used to exchange information between the primary features of lane lines and the primary features of lanes to generate interactive features;

[0051] Lane line decoder: Used to perform the first decoding of the interaction features. The features after the first decoding are input into the lane decoder. The lane decoder receives the decoded features from the first decoding and combines them with the interaction features for the second decoding.

[0052] Lane decoder: Used to perform the first decoding of the interaction features. The features after the first decoding are input into the lane decoder. The lane decoder receives the decoded features from the first decoding of the lane decoder and combines them with the interaction features for the second decoding.

[0053] Interactive constraint loss module: used to constrain the decoding features of the lane line decoder and the second decoding of the lane decoder.

[0054] As an optional implementation, both the lane line primary feature extraction module and the lane primary feature extraction module include a convolutional neural network, a context information extraction unit, and an auxiliary loss branch. A high-resolution image is input into the convolutional neural network of the lane line primary feature extraction module, which outputs initial primary lane line features. These initial primary lane line features are then input into the context information extraction unit of the lane line primary feature extraction module, and the auxiliary loss branch of the lane line primary feature extraction module is used to optimize the initial lane line features, resulting in the lane line primary features. Similarly, a low-resolution image is input into the convolutional neural network of the lane primary feature extraction module, which outputs initial primary lane features. These initial primary lane features are then input into the context information extraction unit of the lane primary feature extraction module, and the auxiliary loss branch of the lane primary feature extraction module is used to optimize the initial lane features, resulting in the lane primary features.

[0055] Before inputting the image into the convolutional neural network, different resolutions are obtained by scaling the original image size by 1x and 0.5x respectively. The convolutional neural network is compatible with inputs of different image sizes, and by using the same image with different resolutions as inputs, the model can focus on object information at different scales. Considering that high-resolution images are beneficial for detecting small-scale objects in lane lines, while reducing the resolution is beneficial for the network to extract large-scale object information in lane lines, this invention uses high-resolution images as input to the primary feature extraction module for lane lines, while using low-resolution images as input to the primary feature extraction module for lane lines.

[0056] As an optional implementation, the convolutional neural network is a backbone feature extraction network based on a modified ResNet. Considering that the input size of the training data for lane lines and lane detection is not large, and that the first five layers of ResNet contain 16x downsampling, this embodiment replaces the ordinary convolutional layers in layers C4 and C5 of ResNet with dilated convolutional layers with dilation rates of 2 and 4, thereby increasing the receptive field size of the convolutional layers without reducing the resolution.

[0057] As an optional implementation, the auxiliary loss branch includes multiple convolutional layers. In this embodiment, the auxiliary loss branch includes two convolutional layers. The initial primary features are processed by the two convolutional layers to obtain the preliminary segmentation results of lane lines or lanes, and the cross-entropy loss is calculated in combination with the ground truth labels to supervise the convolutional neural network.

[0058] As an optional implementation, the context information extraction unit is a context information extraction unit based on regions of the same category, including an attention calculation preprocessing branch and an attention calculation branch. The attention calculation preprocessing branch is used to calculate the weights of the initial segmentation results obtained by the auxiliary loss branch in its own spatial dimension direction through a normalized exponential function, and combine them with the initial primary features to obtain context information. The context information is then transformed into key vectors and value vectors required for subsequent attention calculation through two convolutional layers, and then the initial primary features are transformed into query vectors required for attention calculation through another convolutional layer. The attention calculation branch takes the key vectors, value vectors, and query vectors as inputs, and concatenates the output features with the initial primary features as the output, which is the primary feature.

[0059] As an optional implementation, the feature interaction learning module includes a concatenation layer, a linear preprocessing layer, a global average pooling layer, and multiple linear operation layers connected in sequence. The primary features of the lane lines and the lanes are concatenated by the concatenation layer and then preprocessed by the dimension-invariant linear preprocessing layer to obtain a first fused feature F0. The first fused feature F0 undergoes nonlinear activation and global average pooling by the global average pooling layer to obtain the weighted feature F. gp Weighted features F gp After processing through two linear operation layers, the system is divided into a lane line feature branch and a lane feature branch, with respective weights α1 and α2. α1 and α2 are then processed using a normalized exponential function to obtain the relative weights α′1 and α′2 of the two branches. α′1 and α′2 are then multiplied by the features concatenated by the concatenation layer, and the results are summed to obtain the second fusion feature F1. This results in the attention-weighted fusion of the two branches. This normalized exponential function processing obtains the mutual weights of the intermediate features of the lane lines and lane branches at each location in space, which can be understood as the importance of each pixel position on the feature map to the lane lines and lane features. Finally, the second fusion feature F1 is processed through two linear operation layers to restore it to the same dimension as the first fusion feature F0, resulting in the third fusion feature F2. The residuals of the third fusion feature F2 and the first fusion feature F0 are then added to obtain the final fusion feature.

[0060] Specifically, the expression for the calculation process of the feature interaction learning module is as follows:

[0061] F0 = Linear(Concat(F) Lane Fdrivable ))

[0062] F gp =AvgPool(GELU(F0))

[0063] F = GELU(Linear(F) gp ))

[0064] W Lane W drivable =Softmax(Concat(Linear(F),Linear(F)))

[0065] F1 = W Lane ·F Lane +W drivable ·F drivable

[0066] F2=Dropout(Linear(Linear(LayerNorm(F1))))

[0067] Out=Linear(Dropout(GELU(LayerNorm(F2+F0))))

[0068] In the formula, F0 is the first fusion feature, F Lane As a primary feature of lane markings, F drivable For the primary features of the lane, Concat(*), represents the concatenation operation, Linear(*), represents the linear operation, and F... gp Here, F represents the weighted features, AvgPool(*) is the average pooling operation, F is the intermediate weight generated by the pooling features, and W is the weighted feature. Lane and W driuable F1 represents the weights of the lane line feature branch and the lane feature branch, respectively. F2 represents the second fusion feature, F1 represents the third fusion feature, and Out represents the final fusion feature.

[0069] As an optional implementation, the lane line decoder is a dual-branch upsampling decoder, including a coarse upsampling branch and a fine upsampling branch. The coarse upsampling branch obtains the coarse upsampling output result through 1×1 convolution combined with bilinear interpolation upsampling. The fine upsampling branch obtains the fine upsampling output result through one deconvolution layer and two non-bottleneck layers. The coarse upsampling output result and the fine upsampling output result are directly added together to obtain the fusion of the two decoding features with different levels of fineness.

[0070] As an alternative implementation, the lane decoder is a simple upsampling decoder that obtains the output by combining two convolutional layers with direct bilinear interpolation upsampling.

[0071] As an optional implementation, the features decoded by the lane line decoder in the first decoding stage are input into the lane decoder, and vice versa, through a decoding feature interaction module. The decoding feature interaction module comprises two convolutional layers and a residual structure. While simple in structure, it leverages the spatial geometric relationship between lane lines and lanes to achieve information interaction during the decoding stage. Specifically, lane line and lane information are spatially adjacent. Convolutional layers effectively help each pixel capture information from adjacent positions. In this embodiment, the features decoded by the two decoders in the first decoding stage are passed through convolutional layers and a residual structure, respectively, thereby extracting auxiliary information from the outputs of the two decoders and feeding it into the other decoder. This auxiliary information is then directly added to the original inputs of the two decoders to obtain fused information, which is used as new input for the second decoding stage to obtain the prediction results of the two branches after information interaction during the decoding stage. This achieves the Coarseto Fine decoding output.

[0072] As an optional implementation, the interaction constraint loss module includes:

[0073] Lane loss function: used to constrain the output of the lane line decoder;

[0074] Lane loss function: used to constrain the output of the lane decoder;

[0075] Interactive loss function: used to constrain the outputs of the lane line decoder and the lane decoder, thereby constraining the spatial geometric relationship between the two outputs.

[0076] The lane line loss function is a weighted cross-entropy loss for pixel-by-pixel classification.

[0077] The lane loss function is a structured loss calculated on a region-by-region basis. It considers the correlation between adjacent pixels and the important structured prior information between pixels, using each pixel and its eight surrounding pixels as a whole to represent that pixel, effectively modeling the dependencies between pixels. This loss function transforms the predicted results into a high-dimensional distribution, utilizing high-dimensional interaction information to approach the direction of maximizing similarity. Specifically, this loss function calculates the covariance matrix of the high-dimensional distributions of the two data points and uses the negative logarithm of the determinant of the covariance matrix as an indicator of the interaction similarity between the two regions.

[0078] The interaction loss function is a loss function that utilizes the geometric relationship constraints between lane lines and lane regions. In real-world scenarios, there are lane line positions that are adjacent to but do not overlap with lane positions. To address this geometric constraint, and considering that both results are processed using a segmentation method, this invention adds an intersection-over-union ratio (IoU) loss for lane line and lane interaction. Specifically, the IoU is calculated by combining the lane line prediction result with the lane ground truth value, and vice versa.

[0079] The specific formula for calculating the crossover ratio loss is as follows:

[0080]

[0081] Where L pred and L gt D represents the predicted and ground truth pixel regions of the lane lines, respectively. pred and D gt These represent the predicted and ground truth pixel regions of the lane, respectively.

[0082] The specific expression for calculating the overall loss function of the detection network is as follows:

[0083] Loss = w aux L aux +w iou L iou +wc e L ce +w rmi L rmi

[0084] Where L aux L represents the auxiliary loss. iou To compare the loss between intersection and union, L ce and L rmi These represent the lane line loss function and the lane loss function, respectively. aux、 w iou w ce w rmi The weights representing auxiliary loss, intersection-to-merge ratio loss, lane line loss, and lane loss are preset to 0.2, 0.1, 1.0, and 0.1 respectively in this embodiment.

[0085] In this embodiment, after acquiring RGB images of the vehicle's front view containing real road scenes, a segmentation and annotation tool is used to annotate the lane line regions and lane regions in the images, forming training and testing sets. The training set is processed into high-resolution and low-resolution images with different resolutions. Both high-resolution and low-resolution images are input into the detection network described above. The detection network is trained and optimized by combining the loss functions described above until the loss functions converge and the detection network training is completed. Then, the detection model is used for the lane line and lane detection task. By inputting the RGB image of the road to be tested, the lane lines and lane positions in the image can be obtained.

[0086] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A lane line and lane area detection method based on a road scene, characterized by, The vehicle front-view RGB image containing a real road scene is processed into high-resolution images and low-resolution images with different resolutions, and the high-resolution images and the low-resolution images are input into a detection network which is pre-constructed and trained to obtain the detection results of lane lines and lanes; The detection network comprises: a lane line primary feature extraction module configured to extract primary features of lane lines, and configured to input high-resolution images; a lane primary feature extraction module configured to extract primary features of lanes, and configured to input low-resolution images; a feature interaction learning module configured to interact information of the primary features of the lane lines and the primary features of the lanes to generate interaction features; a lane line decoder configured to decode the interaction features for the first time, and configured to input the features decoded for the first time into a lane decoder, and configured to receive decoding features decoded for the first time by the lane decoder, and configured to decode for the second time in combination with the interaction features; a lane decoder configured to decode the interaction features for the first time, and configured to input the features decoded for the first time into a lane line decoder, and configured to receive decoding features decoded for the first time by the lane line decoder, and configured to decode for the second time in combination with the interaction features; an interaction constraint loss module configured to constrain the decoding features decoded for the second time by the lane line decoder and the lane decoder; The lane line primary feature extraction module and the lane primary feature extraction module each comprise a convolutional neural network, a context information extraction unit and an auxiliary loss branch, the convolutional neural network is configured to extract initial primary features of lane lines / lanes, and the context information extraction unit and the auxiliary loss branch are configured to optimize the initial primary features of the lane lines / lanes to generate the primary features of the lane lines / lanes; The feature interaction learning module comprises a concatenation layer, a linear preprocessing layer, a global average pooling layer and a plurality of linear operation layers connected in sequence; The primary features of the lane lines and the primary features of the lane are spliced via a splicing layer, preprocessed by a linear preprocessing layer into first fusion features , the first fusion features are subjected to nonlinear activation and global average pooling via a global average pooling layer to obtain weight features , the weight features are processed via two linear operation layers, divided into lane line feature branches and lane feature branches, and obtain respective weights and , the and are subjected to normalization exponential function processing to obtain the relative weights of the two branches and , and the and are respectively multiplied by the features spliced by the splicing layer, and the results are added to obtain second fusion features , and then the attention weighted fusion of the two branches is obtained, and finally the second fusion features are processed by two linear operation layers to restore the same dimension as the first fusion features to obtain third fusion features , the third fusion features are added to the first fusion features residual to obtain the final fusion features.

2. The lane line and lane area detection method based on a road scene according to claim 1, characterized in that, The auxiliary loss branch comprises a plurality of convolutional layers, the initial primary features of the lane lines / lanes are processed by the plurality of convolutional layers to obtain preliminary segmentation results of the lane lines / lanes, and a cross-entropy loss is calculated in combination with a true value label to supervise the convolutional neural network; The context information extraction unit comprises an attention calculation preprocessing branch and an attention calculation branch, the attention calculation preprocessing branch is configured to calculate a weight of a spatial dimension direction of the preliminary segmentation results obtained by the auxiliary loss branch by a normalization exponential function, to obtain context information in combination with the initial primary features, to convert the context information into a key vector and a value vector by a convolutional layer, and to convert the initial primary features into a query vector by another convolutional layer, and the attention calculation branch is configured to take the key vector, the value vector and the query vector as inputs, to output a feature after splicing the initial primary features, and to output the primary features. 3.The road scene based lane line and lane area detection method of claim 1, wherein, The expression of the calculation process of the feature interaction learning module is as follows: In the formula, is a first fusion feature, is a primary feature of a lane line, is a primary feature of a lane, is a concatenation operation, is a linear operation, is a weight feature, is an average pooling operation, is an intermediate weight generated by the pooling feature, is a weight of the lane line feature branch and the lane feature branch, respectively, is a second fusion feature, is a third fusion feature, is a final fusion feature. 4.The road scene based lane line and lane area detection method of claim 1, wherein, The lane line decoder comprises a coarse up-sampling branch and a fine up-sampling branch, the coarse up-sampling branch obtains a coarse up-sampling output result by convolution combined with bilinear interpolation up-sampling, the fine up-sampling branch obtains a fine up-sampling output result through a deconvolution layer and a Non-bottleneck layer, the coarse up-sampling output result and the fine up-sampling output result are directly added to obtain lane line decoding features.

5. The lane line and lane area detection method based on a road scene according to claim 4, characterized in that, The decoding feature interaction module inputs the features decoded by the lane line decoder for the first time into the lane decoder and inputs the features decoded by the lane decoder for the first time into the lane line decoder; The decoding feature interaction module comprises a convolution layer and a residual structure, the features decoded by the two decoders for the first time are respectively input into the convolution layer and the residual structure, so that auxiliary information is extracted from the outputs of the two decoders to the other decoder, the auxiliary information is directly added to the original inputs of the two decoders to obtain fusion information, and the fusion information is used as a new input of the decoder to perform second decoding to obtain a prediction result of the two branches after information interaction in the decoding stage.

6. The lane line and lane area detection method based on a road scene according to claim 1, characterized in that, The interaction constraint loss module comprises: A lane line loss function is used to constrain the output result of the lane line decoder; A lane loss function is used to constrain the output result of the lane decoder; An interaction loss function is used to constrain the output results of the lane line decoder and the lane decoder, and a constraint on the spatial geometric relationship of the two output results is realized.

7. The lane line and lane area detection method based on a road scene according to claim 6, characterized in that, The interaction loss function is an intersection over union loss of lane lines and lane interactions, intersection over union loss The specific calculation expression is as follows: where, and represent the predicted and ground truth pixel regions of lane lines, respectively, and represent the predicted and ground truth pixel regions of lanes, respectively. 8.The road scene based lane line and lane region detection method of claim 7, wherein, The overall loss function calculation expression of the detection network is as follows: wherein represents an auxiliary loss, is an intersection over union loss, and are a lane line loss function and a lane loss function, respectively, , , , represent weights corresponding to the auxiliary loss, the intersection over union loss, the lane line loss, and the lane loss.

Images