Lightweight lane line detection method in autonomous driving scenario

Abstract

The application discloses a lightweight lane line detection method in an automatic driving scene. The method comprises the following steps: constructing a lane line teacher detection model and a student detection model, and loading the trained teacher detection model weight; acquiring a lane line data set and inputting the lane line data set into the teacher and student lane line detection models respectively; using an asymmetric weighted distillation module to transfer the knowledge in the teacher model to the student model; using a cascaded feature refinement module to extract lane line feature information and fuse different granularity information to further enhance the features; using a bidirectional aggregation deviation head to expand the spatial receptive field range in the horizontal and vertical directions, improve the topological structure representation capability of the lane line, and obtain the final lane line coordinate prediction sequence. The method realizes asymmetric weighted attention distillation and cascaded feature refinement, effectively reduces the parameter quantity and overall reasoning time of the model while not sacrificing the lane line detection accuracy.

Description

Technical Field

[0001] This invention relates to a lightweight lane detection method for autonomous driving scenarios, belonging to the field of computer vision technology. Background Technology

[0002] Lane detection, as one of the core tasks of intelligent vehicle systems, is a crucial link in achieving environmental perception for autonomous driving. Accurate positioning and real-time identification of lane boundaries provide fundamental support for the safe and efficient operation of autonomous driving. With the rapid development of autonomous driving technology, the accuracy and real-time performance of lane detection have become important indicators for measuring the performance of autonomous driving systems, directly impacting the decision-making ability and operational reliability of autonomous vehicles in complex road conditions.

[0003] The rise of deep learning technology has brought new breakthroughs to lane line detection, changing the technological paradigm of traditional detection methods. Traditional detection methods based on manual features rely on manually designed feature extraction operators, which are not only tedious and complex in design but also have extremely poor adaptability to environmental changes. Deep learning methods, on the other hand, transform the tedious manual feature design process into automated feature learning, significantly improving the accuracy and environmental robustness of lane line detection. In recent years, many deep learning-based lane line detection methods have improved their performance through architectural optimization. Compared to traditional methods based on manual features, deep learning-based detection methods, with their ability to automatically extract deep semantic features of lane markings, can progressively analyze multi-dimensional information such as edges, textures, and shapes from the input image, maintaining stable detection performance even in complex scenarios such as drastic changes in lighting, target occlusion, or worn and blurred lane markings.

[0004] While deep learning-based lane detection methods have made some progress in detection accuracy, they still face many unresolved issues in real-world road environments. First, real-world road scenarios lack clear lane visual cues. In non-standardized road scenarios, lane markings are often missing or irregular. Combined with factors such as occlusion by dynamic traffic participants and visual interference from inclement weather, lane feature blurring or information loss can easily occur, significantly increasing detection difficulty. Second, existing methods focus on local feature learning and have weak capabilities for modeling the global topology of lane lines, making it difficult to effectively model long-distance lane line features. Finally, existing lane detection networks have high computational costs, making it difficult to balance detection accuracy and real-time performance to meet the real-time requirements of autonomous driving. Summary of the Invention

[0005] The purpose of this invention is to solve the above-mentioned problems in existing methods and to propose a lightweight lane detection method for autonomous driving scenarios.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] A lightweight lane detection method for autonomous driving scenarios, characterized by the following steps:

[0008] S1: Obtain the lane line detection dataset and input it into the teacher backbone network and student backbone network respectively to obtain feature maps of different levels of the network, denoted as Ti and Si, where i represents the feature level, 1≤i≤4. The student backbone network is a cascaded feature refinement network, which includes a cascaded feature refinement module and a bidirectional aggregation offset head.

[0009] S2: Using an asymmetric weighted attention distillation module, the attention distance between the teacher feature map Ti and the student feature map Si is calculated, where i represents the feature level, 1≤i≤4. This effectively transmits the teacher feature map information to the student network. This module includes a channel excitation regularizer and an attention distillation loss. The attention distance is calculated by the channel excitation regularizer and the attention loss and is used to measure the difference in the attention regions of the teacher and student networks.

[0010] S3: The cascaded feature refinement module extracts lane line feature information through the region decomposition block and enhances the features through the cascaded information refinement module to improve the student network's ability to model lane line features and improve the network's detection performance. This module includes the region decomposition block RDB and the cascaded information refinement module CIR.

[0011] S4: Using a bidirectional aggregation offset head, the spatial receptive field in the horizontal and vertical directions is expanded through the bidirectional feature aggregation module (BFA) to fully capture the contextual information of the lane lines, further refine the topological representation of the lane lines, and improve the network's fit to the lane lines. This module includes the bidirectional feature aggregation module (BFA).

[0012] S5: Adjust the student feature map S4 and the predefined prior lane anchor A through a bidirectional aggregation offset head, and then output the adjusted prior lane anchor A as the final lane line prediction result.

[0013] A further technical solution is that the asymmetric weighted attention distillation module calculates the attention gap between the teacher and student networks, thereby transmitting the feature information of the teacher network to the student network; the asymmetric weighted attention distillation module includes a channel excitation regularizer R and an attention distillation loss Latt.

[0014] Furthermore, the channel activation regularizer R obtains the importance score of each channel through the importance predictor P and weights it onto the squared value of the L2 norm of each channel between the teacher and student feature maps, so as to give higher attention to important channels in the teacher feature map; the attention distillation loss Latt compresses the feature maps of the teacher network and the student network respectively through the mapping function Q and directly calculates the squared value of the L2 norm between them; the attention distance Z between the teacher and student networks is obtained by adding the channel activation regularizer R and the attention distillation loss Latt; n represents the number of levels of the feature map, v represents the channel mean deviation, and σ represents the channel mean variance. λ and sigmoid are scalar coefficients, sigmoid represents the activation function, Ci represents the number of channels in the i-th layer feature map, [j] represents the j-th feature map, and P [j] Let Q represent the importance score of the j-th feature map, and Q be the mapping function.

[0015]

[0016] Z = R(S,T) + Latt.

[0017] A further technical solution is that the cascaded feature refinement module extracts lane line feature information through the residual structure composed of Region Decomposition Blocks (RDBs) and enhances the feature information through the cascaded information refinement module (CIR), thereby enhancing the student network's ability to model lane line features. The Region Decomposition Blocks (RDBs) include a lightweight RDB-A structure and an RDB-B structure for deep networks. The Region Decomposition Blocks (RDBs) include BSConv convolutional layers, BatchNorm normalization layers, ReLU activation layers, and SimAM attention.

[0018] Further, the RDB-A structure decomposes the standard convolution by merging a 3×1 filter BSConv convolutional layer and a 1×3 filter BSConv convolutional layer, thereby extracting lane line features. The input feature map X is passed through a 3×1 filter BSConv convolutional layer, a BatchNorm normalization layer, and a ReLU activation layer to generate a feature map F1 capturing the horizontal information of the lane lines. Feature map F1 is passed through a 1×3 filter BSConv convolutional layer and a BatchNorm normalization layer to generate a feature map F2 capturing both the horizontal and vertical information of the lane lines. Feature map F2 is then processed through SimAM attention to highlight important spatial feature regions and added to feature map X, and then passed through a ReLU activation layer to obtain the output feature OutA. BN represents the BatchNorm normalization layer, BSConv... 1×3 This represents a BSConv convolutional layer with a 1×3 filter; its specific calculation formula is as follows:

[0019] F1 = BSConv3×1 (BN(ReLU(X))),

[0020] F2 = BSConv 1×3 (BN(F1)),

[0021] OutA = ReLU(SimAM(F2) + X).

[0022] Furthermore, the RDB-B structure extracts lane line features by merging a 3×1 filter BSConv convolutional layer and a 1×3 filter BSConv convolutional layer; the input feature map X is passed through a CBR layer with a 1×1 filter to reduce the number of channels to obtain intermediate features G1, and then through a BB layer with a 3×1 filter and a 1×3 filter to capture lane line features G2; subsequently, the number of channels is restored through a standard convolutional layer with a 1×1 filter, and SimAM is used to focus on important features, finally adding it to the feature map X to obtain the output feature OutB; the CBR layer consists of a Conv convolutional layer, a BatchNorm normalization layer, and a ReLU activation layer; the BB layer consists of a BSConv convolutional layer and a BatchNorm normalization layer; BN represents the BatchNorm normalization layer, and BB... 1×3 Represents a BB layer with a 1×3 filter, CBR 1×1 This represents a convolutional unit constructed from a Conv convolutional layer with a 1×1 filter, a BatchNorm normalization layer, and a ReLU activation layer. Conv1 represents a Conv convolutional layer with a 1×1 filter, and its specific calculation formula is as follows:

[0023] G1 = CBR 1×1 (X),

[0024] G2=BB 1×3 (BB 3×1 (G1)),

[0025] OutB=ReLU(SimAM(BN(Conv1(G2)))+G1).

[0026] Furthermore, the cascaded information refinement module CIR captures coarse-grained and fine-grained features respectively through the DSR convolutional layer and the grouped attention refinement module GAR, and fuses features of different granularities to enhance the features; the DSR convolutional layer consists of a DSConv convolutional layer and a ReLU activation layer. By inputting the feature map Y into the DSConv convolutional layer to capture feature information, this information is then added to the feature map Y and passed to the ReLU activation layer to finally obtain the output feature C1; its specific calculation formula is as follows:

[0027] C1 = ReLU(DSConv(Y) + Y).

[0028] Furthermore, the group attention refinement module GAR includes a global average pooling GAP, a 1×1 Conv convolutional layer, and a Sigmoid activation layer. It divides the input feature L into k groups according to channels to obtain group features Pk, where k represents the group index. Group features Pk capture global features through the global average pooling GAP and capture channel attention weights through the 1×1 Conv convolutional layer and Sigmoid activation layer. These weights are further weighted with the group features to obtain a feature map PEk, where k represents the group index. Subsequently, the weighted group features PEk are concatenated through channels to generate a feature map L1, which is added to the input feature L. After passing through SimAM, the output features L2, which focus on both channels and spatial dimensions, are obtained. Split represents channel grouping, K represents the total number of groups, [·] represents channel concatenation, and Conv1 represents a Conv convolutional layer with a 1×1 filter. The specific calculation formula is as follows:

[0029] Pk = Split(L), k∈1,2,…,K

[0030] PEk=Sigmoid(Conv1(GAP(Pk)))*Pk,

[0031] L1 = [P1, P2, ..., PK],

[0032] L2 = SimAM(L + L1).

[0033] Further, the cascaded information refinement module CIR generates feature maps fs1, fs2, and fs3 from the feature map Fin through three DSR convolutional layers, and then obtains fine-grained features fd1, fd2, and fd3 from the feature maps Fin, fs1, and fs2 through the grouped attention refinement module GAR. Subsequently, the fine-grained features are concatenated by channels and adjusted by a 1×1 Conv convolutional layer to generate feature map fd4. Feature map Fin is added to fs3 to obtain feature map fs4. Then, coarse-grained feature fs4 and fine-grained feature fd4 are concatenated by channels, adjusted by a 1×1 Conv convolutional layer, and then passed through Si... mAM constructs spatial weights w, weights w with fine-grained feature fd4 to obtain a fine-grained attention map, and weights w with coarse-grained feature map fs4 to obtain a coarse-grained feature map. The coarse and fine-grained feature maps are added together and channel-adjusted through a 1×1 Conv convolutional layer to obtain a fused feature map Ffuse. Finally, the fused feature map Ffuse is further modeled through a DSConv convolutional layer to model the channel relationships and added to the feature map Fin to obtain the output feature Fout. [·] represents channel concatenation, and Conv1 represents a Conv convolutional layer with a 1×1 filter. Its specific calculation formula is as follows:

[0034] fd1,fs1=GAR(Fin),DSR(Fin),

[0035] fd2,fs2=GAR(fs1),DSR(fs1),

[0036] fd3,fs3=GAR(fs2),DSR(fs2),

[0037] fd4,fs4=Conv1([fd1,fd2,fd3]),fs3+Fin,

[0038] w = SimAM(Conv1[fd4,fs4]),

[0039] Ffuse=Conv1(w*fd4+(1-w)*fs4),

[0040] Fout = DSConv(Ffuse) + Fin.

[0041] A further technical solution is that the bidirectional aggregation offset head expands the spatial receptive field range in the horizontal and vertical directions through the bidirectional feature aggregation module BFA, constructs the offset representation of the prior lane anchor, further refines the prior lane anchor, and improves the topological structure representation of the lane line.

[0042] Furthermore, the bidirectional feature aggregation module BFA interacts the horizontal sampled feature map H and the vertical sampled feature map V with the prior lane feature ROI, and adds the interacted features JH and JV to obtain the output feature Jout. The interaction process generates a query vector query from the prior lane feature ROI through a 1×1 Conv convolutional layer, while the sampled feature map generates a key vector key and a value vector value through a 1×1 Conv convolutional layer. The query vector query is multiplied by the key vector key to obtain an attention matrix, which is then multiplied by the value vector value to obtain the output feature. This output feature is then added to the prior lane feature ROI after passing through a 1×1 Conv convolutional layer to obtain the interactive output feature I. Conv1 represents a Conv convolutional layer with a 1×1 filter, and its specific calculation formula is as follows:

[0043] query = Conv1(ROI),

[0044] key,value=Conv1(H),Conv1(H),

[0045] JH=ROI+Conv1((query*key)*value),

[0046] key,value=Conv1(V),Conv1(V),

[0047] JV=ROI+Conv1((query*key)*value),

[0048] Jout = JH + JV.

[0049] Furthermore, the bidirectional aggregation offset head generates a priori lane feature ROI by sampling the feature information of the prior lane anchor on the feature map W. The feature map is sampled in both horizontal and vertical directions to obtain a horizontally sampled feature map H and a vertically sampled feature map V. Subsequently, the prior lane feature ROI, the horizontally sampled feature map H, and the vertically sampled feature map V are processed by the bidirectional feature aggregation module BFA to obtain the output feature Jout, and the offset representation of the prior lane anchor is obtained through a fully connected layer FC. Then, the prior lane feature ROI is processed by a Conv convolutional layer and a fully connected layer FC to obtain the prediction result pred for the output lane. The offset representation offset is further added to the prediction result pred to generate the adjusted output lane prediction anchor lane. The specific calculation formula is as follows:

[0050] offset = FC(Jout),

[0051] pred = FC(Conv1(ROI)),

[0052] lane = pred + offset.

[0053] A further technical solution involves training the lane detection network using the following steps:

[0054] Construct a lane line teacher detection network and a student detection network, and load the pre-trained teacher detection network weights;

[0055] Construct a training set, which is a sequence of video frames and their lane line ground value coordinates;

[0056] The training set is fed into the lane line teacher and student detection networks, and the student detection network is trained.

[0057] The teacher detection network outputs intermediate layer feature maps and prior lane anchor parameters;

[0058] The student detection network outputs lane coordinate prediction sequences, intermediate layer feature maps, and prior lane anchor parameters.

[0059] Calculate the differences between the intermediate feature maps and prior lane anchor parameters of the teacher network and student network, and backpropagate.

[0060] The difference between the predicted sequence and the true coordinate sequence of the student detection network is calculated and backpropagated.

[0061] When the loss value reaches its minimum, the network converges, training stops, and a well-trained lane detection network is obtained.

[0062] The beneficial effects of the above technical solution are as follows: This invention provides an asymmetric weighted attention distillation module, which calculates the attention gap between the teacher and student networks, thereby constraining the student network's feature information to learn the teacher network's feature information; This invention designs a cascaded feature refinement module, which extracts lane line feature information through region decomposition blocks and enhances the features through the cascaded information refinement module, thereby enhancing the student network's ability to model lane line features and improving the network's lane line detection performance; This invention develops a bidirectional aggregation offset head, which expands the spatial receptive field in the horizontal and vertical directions through a bidirectional feature aggregation module, further refining the topological representation of lane lines; The three modules are integrated into the network, significantly reducing the number of network parameters and inference time while ensuring that the detection accuracy does not decrease or even improves, demonstrating the advantages of the proposed technical solution. Attached Figure Description

[0063] To more clearly illustrate the technical solutions of the embodiments of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0064] Figure 1 This is a flowchart illustrating the overall network process according to an embodiment of the present invention.

[0065] Figure 2 This is a diagram illustrating the overall network architecture of an embodiment of the present invention.

[0066] Figure 3 This is a structural diagram of the asymmetric weighted attention distillation module in an embodiment of the present invention;

[0067] Figure 4 This is a diagram of the region decomposition block structure in an embodiment of the present invention;

[0068] Figure 5 This is a structural diagram of the cascaded information refinement module in an embodiment of the present invention;

[0069] Figure 6 This is a structural diagram of the bidirectional aggregation offset head in an embodiment of the present invention;

[0070] Figure 7 This is a result diagram of an embodiment of the present invention. Detailed Implementation

[0071] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0072] This invention provides a lightweight lane detection method for autonomous driving scenarios, such as... Figure 1 As shown, it includes the following steps:

[0073] S1: Construct the training set, which consists of video frame sequences and their corresponding ground-value coordinate sequences of lane lines. Three widely used datasets are used for training: Tusimple, CULane, and LLAMAS. Tusimple is a classic public dataset primarily covering highway scenes; CULane is a commonly used public dataset in lane line detection, containing 88,880 training images and a test set covering eight challenging scenarios; LLAMAS is a public dataset based on real-world road lane line annotations, with data sourced from European urban and highway scenes, containing over 100,000 high-resolution images.

[0074] S2: Construct a lane line teacher detection network; its structure is shown below. Figure 2 The teacher detection network weights are loaded; the lane line dataset is input into the teacher backbone network and the student backbone network to obtain feature maps of different levels of the network, denoted as Ti and Si, where i represents the level of the feature, 1≤i≤4. The student backbone network is a cascaded feature refinement network, which includes a cascaded feature refinement module and a bidirectional aggregation offset head.

[0075] S3: Construct an asymmetric weighted attention distillation module; its structure is shown below. Figure 3 ;

[0076] S3-1: Asymmetric weighted attention distillation module, which calculates the attention gap between teacher and student networks to transmit the feature information of the teacher network to the student network; the asymmetric weighted attention distillation module includes a channel excitation regularizer R and an attention distillation loss Latt.

[0077] S3-2: The channel excitation regularizer R obtains the importance score of each channel through the importance predictor P and weights it onto the squared value of the L2 norm of each channel between the teacher and student feature maps, so as to give higher attention to important channels in the teacher feature map; the attention distillation loss Latt compresses the feature maps of the teacher network and the student network respectively through the mapping function Q and directly calculates the squared value of the L2 norm between them; the attention distance Z between the teacher and student networks is obtained by adding the channel excitation regularizer R and the attention distillation loss Latt; n represents the number of levels of the feature map, v represents the channel mean deviation, and σ represents the channel mean variance. λ and sigmoid are scalar coefficients, sigmoid represents the activation function, Ci represents the number of channels in the i-th layer feature map, [j] represents the j-th feature map, and P [j] Let Q represent the importance score of the j-th feature map, and Q be the mapping function.

[0078]

[0079] Z = R(S,T) + Latt.

[0080] S4: Construct a cascaded feature refinement network, including a cascaded feature refinement module. See the structure below. Figure 4 And a bidirectional aggregation offset head, the structure of which can be found in [reference needed]. Figure 5 ;

[0081] S4-1: Cascaded Feature Refinement Module. This module extracts lane line feature information through the residual structure composed of Region Decomposition Blocks (RDBs) and enhances the feature information through the Cascaded Information Refinement Module (CIR), thereby enhancing the student network's ability to model lane line features. The Region Decomposition Blocks (RDBs) include a lightweight RDB-A structure and a deep network RDB-B structure. Each RDB contains a BSConv convolutional layer, a BatchNorm normalization layer, a ReLU activation layer, and a SimAM attention layer.

[0082] S4-2: The structure of the Domain Decomposition Block (RDB-A) is shown in [link to documentation]. Figure 4a) Standard convolutions are decomposed by merging 3×1 filter BSConv convolutional layers and 1×3 filter BSConv convolutional layers to extract lane line features. The input feature map X is passed through a 3×1 filter BSConv convolutional layer, a BatchNorm normalization layer, and a ReLU activation layer to generate a feature map F1 that captures the horizontal information of the lane lines. Feature map F1 is passed through a 1×3 filter BSConv convolutional layer and a BatchNorm normalization layer to generate a feature map F2 that captures both the horizontal and vertical information of the lane lines. Feature map F2 is then processed through SimAM attention to highlight important spatial feature regions and added to feature map X, and then passed through a ReLU activation layer to obtain the output feature OutA. BN represents the BatchNorm normalization layer, BSConv 1×3 This represents a BSConv convolutional layer with a 1×3 filter; its specific calculation formula is as follows:

[0083] F1 = BSConv 3×1 (BN(ReLU(X))),

[0084] F2 = BSConv 1×3 (BN(F1)),

[0085] OutA = ReLU(SimAM(F2) + X).

[0086] S4-3: The structure of the Domain Decomposition Block (RDB-B) is shown below. Figure 4 b) The structure extracts lane line features by merging a 3×1 filter BSConv convolutional layer and a 1×3 filter BSConv convolutional layer. The input feature map X is passed through a CBR layer with a 1×1 filter to reduce the number of channels, obtaining intermediate feature G1. Lane line features G2 are then captured through a BB layer with one 3×1 filter and one 1×3 filter. Subsequently, the number of channels is restored through a standard convolutional layer with a 1×1 filter, and SimAM is used to focus on important features. Finally, the output feature OutB is obtained by adding it to the feature map X. The CBR layer consists of a Conv convolutional layer, a BatchNorm normalization layer, and a ReLU activation layer. The BB layer consists of a BSConv convolutional layer and a BatchNorm normalization layer. BN represents the BatchNorm normalization layer, and BB... 1×3 Represents a BB layer with a 1×3 filter, CBR 1×1 This represents a convolutional unit constructed from a Conv convolutional layer with a 1×1 filter, a BatchNorm normalization layer, and a ReLU activation layer. Conv1 represents a Conv convolutional layer with a 1×1 filter, and its specific calculation formula is as follows:

[0087] G1 = CBR 1×1 (X),

[0088] G2=BB 1×3 (BB 3×1 (G1)),

[0089] OutB=ReLU(SimAM(BN(Conv1(G2)))+G1).

[0090] S4-4: The Cascaded Information Refinement (CIR) module captures coarse-grained and fine-grained features respectively through the DSR convolutional layer and the Grouped Attention Refinement (GAR) module, and fuses features of different granularities to enhance the features.

[0091] S4-5: The DSR convolutional layer consists of a DSConv convolutional layer and a ReLU activation layer. It captures feature information by inputting the feature map Y into the DSConv convolutional layer, then adds this information to the feature map Y and passes it to the ReLU activation layer to finally obtain the output feature C1. Its specific calculation formula is as follows:

[0092] C1 = ReLU(DSConv(Y) + Y).

[0093] S4-6: The Group Attention Refinement Module (GAR) consists of a Global Average Pooling (GAP), a 1×1 Conv convolutional layer, and a Sigmoid activation layer. It divides the input feature L into k groups according to channels to obtain group features Pk, where k represents the group index. Group features Pk capture global features through the GAP and channel attention weights through the 1×1 Conv convolutional layer and Sigmoid activation layer. These weights are further weighted with the group features to obtain a feature map PEk, where k represents the group index. Subsequently, the weighted group features PEk are concatenated through channels to generate feature map L1, which is added to the input feature L. After passing through SimAM, the output features L2, which focus on both channels and spatial dimensions, are obtained. Split represents channel grouping, K represents the total number of groups, [·] represents channel concatenation, and Conv1 represents a Conv convolutional layer with a 1×1 filter. The specific calculation formula is as follows:

[0094] Pk = Split(L), k∈1,2,…,K

[0095] PEk=Sigmoid(Conv1(GAP(Pk)))*Pk,

[0096] L1 = [P1, P2, ..., PK],

[0097] L2 = SimAM(L + L1).

[0098] S4-7: Cascaded Information Refinement Module (CIR), its structure is shown in [link to CIR]. Figure 5The feature map Fin is passed through three DSR convolutional layers to generate feature maps fs1, fs2, and fs3, respectively. Then, feature maps Fin, fs1, and fs2 are passed through a Group Attention Refinement (GAR) module to obtain fine-grained features fd1, fd2, and fd3, respectively. Subsequently, the fine-grained features are concatenated by channels and adjusted using a 1×1 Conv convolutional layer to generate feature map fd4. Feature map Fin is added to fs3 to obtain feature map fs4. Finally, coarse-grained feature fs4 and fine-grained feature fd4 are concatenated by channels, adjusted using a 1×1 Conv convolutional layer, and spatial weights are constructed using SimAM. w, the spatial weight w is weighted with the fine-grained feature fd4 to obtain the fine-grained attention map, the spatial weight 1-w is weighted with the coarse-grained feature map fs4 to obtain the coarse-grained feature map, the coarse-grained and fine-grained feature maps are added together and channel adjustment is performed through a 1×1 Conv convolutional layer to obtain the fused feature map Ffuse; finally, the fused feature map Ffuse is further modeled with the inter-channel relationship through a DSConv convolutional layer and added to the feature map Fin to obtain the output feature Fout; [·] represents channel concatenation, Conv1 represents a Conv convolutional layer with a 1×1 filter, and its specific calculation formula is as follows:

[0099] fd1,fs1=GAR(Fin),DSR(Fin),

[0100] fd2,fs2=GAR(fs1),DSR(fs1),

[0101] fd3,fs3=GAR(fs2),DSR(fs2),

[0102] fd4,fs4=Conv1([fd1,fd2,fd3]),fs3+Fin,

[0103] w = SimAM(Conv1[fd4,fs4]),

[0104] Ffuse=Conv1(w*fd4+(1-w)*fs4),

[0105] Fout = DSConv(Ffuse) + Fin.

[0106] S5: Construct a bidirectional aggregate offset head; its structure is shown below. Figure 6 ;

[0107] S5-1: Bidirectional Aggregation Offset Head. It expands the spatial receptive field range in the horizontal and vertical directions through the bidirectional feature aggregation module BFA, constructs the offset representation of the prior lane anchor, further refines the prior lane anchor, and improves the topological structure representation of the lane line.

[0108] S5-2: The bidirectional feature aggregation module BFA interacts the horizontal sampled feature map H and the vertical sampled feature map V with the prior lane feature ROI, and adds the interacted features JH and JV to obtain the output feature Jout. The interaction process generates a query vector query from the prior lane feature ROI through a 1×1 Conv convolutional layer, while the sampled feature map generates a key vector key and a value vector value through a 1×1 Conv convolutional layer. The query vector query is multiplied by the key vector key to obtain the attention matrix, which is then multiplied by the value vector value to obtain the output feature. This output feature is then added to the prior lane feature ROI after passing through a 1×1 Conv convolutional layer to obtain the interactive output feature I. Conv1 represents a Conv convolutional layer with a 1×1 filter, and its specific calculation formula is as follows:

[0109] query = Conv1(ROI),

[0110] key,value=Conv1(H),Conv1(H),

[0111] JH=ROI+Conv1((query*key)*value),

[0112] key,value=Conv1(V),Conv1(V),

[0113] JV=ROI+Conv1((query*key)*value),

[0114] Jout = JH + JV.

[0115] S5-3: Bidirectional aggregation offset head. It generates a priori lane feature ROI by sampling feature information of the prior lane anchor on the feature map W. The feature map is sampled horizontally and vertically to obtain a horizontally sampled feature map H and a vertically sampled feature map V. Then, the prior lane feature ROI, the horizontally sampled feature map H, and the vertically sampled feature map V are processed by the bidirectional feature aggregation module BFA to obtain the output feature Jout, and the offset representation of the prior lane anchor is obtained through a fully connected layer FC. Subsequently, the prior lane feature ROI is processed through a Conv convolutional layer and a fully connected layer FC to obtain the prediction result pred for the output lane. The offset representation is further added to the prediction result pred to generate the adjusted output lane prediction anchor lane. The specific calculation formula is as follows:

[0116] offset = FC(Jout),

[0117] pred = FC(Conv1(ROI)),

[0118] lane = pred + offset.

[0119] S6: Construct a student detection model, such as Figure 2 As shown, training will be conducted;

[0120] S6-1: The training set is input into both the teacher detection network and the student detection network for training the student detection network. The resolution of the input images is uniformly adjusted to 320×800, and data augmentation is performed using methods such as horizontal flipping, random adjustment of image brightness, and blurring. The AdamW algorithm is used to train the network, with a batch size of 16 and an initial learning rate of 6e-4.

[0121] S6-2: The teacher detection network outputs the intermediate layer feature map and prior lane anchor parameters; the student detection network outputs the intermediate layer feature map, prior lane anchor parameters, and lane coordinate prediction sequence; the intermediate layer feature map and prior lane anchor parameters of the teacher and student detection networks are used to calculate the distillation loss, and the lane coordinate prediction sequence is used to calculate the prediction loss with the ground truth coordinate sequence.

[0122] S6-3: Calculate the distillation loss for the output of the teacher-student network. The L2 norm is used as the loss function, T... fea and S fea Let T represent the intermediate layer feature maps of the teacher network and the student network, respectively. prior and S prior Let represent the prior anchor parameters of the teacher network and the prior anchor parameters of the student network, respectively. Then, the expression for the distillation loss function Loss1 is as follows:

[0123] Loss1=L L2 (T fea ,S fea )+L L2 (T prior ,S prior ).

[0124] S6-4: Calculate the prediction loss between the predicted lane coordinate sequence and the actual lane line coordinate sequence. Kornia FocalLoss, SmoothL1Loss, LineIOU, and NLLLoss are used as loss functions. p and G cls_t B represents the predicted and true values ​​of the foreground and background classification results, respectively. p and G anchor_t C represents the predicted and actual values ​​of the anchor parameter, respectively. seg and G seg_t D represents the predicted mask and the ground truth mask, respectively, segmented pixel by pixel. p and G coord_t Let the predicted and true values ​​of the coordinate sequence be represented. Then, the expression for the prediction loss function Loss2 is as follows:

[0125] Loss2=L cls (A p G cls_t )+L xylθ (B p G anchor_t )+L seg (C seg G seg_t )+L LineIoU (D p G coord_t ).

[0126] S6-5: The final loss is the sum of the distillation loss and the prediction loss. Therefore, the final loss function expression is as follows:

[0127] Loss = Loss1 + Loss2.

[0128] S6-6: When the loss value reaches the minimum, the student network converges, training stops, parameters are saved, and the trained lane detection network is obtained.

[0129] S7: Input the image to be detected into the trained lane detection network to output the predicted coordinate sequence of the image to be detected and generate the final lane prediction map.

[0130] To verify the effectiveness of the above examples, the method of this invention was compared with other state-of-the-art methods on three datasets: Tusimple, CULane, and LLAMAS. Four metrics were selected for the Tusimple dataset: F1, Acc, FP, and FN. Eleven metrics were selected for the CULane dataset: F1... 50 F1 75 The eleven metrics are: Normal, Crowd, Dazzle, Shadow, Noline, Arrow, Curve, Cross, and Night. Except for Cross, F1... 50 F1 75 Larger values ​​for Normal, Crowd, Dazzle, Shadow, Noline, Arrow, Curve, and Night indicate better performance. On the LLAMAS dataset, two metrics were used on the validation set: mF1 and F1@50. Three metrics were used on the test set: F1@50, Pre, and Rec. The experimental results are shown in Table 1.

[0131] Table 1 shows the comparison results of detection accuracy on the CULane dataset.

[0132] Method Backbone F1 Acc FP FN SCNN VGG16 95.97 96.53 6.17 1.80 RESA Res34 96.93 96.82 3.63 2.48 UFLD Res34 88.02 95.86 18.91 3.75 UFLDv2 Res34 96.22 95.56 3.18 4.37 LaneATT Res18 96.71 95.57 3.56 3.01 LaneATT Res34 96.77 95.63 3.53 2.92 CondLaneNet Res18 97.01 95.48 2.18 3.80 CondLaneNet Res34 96.98 95.37 2.20 3.82 CLRNet Res18 97.89 96.84 2.28 1.92 CLRNet Res34 97.82 96.87 2.27 2.08 Our(CFRNet) ResCFR-A 97.57 96.80 2.87 1.95 Our(CFRNet) ResCFR-B 97.46 96.54 2.69 2.36

[0133] As shown in Table 1, this embodiment uses the near-teacher network CLRNet on the Tusimple dataset.

[0134] Table 2 shows the comparison results of detection accuracy on the CULane dataset.

[0135]

[0136] As shown in Table 2, this embodiment maintains excellent detection performance on the CULane dataset while reducing the number of parameters and the overall inference time, demonstrating the effectiveness of the method in this embodiment.

[0137] Table 3 shows the comparison results of detection accuracy on the LLAMAS dataset.

[0138]

[0139] As shown in Table 3, the detection performance of this embodiment on the LLAMAS dataset is close to that of the teacher network CLRNet, which proves the effectiveness of the method in this embodiment.

[0140] Figure 7 The image shows a comparison of the results of the method of this invention. The first column is the original image, the second column is the mask image (a binary mask of the lane ground truth), the third column is the ground truth image (a visualization of the real labels on the original image), and the fourth column is the output image of the method, which is the prediction image displayed after post-processing of the prediction sequence output by the network. The comparison shows that the solution provided in this example can accurately locate lane line objects and precisely pinpoint lane line targets.

[0141] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and these variations still fall within the protection scope of the present invention.

Claims

1. A lightweight lane detection method for autonomous driving scenarios, characterized in that... Includes the following steps: S1: Obtain the lane line detection dataset and input it into the teacher backbone network and student backbone network respectively to obtain feature maps of different layers of the network, denoted as Ti and Si, where i represents the feature level. The student backbone network is a cascaded feature refinement network, which includes a cascaded feature refinement module and a bidirectional aggregation offset head. S2: Using an asymmetric weighted attention distillation module, the attention distance between the teacher feature map Ti and the student feature map Si is calculated, where i represents the feature level. This module effectively transmits teacher feature map information to the student network. It includes a channel excitation regularizer and an attention distillation loss. The attention distance is calculated by the channel excitation regularizer and the attention loss and is used to measure the difference in the attention regions of the teacher and student networks. The asymmetric weighted attention distillation module is used to calculate the attention gap between the teacher and student networks, thereby transferring the feature information of the teacher network to the student network. The asymmetric weighted attention distillation module includes a channel activation regularizer R and an attention distillation loss Latt. The channel activation regularizer R obtains the importance score of each channel through an importance predictor P and weights it onto the squared L2 norm of each channel between the teacher and student feature maps, thus giving higher attention to important channels in the teacher feature map. The attention distillation loss Latt compresses the feature maps of the teacher and student networks respectively through a mapping function Q and directly calculates the squared L2 norm between them. The attention distance Z between the teacher and student networks is obtained by adding the channel activation regularizer R and the attention distillation loss Latt. S3: The cascaded feature refinement module extracts lane line feature information through the region decomposition block and enhances the features through the cascaded information refinement module to improve the student network's ability to model lane line features and improve the network's detection performance. This module includes the region decomposition block RDB and the cascaded information refinement module CIR. S4: Using a bidirectional aggregation offset head, the spatial receptive field in the horizontal and vertical directions is expanded through the bidirectional feature aggregation module (BFA) to fully capture the contextual information of the lane lines, further refine the topological representation of the lane lines, and improve the network's fit to the lane lines. This module includes the bidirectional feature aggregation module (BFA). S5: Adjust the student feature map S4 and the predefined prior lane anchor A through a bidirectional aggregation offset head, and then output the adjusted prior lane anchor A as the final lane line prediction result.

2. The lightweight lane detection method for autonomous driving scenarios as described in claim 1, characterized in that, The cascaded feature refinement module extracts lane line feature information through the residual structure composed of Region Decomposition Blocks (RDBs) and enhances the feature information through the cascaded Information Refinement (CIR) module, thereby enhancing the student network's ability to model lane line features. The RDBs include a lightweight RDB-A structure and an RDB-B structure for deep networks. Each RDB contains a BSConv convolutional layer, a BatchNorm normalization layer, a ReLU activation layer, and a SimAM attention layer. The RDB-A structure combines a 3×1 filter BSConv convolutional layer and a 1×3 filter BSConv convolutional layer. Conv convolutional layers decompose standard convolutions to extract lane line features. The input feature map X is passed through a BSConv convolutional layer with a 3×1 filter, a BatchNorm normalization layer, and a ReLU activation layer to generate a feature map F1 that captures the horizontal information of the lane lines. Feature map F1 is passed through a BSConv convolutional layer with a 1×3 filter and a BatchNorm normalization layer to generate a feature map F2 that captures both the horizontal and vertical information of the lane lines. Feature map F2 is then passed through a SimAM attention layer to highlight important spatial feature regions and added to feature map X. Finally, it is passed through a ReLU activation layer to obtain the output feature OutA. The RDB-B structure extracts lane line features by merging a 3×1 filter BSConv convolutional layer and a 1×3 filter BSConv convolutional layer. The input feature map X is passed through a CBR layer with a 1×1 filter to reduce the number of channels, obtaining intermediate features G1. Lane line features G2 are then captured through a BB layer with one 3×1 filter and one 1×3 filter. Subsequently, the number of channels is restored through a standard convolutional layer with a 1×1 filter, and SimAM is used to focus on important features. Finally, the output features OutB are added to the feature map X. The CBR layer consists of a Conv convolutional layer, a BatchNorm normalization layer, and a ReLU activation layer. The BB layer consists of a BSConv convolutional layer and a BatchNorm normalization layer. The cascaded information refinement module CIR captures coarse-grained and fine-grained features respectively through a DSR convolutional layer and a group attention refinement module GAR, and fuses features of different granularities to enhance the features. The DSR convolutional layer consists of a DSConv convolutional layer and a ReLU activation layer. By inputting the feature map Y into the DSConv convolutional layer to capture feature information, the information is then added to the feature map Y and passed to the ReLU activation layer to finally obtain the output feature C1. The group attention refinement module GAR includes global average pooling (GAP) and a 1×1... The Conv convolutional layer and Sigmoid activation layer divide the input feature L into k groups according to channels to obtain grouped features Pk, where k represents the group index. The grouped features Pk capture global features through global average pooling (GAP) and channel attention weights through a 1x1 Conv convolutional layer and Sigmoid activation layer. These weights are further weighted with the grouped features to obtain feature map PEk, where k represents the group index. Subsequently, the weighted grouped features PEk are concatenated through channels to generate feature map L1, which is then added to feature map L. After passing through SimAM, output features L2 with channel and spatial dimension attention are obtained. The cascaded information refinement module CIR generates feature maps fs1, fs2, and fs3 from feature map Fin through three DSR convolutional layers. Feature maps Fin, fs1, and fs2 are then passed through the group attention refinement module GAR to obtain fine-grained features fd1, fd2, and fd3. Subsequently, the fine-grained features are concatenated through channels and processed using a 1x1 Conv convolutional layer. The Conv convolutional layer performs channel adjustment to generate feature map fd4; feature map Fin is added to fs3 to obtain feature map fs4; then, the coarse-grained feature fs4 and the fine-grained feature fd4 are concatenated, and after channel adjustment by a 1×1 Conv convolutional layer, spatial weight w is constructed through SimAM. The spatial weight w is weighted with the fine-grained feature fd4 to obtain a fine-grained attention map. The spatial weight 1-w is weighted with the coarse-grained feature map fs4 to obtain a coarse-grained feature map. The coarse-grained and fine-grained feature maps are added together and channel adjusted by a 1×1 Conv convolutional layer to obtain the fused feature map Ffuse.Finally, the fused feature map Ffuse is further modeled through a DSConv convolutional layer to model the inter-channel relationships and added to the feature map Fin to obtain the output feature Fout.

3. The lightweight lane detection method for autonomous driving scenarios as described in claim 1, characterized in that, The bidirectional aggregation offset head expands the spatial receptive field in the horizontal and vertical directions through the bidirectional feature aggregation module BFA to construct the offset representation of the prior lane anchor, further refine the prior lane anchor, and improve the topological structure representation of the lane line; the bidirectional feature aggregation module BFA interacts the horizontal sampling feature map H and the vertical sampling feature map V with the prior lane feature ROI, and adds the interacted features JH and JV to obtain the output feature Jout; The interaction process generates a query vector (query) from the prior lane feature ROI through a 1×1 Conv convolutional layer, while the sampled feature map generates a key vector (key) and a value vector (value) through a 1×1 Conv convolutional layer. The query vector is multiplied by the key vector to obtain the attention matrix, which is then multiplied by the value vector to obtain the output feature, and finally processed through a 1×1... The Conv convolutional layer is added to the prior lane feature ROI to obtain the interactive output feature I. The bidirectional aggregation offset head generates the prior lane feature ROI by sampling the feature information of the prior lane anchor on the feature map W. The feature map is sampled in the horizontal and vertical directions to obtain the horizontally sampled feature map H and the vertically sampled feature map V. Then, the prior lane feature ROI, the horizontally sampled feature map H, and the vertically sampled feature map V are passed through the bidirectional feature aggregation module BFA to obtain the output feature Jout, and the offset representation of the prior lane anchor is obtained through the fully connected layer FC. Subsequently, the prior lane feature ROI is passed through the Conv convolutional layer and the fully connected layer FC to obtain the prediction result pred of the output lane. The offset representation offset is further added to the prediction result pred to generate the adjusted output lane prediction anchor lane.

4. The lightweight lane detection method for autonomous driving scenarios as described in claim 1, characterized in that, The training steps for a trained lane detection network include: Construct a lane line teacher detection network and a student detection network, and load the pre-trained teacher detection network weights; Construct a training set, which is a sequence of video frames and their lane line ground value coordinates; The training set is fed into the lane line teacher and student detection networks, and the student detection network is trained. The teacher detection network outputs intermediate feature maps and prior lane anchor parameters; The student detection network outputs a lane coordinate prediction sequence, intermediate feature maps, and prior lane anchor parameters. Calculate the differences between the intermediate feature maps and prior lane anchor parameters of the teacher network and student network, and backpropagate. The difference between the predicted sequence and the true coordinate sequence of the student detection network is calculated and backpropagated. When the loss value reaches its minimum, the network converges, training stops, and a well-trained lane detection network is obtained.

Images