3D lane line detection method based on multi-scale context information enhancement and dual-channel attention

Abstract

The invention discloses a 3D lane line detection method based on multi-scale context information enhancement and dual-channel attention, and the method comprises the steps: 1, providing a multi-scale context enhancement module and a dual-channel attention module, and integrating the provided 3D lane line detection method; 2, training the model, and obtaining a detection result; step 3, performing an ablation experiment; and step 4, realizing deployment. The invention provides a new multi-scale context enhancement module and a dual-channel attention module. According to the multi-scale context enhancement module, multi-expansion convolution of sawtooth wave design is adopted, cross-branch diversified feature learning is achieved, and robustness under different road conditions is improved. And a dual-channel attention module is provided, dual channels are used to suppress noise and emphasize prominent features, the ability of the model to represent the dependency relationship between complex channels is enhanced, and the accuracy of the model to lane detection is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision processing technology, and in particular to a 3D lane line detection method based on multi-scale context information enhancement and dual-channel attention. Background Art

[0002] Lane detection is one of the core functions of intelligent transportation systems. Its accuracy and performance in different environments have an important impact on autonomous driving, vehicle-mounted AR systems, etc. It can provide more reliable road information and enhance the vehicle's environmental perception ability. The performance of existing 3D lane detection algorithms in extreme weather conditions, such as heavy rain, still needs to be improved.

[0003] With the rapid development of intelligent driving and assisted driving technologies, the industry has put forward higher and higher requirements on the accuracy and robustness of lane detection. Although the existing lane detection model technology has made some progress, there are still many problems.

[0004] On the other hand, the complexity of the lane detection environment, such as extreme weather such as heavy rain, or image blur caused by dynamic vehicle movement, poses great challenges to lane detection. On the other hand, existing lane detection technologies often rely on two-dimensional images collected by monocular cameras, which are often easily interfered by environmental and equipment noise, resulting in low detection accuracy and poor model robustness.

[0005] Therefore, innovative solutions are urgently needed to break through the existing bottlenecks and improve the accuracy and robustness of lane detection.

[0006] The information disclosed in this background technology section is only intended to increase the understanding of the overall background of the invention and should not be regarded as an admission or suggestion in any form that the information constitutes the prior art already known to a person skilled in the art. Summary of the invention

[0007] The purpose of the present invention is to provide a 3D lane line detection method based on multi-scale contextual information enhancement and dual-channel attention, thereby improving the accuracy of the model for lane detection and its robustness and anti-interference ability in different weather conditions, and solving the problem that the model detection performance is poor if the image is affected by the environment, such as heavy rain or dynamic blur.

[0008] To achieve the above object, the present invention provides a 3D lane detection method MRDALane, comprising:

[0009] Step 1: Propose a multi-scale context enhancement module and a dual-channel attention module, and integrate them into the proposed 3D lane detection method;

[0010] Step 2: Train the model to obtain an efficient training model and obtain the test results;

[0011] Step 3: Conduct ablation experiment.

[0012] Step 4: Implement deployment and visualize the actual application effect of the method.

[0013] Preferably, in the above technical solution, step 1 proposes two new modules including:

[0014] 1) A dual-channel attention module is proposed and integrated into the backbone network of MRDALane. The dual-channel attention module introduces a unique attention module structure and linearly integrates two channel attention modules. The process of this module can be expressed as:

[0015]

[0016] Among them, Attention1 represents the first channel attention module, and Attention2 represents the second channel attention module. The process of Attention1 can be expressed as

[0017]

[0018] F is the input image feature, σ is the sigmoid activation function, MaxPool is the maximum pooling, AvgPool is the average pooling, and MLP is the multi-layer perceptron.

[0019] The process of Attention2 can be described as

[0020]

[0021] F is the input image feature, GAP is the global average pooling, and σ is the sigmoid activation function.

[0022] (2) A multi-scale context enhancement module is proposed and integrated into the backbone network of MRDALane. The process of this module can be expressed as follows:

[0023] Y=Dropout(ReLU(BN(Conv1×1(concat[Conv1×1(F),DilatedConv6(F),GAP(F)]))))

[0024] BN, ReLU, and Dropout represent batch normalization, ReLU activation, and Dropout operations, respectively.

[0025] Preferably, in the above technical solution, step 2 trains the training set of Apollo and Openlane lane image data sets to obtain the trained efficient detection model of the improved method, inputs the original lane image into the trained model for testing, and obtains the F1 value, error value, etc. of the model.

[0026] Preferably, in the above technical solution, step 3 of conducting an ablation experiment on the Apollo dataset includes first comparing the FPS and parameter quantity of the dual-channel attention module with other attention modules, and then training each module separately to obtain the parameter quantity and running speed of the dual-channel attention module, and the F1 value and error value of each model separate training, and compare the specific results.

[0027] Preferably, in the above technical solution, step 4 is characterized in that the code is open sourced on github and deployment documents are provided, visual model detection results are provided, and predicted images with detected lane markings are output to provide assistance for enterprise or team experiments and use.

[0028] (1) The present invention proposes a 3D lane detection method based on multi-scale context information enhancement and dual-channel attention. It innovatively proposes a multi-scale context enhancement module and adopts multiple dilated convolutions designed with sawtooth dilation rate to achieve diversified feature learning across branches and improve robustness under different road conditions. A dual-channel attention module is also proposed, which uses the dual-channel attention mechanism to effectively suppress noise and emphasize prominent features. These two modules not only improve the detection accuracy of the model, exceeding multiple 3D lane detection models such as LATR, but also significantly improve the model's anti-interference ability and robustness in different environments, such as rainy environments and dynamic blur.

[0029] (2) The present invention proposes a novel 3D lane detection model MRDALane and provides source code and development documentation to reduce the time cost of deploying this model. When applied to lane detection in the real world, it will provide enterprises or other teams with novel ideas and help. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is an overall flow chart of the 3D lane detection model MRDALane according to the present invention;

[0031] Figure 2 It is a framework diagram of the lane detection model proposed by the present invention;

[0032] Figure 3 This is the structure diagram of the added dual-channel attention module;

[0033] Figure 4 This is the structure diagram of the first channel attention module in the dual-channel attention module;

[0034] Figure 5 This is the structure diagram of the second channel attention module in the dual-channel attention module;

[0035] Figure 6 This is the structure diagram of the added multi-scale context enhancement module;

[0036] Figure 7 Actual application effect diagram; DETAILED DESCRIPTION

[0037] The specific implementation modes of the present invention are described in detail below in conjunction with the accompanying drawings, but it should be understood that the protection scope of the present invention is not limited by the specific implementation modes.

[0038] Unless explicitly stated otherwise, throughout the specification and claims, the term “comprise” or variations such as “include” or “comprising”, etc., will be understood to include the stated elements or components but not to exclude other elements or components.

[0039] like Figures 1 to 7 As shown, according to a specific embodiment of the present invention, a 3D lane line detection method based on multi-scale context information enhancement and dual-channel attention includes the following steps:

[0040] Step 1: Propose a multi-scale context enhancement module and a dual-channel attention module, and integrate them into the proposed 3D lane line detection method.

[0041] (1) A dual-channel attention module is integrated into the backbone network of the proposed 3D lane detection method to improve feature representation by selectively focusing on the regions in the input image that are most relevant to 3D lane detection (such as lane markings) while suppressing irrelevant or noisy information. The dual-channel attention mechanism consists of two channel attention sub-modules, each of which is designed to capture different aspects of the image. This significantly enhances the ability to capture lane details in complex environments.

[0042] A dual-channel attention module is proposed and integrated into the backbone network of the proposed 3D lane detection method. The dual-channel attention module introduces a unique attention module structure and linearly integrates two channel attention modules. The process of this module can be expressed as:

[0043]

[0044] Among them, Attention1 represents the first channel attention module, and Attention2 represents the second channel attention module. The process of Attention1 can be expressed as

[0045]

[0046] F is the input image feature, σ is the sigmoid activation function. The process of Attention2 can be expressed as: MaxPool is the maximum pooling, AvgPool is the average pooling, and MLP is the multi-layer perceptron.

[0047] The process of Attention2 can be expressed as

[0048]

[0049] F is the input image feature, GAP is the global average pooling, and σ is the sigmoid activation function.

[0050] The experimental results show that the dual-channel attention module achieves high frames per second (FPS), reduces the number of parameters, and enhances the model's ability to represent complex inter-channel dependencies and resist noise interference.

[0051] (2) A multi-scale context enhancement module is proposed and integrated into the backbone network of the proposed 3D lane detection method, using multiple dilution convolutions and sawtooth dilution rates. Dilution convolutions can expand the receptive field of the model with minimal computational overhead, while sawtooth dilution rates enable branches with the same dilution rate to learn different feature representations. The process of this module can be expressed as:

[0052] Y = Dropout(ReLU(BN(Conv 1×1 (concat[Conv 1×1 (F), DilatedConv6(F)), GAP(F)]))))BN, ReLU, and Dropout represent batch normalization, ReLU activation, and Dropout operations, respectively.

[0053] In the multi-scale context enhancement module, we adopt a zigzag dilation rate configuration. Although some dilation rates are repeated, each branch is initialized and trained independently. This design makes it possible for branches with the same dilation rate to learn different feature representations, thus providing a more favorable parameter space for the optimization algorithm. This design greatly enhances the robustness and overall performance of the model and gives full play to the advantages of multi-scale feature extraction.

[0054] Step 3: Train the model to obtain an efficient training model and obtain the test results

[0055] During the training process, the training set of Apollo and Openlane datasets is used to train the model, and the validation set is used for validation during the training process to obtain an efficient MRDALane detection model, which enables the model to maintain a certain accuracy and superior speed during the lane detection task.

[0056] Step 4: Conduct ablation experiment

[0057] First, the dual-channel attention module is compared with other attention modules. The experiment shows that the dual-channel attention module has the advantages of fewer parameters and higher FPS compared with other attention modules. Ablation experiments are performed on the method to verify the effectiveness of each module and analyze the reasons why the module produces the effect. By ablating each module, we can verify the effectiveness of each module. As a result, when we only integrate the dual-channel attention module, the F1 score increases from 96.1 to 96.6, while the error remains almost unchanged. After that, we incorporate the multi-scale context enhancement module into the existing framework. Compared with the LATR model, the addition of the multi-scale context enhancement module improves the F1 score and achieves better results in the far area of ​​x and the near and far areas of z.

[0058] Step 5: Implement deployment and visualize the actual detection effect of the method.

[0059] (1) The source code is open sourced on GitHub and deployment documentation is provided. The model detection lane is visualized to provide enterprises and teams with a more intuitive understanding of the performance of the model detection lane.

[0060] The foregoing description of specific exemplary embodiments of the present invention is for the purpose of illustration and demonstration. These descriptions are not intended to limit the present invention to the precise form disclosed, and it is clear that many changes and variations can be made based on the above teachings. The purpose of selecting and describing the exemplary embodiments is to explain the specific principles of the present invention and its practical application, so that those skilled in the art can realize and utilize various different exemplary embodiments of the present invention and various different selections and changes. The scope of the present invention is intended to be limited by the claims and their equivalents.

Claims

1. A 3D lane detection method based on multi-scale contextual information enhancement and dual-channel attention, characterized in that: include: Step 1: Propose a multi-scale context enhancement module and a dual-channel attention module, and integrate them into the proposed 3D lane detection method; Step 2: Train the model to obtain an efficient training model and obtain the test results; Step 3: Conduct ablation experiment; Step 4: Implement deployment and visualize the actual application effect of the method.

2. According to claim 1, a 3D lane detection method based on multi-scale context information enhancement and dual-channel attention is characterized in that: Step 1 proposes a model that includes: (1) A dual-channel attention module is proposed and integrated into the backbone network of the proposed 3D lane detection method. The dual-channel attention module introduces a unique attention module structure and linearly integrates two channel attention modules. The process of this module can be expressed as: Among them, Attention1 represents the first channel attention module, and Attention2 represents the second channel attention module, and F is the input image feature. The process of Attention1 can be expressed as: Among them, F is the input image feature, σ is the sigmoid activation function, the process of Attention2 can be expressed as, MaxPool is the maximum pooling, AvgPool is the average pooling, MLP is the multi-layer perceptron, and the process of Attention2 can be expressed as: Where F is the input image feature, GAP is the global average pooling, and σ is the sigmoid activation function. (2) A multi-scale context enhancement module is proposed and integrated into MRDALan. e The backbone network part of this module can be expressed as: Y=Dropout(ReLU(BN(Conv 1×1 (concat[Conv 1×1 (F),DilatedConv6(F),GAP(F)])))) Among them, BN, ReLU, and Dropout represent batch normalization, ReLU activation, and Dropout operations, respectively.

3. According to claim 1, a 3D lane detection method based on multi-scale context information enhancement and dual-channel attention is characterized in that: Step 2 trains the training sets of Apollo and Openlane data sets to obtain the trained efficient detection model of the improved method.

4. The 3D lane detection method based on multi-scale context information enhancement and dual-channel attention according to claim 1, characterized in that: Step 2: Input the original lane image into the trained model for testing and obtain the F1 value of the model. e rror value.

5. The 3D lane detection method based on multi-scale context information enhancement and dual-channel attention according to claim 1, characterized in that: Step 3 conducts an ablation experiment on the Apollo dataset, including first comparing the FPS and parameter quantity of the dual-channel attention module with other attention modules, and then training each module separately to obtain the parameter quantity and running speed of the dual-channel attention module, and the F1 value and error value of each model separate training, and compare the specific results.

6. The 3D lane detection method based on multi-scale context information enhancement and dual-channel attention according to claim 1, characterized in that: Step 4: Open source the code on GitHub and provide deployment documentation, as well as visual model detection results to help enterprises or teams experiment and use.

Images