Snowfield target detection method for self-driving automobile

Abstract

The invention discloses a snowfield target detection method for an autonomous vehicle. The method comprises the following steps: step 1, acquiring a snowfield data set; 2, a target detection model YOLOv8 is improved, and a multivariate feature learning module (MFL), a relevance space information enhancement module and light-weight easily-fused C2f (LFC2f) are provided; 3, training the improved model and obtaining a test result; and 4, carrying out an ablation experiment, and verifying the effectiveness of each module and the optimal position of the MFL module. An MFL module and an information enhancement module are innovatively provided on a YOLOv8 model, and an LFC2f module is added. According to the method, the trunk part of the YOLOv8 is improved, a C2f module at a specific position is replaced with an MFL module, an excellent detection effect is achieved with the lowest calculation cost, LConv is introduced to the remaining C2f modules, an LFC2f module is proposed, and the model is lighter while the precision is ensured. And an information enhancement module is provided to enhance the attention on spatial features, so that the model can better adapt to a complex snowfield environment.

Description

Technical Field

[0001] The present invention relates to the field of computer vision processing technology, and in particular to a snow target detection method for an autonomous driving vehicle. Background Art

[0002] In a snowy environment, the information of many objects will be covered by heavy snow, and the object information will be seriously missing. When snowflakes fall in the air, the vision will be blurred, making it difficult to detect the target, which greatly affects the safety of the autonomous driving process.

[0003] In addition, the heavy snow cover caused a large loss of object information, making it difficult for traditional detection algorithms to distinguish targets. The loss of key details such as color, shape and surface texture complicates the task of distinguishing different types of objects, including pedestrians, vehicles and road signs.

[0004] Although the state-of-the-art model CF-YOLO has shown strong performance in real snow scenes, its improvement over YOLOv5s is still not ideal, and exploring and developing more effective methods remains a promising research direction. There is an urgent need for new methods that can incorporate the complexity of snow scenes and more reliably detect and track objects in real time. Addressing this need is critical to ensuring the safety and reliability of autonomous driving systems, especially in areas prone to heavy snow.

[0005] However, improving target detection accuracy is often accompanied by an increase in computing power. During autonomous driving, computing power is often limited by the hardware capabilities of the onboard system, so it is crucial to focus on lightweight design.

[0006] Therefore, there is an urgent need for a lightweight and efficient method to deal with complex snowy environments.

[0007] The information disclosed in this background technology section is only intended to increase the understanding of the overall background of the invention and should not be regarded as an admission or suggestion in any form that the information constitutes the prior art already known to a person skilled in the art. Summary of the invention

[0008] The purpose of the present invention is to provide a snow target detection method for an autonomous driving vehicle, so as to solve the existing problems of a large amount of object information loss due to heavy snow cover and blurred vision caused by falling snowflakes, as well as limited hardware capabilities of the onboard system.

[0009] To achieve the above object, the present invention provides a snow target detection method for an autonomous driving vehicle, comprising:

[0010] Step 1: Get the snow data set;

[0011] Step 2: Improve the YOLOv8 model and propose the multi-feature learning module (MFL), lightweight and easy to fuse C2f (LF_C2f) and associative spatial information enhancement module (ASI Enhancement);

[0012] Step 3: Train the improved model and get the test results;

[0013] Step 4: Conduct ablation experiments to verify the effectiveness of each module and the optimal position of the MFL module.

[0014] Preferably, in the above technical solution, step 1 obtains a high-quality real and public snow data set RSOD, which mainly includes cars, pedestrians, traffic lights and other things that are often encountered when performing autonomous driving on snowy days.

[0015] Preferably, in the above technical solution, step 2 of improving the YOLOv8 model includes:

[0016] (1) The MFL module is inserted in a specific position of the backbone of YOLOv8. Inspired by the ShuffleNet module, the present invention re-parameterizes the convolution based on the channel Shuffle structure, called RCS, and reuses the features by stacking RCS. The outputs of the previous RCS and RepVGG are aggregated at one time to obtain the final output result, which constitutes the MFL module of the present invention.

[0017] (2) The C2f module at the neck of the original YOLOv8 model and the remaining position after MFL is inserted is replaced by the proposed LF_C2f module. The main improvement compared to the traditional C2f is the introduction of LConv (Lighter Conv), which only performs convolution operations on part of the input channels. If only 1 / 4 of the channels are calculated, according to the formula:

[0018]

[0019] Its FLOPs is only 1 / 16 of the traditional convolution. In addition, its memory access is:

[0020]

[0021] It is only 1 / 4 of the traditional convolution, which makes the model more lightweight.

[0022] (3) Adding the Associative Spatial Information Enhancement module (ASIEnhancement) after the SPPF module of the original YOLOv8 model. This method strengthens the correlation between the channel dimension and the spatial dimension by rotating tensors and performing convolution operations, and by aggregating the outputs of the three branches, thereby increasing the model's attention to spatial features:

[0023]

[0024] Among them, σ(·) is the sigmoid activation function, ψ1, ψ2, ψ3 are the standard convolutional layers in the three branches. Finally, average aggregation is performed:

[0025]

[0026] Preferably, in the above technical solution, step 3 trains the training set of the RSOD data set to obtain a trained efficient detection model of the improved method.

[0027] Preferably, in the above technical solution, step 3 inputs the original snow scene picture into the trained model to obtain the test result of the model, and outputs the predicted picture with the target prediction box and confidence.

[0028] Preferably, in the above technical solution, step 4 of performing ablation experiments on the RSOD dataset includes training each module in combination. And training is performed for each MFL module that may be inserted into the backbone. The map50 value and map50-95 value and GFLOPs of each model are obtained, and the specific results are compared. Compared with the prior art, the present invention has the following beneficial effects:

[0029] (1) The present invention is a snow target detection method for an autonomous vehicle. In order to overcome the problem of information loss caused by heavy snow cover and efficiently capture more diverse and rich information, the present invention proposes a multi-feature learning module (MFL) to replace the C2f in a specific position in YOLOv8, so that the model can capture more diverse features. In addition, in order to enhance the model's attention to spatial information, the present invention proposes an associative spatial information enhancement module (ASI Enhancement) to improve the dependency between spatial dimensions (W and H) and channel dimensions (C). The present invention also incorporates Lighter fusion C2f (LF_C2f), so that the method of the present invention has a low amount of computation while having high accuracy, greatly improving the efficiency of target detection in harsh environments.

[0030] (2) The present invention is a snow target detection method for autonomous driving vehicles, which has a significant performance improvement compared to the traditional YOLOv8 model and the SOTA model CF-YOLO, demonstrating its superiority in snow target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is an overall flow chart of the improved snow target detection method based on the YOLOv8 model according to the present invention;

[0032] Figure 2This is the improved YOLOv8 model framework diagram;

[0033] Figure 3 It is the structural diagram of the MFL module;

[0034] Figure 4 This is the RCS structure diagram in the MFL module;

[0035] Figure 5 This is an experimental diagram of the best position for MFL placement in the trunk.

[0036] Figure 6 This is the structure diagram of the ASI Enhancement module;

[0037] Figure 7 This is the structure diagram of LF_C2f;

[0038] Figure 8 It is the detection effect diagram of the improved method model; DETAILED DESCRIPTION

[0039] The specific implementation modes of the present invention are described in detail below in conjunction with the accompanying drawings, but it should be understood that the protection scope of the present invention is not limited by the specific implementation modes.

[0040] Unless explicitly stated otherwise, throughout the specification and claims, the term “comprise” or variations such as “include” or “comprising”, etc., will be understood to include the stated elements or components but not to exclude other elements or components.

[0041] like Figures 1 to 8 As shown, a snow target detection method for an autonomous driving vehicle according to a specific embodiment of the present invention includes the following steps:

[0042] Step 1: Obtain snow target detection dataset

[0043] Taking into account the existing problems faced in the snow target detection task, among all the existing data sets, the present invention selects the high-quality real snow data set RSOD, which contains 2100 high-resolution snow scene images, which mainly include pedestrians, vehicles, traffic lights and other common things in the process of autonomous driving in snowy weather. It can fully simulate the real snow scene conditions, and the research results obtained using this data set are more authoritative.

[0044] Step 2: Improve the YOLOv8 model and propose the Multi-Feature Learning Module (MFL), Lightweight Fusion C2f (LF_C2f) and Associative Spatial Information Enhancement Module (ASI Enhancement)

[0045] (1) The MFL module is inserted in a specific position of the main part of YOLOv8. Inspired by the ShuffleNet module, the present invention re-parameterizes the convolution based on the channel Shuffle structure, called RCS, and reuses the features by stacking RCS. This feature reuse operation can enhance the model's extraction effect on subtle features and greatly improve the detection accuracy. In addition, the RepVGG structure in MFL has no branches for convolution operations in the inference stage, which reduces the computational resource consumption of the model. The final output result is obtained by aggregating the outputs of the previous RCS and RepVGG at one time, which constitutes the MFL module of the present invention.

[0046] (2) The C2f module at the neck of the original YOLOv8 model and the remaining position after the MFL is inserted is replaced by the proposed LF_C2f module. The main improvement compared to the traditional C2f is the introduction of LConv (Lighter Conv). Based on LConv, the present invention proposes Easy fusion Block to replace Bottleneck in C2f. This module can be easily integrated into other modules. The key part LConv only performs convolution operations on part of the input channels. If only 1 / 4 of the channels are calculated, according to the formula:

[0047]

[0048] Its FLOPs is only 1 / 16 of the traditional convolution. In addition, its memory access is:

[0049]

[0050] It is only 1 / 4 of the traditional convolution, which makes the model more lightweight.

[0051] (3) Adding the Associative Spatial Information Enhancement module (ASIEnhancement) after the SPPF module of the original YOLOv8 model. This method strengthens the correlation between the channel dimension and the spatial dimension by rotating tensors and performing convolution operations, and by aggregating the outputs of the three branches, thereby increasing the model's attention to spatial features:

[0052]

[0053] Among them, σ(·) is the sigmoid activation function, ψ1, ψ2, ψ3 are the standard convolutional layers in the three branches. Finally, average aggregation is performed:

[0054]

[0055] Step 3: Train the improved model and get test results

[0056] During the training process, the training set in RSOD is used for training, and the validation set is used for verification to verify the effectiveness of each improvement point and their combination, and it can also be used to explore the best placement of MFL in the backbone.

[0057] Step 4: Conduct ablation experiments to verify the effectiveness of each module and the optimal position of the MFL module

[0058] Ablation experiments were conducted on the three improved methods to verify the effectiveness of each module and analyze the reasons why the module produced the effect. By comparing the ablation of each module, the present invention can verify the effectiveness of each module. As a result, after adding the MFL module, although the FLOPs of the model increased slightly, the map50 value of the model was significantly improved. After adding the LF_C2f module, the map50 value of the model did not increase significantly, but the overall FLOPs decreased significantly, thereby verifying the efficiency of the module. After adding the ASI Enhancement module, map50 was ideally improved without increasing any computing resource consumption, thereby improving the effectiveness of the model.

[0059] The present invention changes the placement and number of MFLs in the trunk, taking all possible situations into consideration, and finds that the detection effect is best when the MFL modules are simultaneously placed in positions two and three, which greatly saves the consumption of computing resources.

[0060] The foregoing description of specific exemplary embodiments of the present invention is for the purpose of illustration and demonstration. These descriptions are not intended to limit the present invention to the precise form disclosed, and it is clear that many changes and variations can be made based on the above teachings. The purpose of selecting and describing the exemplary embodiments is to explain the specific principles of the present invention and its practical application, so that those skilled in the art can realize and utilize various different exemplary embodiments of the present invention and various different selections and changes. The scope of the present invention is intended to be limited by the claims and their equivalents.

Claims

1. A snow target detection method for an autonomous vehicle, characterized in that: include: Step 1: Get the snow data set; Step 2: Improve the target detection model YOLOv8, propose the multi-feature learning module (MFL), lightweight easy-to-melt C2f (LF_C2f) and associative spatial information enhancement module (ASI Enhancement); Step 3: Train the improved model and get the test results; Step 4: Conduct ablation experiments to verify the effectiveness of each module and the optimal position of the MFL module.

2. A snow target detection method for an autonomous driving vehicle according to claim 1, characterized in that: Step 1: Obtain a high-quality, real and public snow dataset RSOD, which mainly contains cars, pedestrians, traffic lights and other objects that are often encountered when performing autonomous driving on snowy days.

3. The snow target detection method for an autonomous driving vehicle according to claim 1, characterized in that: Step 2 improves the YOLOv8 model by: (1) The MFL module is inserted in a specific position of the backbone of YOLOv8. Inspired by the ShuffleNet module, the present invention re-parameterizes the convolution based on the channel Shuffle structure, called RCS, and reuses the features by stacking RCS. The outputs of the previous RCS and RepVGG are aggregated at one time to obtain the final output result, which constitutes the MFL module of the present invention. (2) The C2f module at the neck of the original YOLOv8 model and the remaining position after MFL is inserted is replaced by the proposed LF_C2f module. The main improvement compared to the traditional C2f is the introduction of LConv (Lighter Conv), which only performs convolution operations on part of the input channels. If only 1 / 4 of the channels are calculated, according to the formula: Its FLOPs is only 1 / 16 of the traditional convolution. In addition, its memory access is: It is only 1 / 4 of the traditional convolution, which makes the model more lightweight. (3) Adding the Associative Spatial Information Enhancement module (ASIEnhancement) after the SPPF module of the original YOLOv8 model. This method strengthens the correlation between the channel dimension and the spatial dimension by rotating tensors and performing convolution operations, and by aggregating the outputs of the three branches, thereby increasing the model's attention to spatial features: Among them, σ(·) is the sigmoid activation function, ψ1, ψ2, ψ3 are the standard convolutional layers in the three branches. Finally, average aggregation is performed:

4. The snow target detection method for an autonomous driving vehicle according to claim 1, characterized in that: Step 3 trains the training set of the RSOD data set to obtain an efficient detection model after training of the improved method.

5. The snow target detection method for an autonomous driving vehicle according to claim 1, characterized in that: Step 3: Input the original snow scene image into the trained model, obtain the test result of the model, and output the predicted image with the target prediction box and confidence.

6. The snow target detection method for an autonomous driving vehicle according to claim 1, characterized in that: Step 4: Perform ablation experiments on the RSOD dataset, including training each module in combination and training each possible MFL module in the backbone. Obtain the map50 value, map50-95 value and GFLOPs of each model, and compare the specific results.

Images