A feature extraction and autonomous driving perception method based on channel and spatial attention modules

Abstract

The present invention provides a feature extraction and autonomous driving perception method based on channel and spatial attention modules, wherein the method comprises: embedding an adaptive attention module (AAM) into a YOLOP model, and given an intermediate special preprocessed feature map of a BDD100K training set as input, the AAM first uses the channel attention module to infer a one-dimensional channel attention feature map, and multiplies it with the input feature map to obtain an intermediate channel refined feature; the spatial attention module groups the intermediate channel refined features into multiple sub-features along the channel dimension, and then refines them in parallel; subsequently, all refined sub-features are aggregated, and a channel shuffling mechanism is adopted to generate an adaptively emphasized final refined feature, thereby enhancing the target detection capability of the model in autonomous driving; the present invention proposes a lightweight and efficient adaptive attention module, which significantly improves its performance in different driving perception tasks and balances the complexity of the model.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a feature extraction and autonomous driving perception method based on channel and spatial attention modules. Background Art

[0002] In recent years, autonomous driving technology has become a hot research area in the automotive industry. It relies on perception algorithms to obtain information about the surrounding environment for accurate environmental perception and decision-making. Current autonomous driving perception algorithms mainly use traditional computer vision and sensor technologies, such as cameras, radars, and lidars, to perceive the road environment. This method is expensive, the subsequent information fusion process is complex, and traditional perception algorithms still face some challenges in complex traffic environments and extreme weather conditions. However, the breakthrough progress made by neural networks and deep learning technologies in the field of image processing has provided new ideas and methods for the research of autonomous driving autonomous perception technology. Recent studies have shown that lightweight, high-performance networks are very important in visual perception systems, and that attention mechanisms can significantly improve network performance.

[0003] Existing neural networks and deep learning technologies either ignore the significance of using two types of attention mechanisms (channel and space) at the same time, or increase the complexity of the model and fail to balance detection accuracy and speed; such as the CBAM module, given a feature map, the CBAM module can sequentially generate attention feature map information in both channel and space dimensions, and then the two feature map information is multiplied with the original input feature map for adaptive feature correction to generate the final feature map; the channel attention module in CBAM uses global maximum pooling and global average pooling to utilize different information respectively, while the AAM in the present invention assumes that the features extracted by the two pooling methods play different roles at different stages of the network and occupy different weights. Therefore, the present invention designs an adaptive mechanism to change the weights of the features extracted by average pooling and maximum pooling. In addition, the AAM provided by the present invention uses one-dimensional convolution to avoid channel degradation and ensure the interaction of feature information between channels.

[0004] Based on this, a feature extraction method based on channel and spatial attention modules is needed to enhance the performance of neural networks for autonomous driving. Summary of the Invention

[0005] The object of the present invention is to provide a feature extraction method based on channel and spatial attention modules, comprising the following steps:

[0006] S1: intermediate special preprocessing feature map input to the neural network;

[0007] S2: The channel attention module infers a one-dimensional channel attention feature map and multiplies the one-dimensional channel attention feature map with the input feature map to obtain the intermediate channel refined features;

[0008] S3: The spatial attention module groups the intermediate channel refined features into multiple sub-features along the channel dimension and then processes them in parallel;

[0009] S4: Generate spatial attention feature descriptors and refined sub-features along the channel dimension for each sub-feature;

[0010] S5: Aggregate the refined sub-features and use a “channel shuffling” mechanism to transfer information between different sub-features to generate a final refined feature with adaptive emphasis on important areas.

[0011] Furthermore, the intermediate special preprocessing feature map is a feature map processed by the resblock module.

[0012] Furthermore, S2 includes:

[0013] S2.1: The channel attention module uses average pooling and maximum pooling operations to aggregate spatial dimension information. These operations generate two one-dimensional vectors and They represent the features extracted by global average pooling and global maximum pooling respectively;

[0014] S2.2: Subsequently, the two vectors are sent to the adaptive attention block to obtain the adaptive feature Z C , the adaptive attention module has two trainable parameters α and β, which can participate in the joint training of the network; the internal reasoning process of AAM is:

[0015]

[0016] Where α and β are both floating point numbers greater than 0 and less than 1;

[0017] S2.3: To address the problem of information loss caused by channel dimensionality reduction, a 1D convolution with a convolution kernel of size k is used to capture the interaction information between channels. To determine the value of k, use the formula:

[0018]

[0019] Where, |t| odd represents the nearest odd number of t, C is the number of channels, γ and b are hyperparameters, which are usually set to 2 and 1 respectively in all experiments in this study; using this mapping, the kernel size k can be adaptively determined by the number of channels C;

[0020] S2.4: In addition, ZC After the one-dimensional convolution, a vector is generated, and then the Leaky ReLU function is used to activate this vector; the calculation process of the channel attention module is summarized as follows:

[0021]

[0022] S2.5: The channel attention module infers the one-dimensional channel attention feature map, which is Multiply it with the input feature map to obtain the channel-refined feature as

[0023] Preferably, S3 is specifically: the spatial attention module refines the channel features Divide into g groups along the channel dimension, that is, X′=[X′1,X′2,...,X′ g ],

[0024] Furthermore, S4 includes:

[0025] S4.1: The spatial attention module uses the sub-feature X′ along the channel dimension i The average pooling and maximum pooling operations are used to generate matrices and It is cascaded along the channel dimension;

[0026] S4.2: Use a 1×1 kernel 2D convolution to perform inter-channel information interaction and information compression; In addition, f c (·) is used to enhance X″ i The representation of , that is, generating a spatial attention feature descriptor and a refined sub-feature along the channel dimension for each sub-feature, and the final output is obtained by the following equation:

[0027]

[0028] Where, f c (·) is a linear operation, W and b are Parameters, is a convolution operation, where the superscript represents the convolution kernel size, the subscript represents the convolution dimension, and F cat It is a channel cascade operation.

[0029] Preferably, S5 is specifically as follows: aggregate all refined sub-features and use the “channel shuffle” mechanism to achieve cross-group information flow along the channel dimension, where X″=[X″1,X″2,...,X″ g ],i=1,2,...,g, To generate the final refined features with adaptive emphasis on important areas; the final output X″ of the AAM is the same size as the X input; in addition, W, b and the parameters of the two-dimensional convolution are only the parameters introduced into the spatial attention module; in a single spatial attention module, the number of channels of each branch is C / g, and the total number of parameters is 2(1+H×W)×C / g (g is usually 32 or 64).

[0030] The present invention also includes an autonomous driving perception method based on a channel and spatial attention module, comprising the following steps:

[0031] S1: The above-mentioned feature extraction method based on channel and spatial attention module is used to embed the neural network;

[0032] S2: Select a data set and divide the data set into a training set and a validation set;

[0033] S3: Applying the training set to the neural network to perform model training;

[0034] S4: Input the image to be identified into the trained model for target detection and recognition.

[0035] Furthermore, the data set adopts the BDD100K data set; the neural network is the YOLOP neural network.

[0036] Compared with the prior art, the at least one technical solution adopted in the embodiments of this specification can achieve the following beneficial effects:

[0037] First, this paper proposes a new and efficient attention module (AAM) to improve the representational capabilities of CNN networks. The AAM consists of two submodules: a channel attention module and a spatial attention module. By using one-dimensional convolution and the independently designed AAM in the channel attention module, better channel attention maps can be obtained. In the spatial attention module, the channel dimensions are first grouped into multiple sub-features, generating spatial attention feature maps and refined sub-features along the channel dimension of each sub-feature, making all feature maps more semantically powerful. In the spatial attention module, a grouping method is usually adopted to divide the input channels into multiple sub-feature groups. The purpose of this grouping process is to better capture the spatial relationships and semantic information of the input data. Through this grouping and refinement, the spatial attention module can make all feature maps more semantically powerful, which means that the network can better understand the structure and content of the input data, thereby improving the performance of the model in various tasks. This technology performs well in improving the perception and generalization capabilities of the model.

[0038] Second, the present invention uses a "channel shuffle" operator to exchange information between different sub-features, enabling the network to better locate feature distributions. By using this operator, the neural network's ability to locate feature distributions can be significantly improved. The core idea of this technique is to rearrange or mix the channels or feature maps in the network, allowing the model to better capture the correlations between different features and improve feature representation.

[0039] Channel shuffling can be considered a feature reordering method, helping models better learn feature combinations and associations when handling complex tasks. This helps the network better understand the structure and characteristics of the input data, thereby improving its performance. By introducing channel shuffling, neural networks can more freely exchange information to adapt to the characteristics of different tasks and data. This technique has been widely used in the field of computer vision, particularly in tasks such as image classification, object detection, and semantic segmentation, where it can improve model robustness and generalization.

[0040] Third, experimental results show that the AAM module is an efficient and lightweight module that can significantly improve the performance of convolutional neural network architectures in the field of autonomous driving, especially in target detection tasks, where it enhances the system's capabilities. The lightweight design of the AAM module enables it to perform well on embedded systems or mobile devices with limited computing resources. This module introduces an adaptive attention mechanism that helps the network better focus on areas in the image related to target detection, thereby improving the accuracy and robustness of target detection.

[0041] In the field of autonomous driving, object detection is a crucial task, helping vehicles identify and understand obstacles, traffic signs, and other vehicles in their surroundings. The CNN architecture using the AAM module can better handle complex road conditions and images under varying lighting conditions, helping to improve the safety and reliability of autonomous driving systems. The introduction of the AAM module provides an effective tool for autonomous driving systems, significantly improving object detection capabilities while maintaining a lightweight model. This technology opens up new possibilities for the development and application of autonomous driving technology, and is expected to improve traffic safety on the road and the performance of autonomous vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0043] Figure 1 A flowchart of the steps of the method of the present invention;

[0044] Figure 2 Schematic diagram of the steps of the method of the present invention;

[0045] Figure 3 Schematic diagram of the steps of the channel attention module of the present invention;

[0046] Figure 4 Schematic diagram of the steps of the spatial attention module of the present invention;

[0047] Figure 5 Schematic diagram of the present invention integrating the ResBlock module. DETAILED DESCRIPTION

[0048] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0049] The following describes the embodiments of the present invention through specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0050] The present invention aims to provide an autonomous driving perception algorithm based on channel and spatial attention modules to address the poor performance of target detection in complex traffic environments and extreme weather conditions. The invention is dedicated to the research and development of lightweight, high-performance neural networks. This invention proposes a lightweight and efficient adaptive attention module based on channel and spatial attention modules to balance model performance and complexity, significantly enhancing the representation capabilities of CNN networks.

[0051] like Figure 1 and Figure 2 As shown, in order to achieve the above-mentioned purpose of the invention, the technical solutions adopted to solve the technical problems are as follows:

[0052] S1: Embedding AAM into the YOLOP model, given the intermediate special preprocessed feature maps of the BDD100K dataset As input; the spatial attention module consists of the following steps:

[0053] S2: The channel attention module infers a one-dimensional channel attention feature map and multiplies it with the input feature map to obtain the intermediate channel refined features;

[0054] The channel attention module uses average pooling and maximum pooling operations to aggregate spatial dimension information. These operations generate two one-dimensional vectors and The two one-dimensional vectors are sent to the adaptation block to obtain the adaptive features Z C , the adaptive attention module (AAM) has two trainable parameters α and β, which can participate in the joint training of the network. The internal reasoning process of AAM is:

[0055]

[0056] Where α and β are floating point numbers greater than 0 and less than 1. These two trainable parameters generate an adaptive mechanism between the global average pooling feature and the global maximum pooling feature.

[0057] In order to solve the problem that channel dimensionality reduction may cause information loss, a 1D convolution with a convolution kernel of size k is used to capture the interaction information between channels. In order to determine the value of k, the formula is used:

[0058]

[0059] Where, |t| odd represents the nearest odd number of t, C is the number of channels, γ and b are hyperparameters, usually set to 2 and 1 respectively;

[0060] Using this mapping, the kernel size k can be adaptively determined by the number of channels C;

[0061] Z C After the one-dimensional convolution, a vector is generated and activated with the Leaky ReLU function;

[0062] The calculation process of the channel attention module is summarized as follows:

[0063]

[0064] The channel attention module infers the one-dimensional channel attention feature map, which is Multiply it with the input feature map to obtain the channel-refined feature as

[0065] The spatial attention module consists of the following steps:

[0066] S3: The spatial attention module groups the intermediate channel refined features into multiple sub-features along the channel dimension and then processes them in parallel;

[0067] Channel refinement feature map Divide into g groups along the channel dimension, that is, X′=[X′1,X′2,...,X′ g ],

[0068] Each sub-feature X′ i Gradually capture specific spatial semantic information during training;

[0069] S4: The spatial attention module generates a spatial attention feature descriptor and refined sub-features along the channel dimension for each sub-feature;

[0070] The spatial attention module uses the sub-features X′ along the channel dimension i The average pooling and maximum pooling operations are used to generate matrices and

[0071] Use a two-dimensional convolution with a kernel to interact and compress information between channels;

[0072] Use f c (·) to enhance X″ i The representation of , that is, generating a spatial attention feature descriptor and a refined sub-feature along the channel dimension for each sub-feature, and the final output is obtained by the following equation:

[0073]

[0074] Where, f c (·) is a linear operation, W and b are Parameters;

[0075] is a convolution operation, where the superscript represents the convolution kernel size, the subscript represents the convolution dimension, and F cat It is a channel cascade operation;

[0076] S5: Aggregate all refined sub-features;

[0077] Similar to ShuffleNet v2, it uses the “channel shuffling” mechanism to achieve information exchange between different sub-features, where X″=[X″1,X″2,...,X″ g ],i=1,2,...,g, To generate the final refined features with adaptive emphasis on important areas;

[0078] The final output X″ of the AAM module has the same size as the input X. The parameters W, b, and the two-dimensional convolution are only the parameters introduced into the spatial attention module.

[0079] In a single spatial attention module, the number of channels in each branch is C / g, and the total number of parameters is 2(1+H×W)×C / g (g is usually 32 or 64); the target in the predicted image or video is obtained by processing the autonomous driving perception algorithm with channel and spatial attention modules.

[0080] like Figure 3 and Figure 4 As shown in the figure, the present invention proposes a new efficient attention module AAM to improve the representation ability of the CNN network; the AAM consists of two sub-modules: a channel attention module and a spatial attention module. By using one-dimensional convolution and the independently designed AAM, a better channel attention map can be obtained. In the spatial attention module, the dimensions of the channel are first grouped into multiple sub-features, generating a spatial attention feature map and refined sub-features along the channel dimension of each sub-feature, so that all feature maps become more semantically powerful. Subsequently, all refined sub-features are aggregated, and a "channel shuffle" operator is used to realize information exchange between different sub-features, so that the network can better locate the feature distribution. Experimental results show that the AAM module is a very lightweight module that can significantly improve the performance of the CNN architecture and enhance the detection capability of targets in autonomous driving.

[0081] like Figure 5 As shown in the figure, the AAM provided by the present invention can integrate the Resblock module, further improving the computational efficiency of the CNN architecture.

[0082] To better understand the solution of the present invention, the following uses the AAM fusion YOLOP model as an example to describe the application of the autonomous driving perception algorithm based on the channel and spatial attention modules:

[0083] Example 1: AAM Embedded in ResNet50 and ResNet101

[0084] Compared with the mainstream attention mechanism:

[0085] (The data set is CIFAR-100)

[0086]

[0087]

[0088] Table 1

[0089] As shown in Table 1, the AAM provided by the present invention exhibits excellent generalization and expansion capabilities when processing network models on large data sets. The success of combining AAM with ResNet50 and ResNet101 and performing image classification tasks demonstrates the versatility of AAM, indicating that it can be used in various neural network architectures and has wide applicability. Embedding AAM into ResNet50 and ResNet101 for image classification tasks shows that AAM can work with different deep learning architectures to improve their performance, rather than being limited to specific network structures. AAM exhibits excellent generalization capabilities on large data sets. AAM is able to adaptively adjust its area of interest to better capture important features in the image, and this adaptability is a key aspect of versatility because it allows AAM to be applicable to a variety of different types of tasks and data sets. In addition, AAM is generally a modular component that can be relatively easily integrated into different neural network architectures. This modularity allows researchers and engineers to use AAM in different neural networks without making large-scale modifications to the entire network, and to apply AAM to different tasks and models, including image classification, object detection, semantic segmentation, etc.; AAM combined with ResNet50 and ResNet101 in various tasks has demonstrated its versatility and applicability, making it a promising attention mechanism module that can be widely used in various neural network architectures; experiments have shown that it performs better than mainstream channel attention mechanisms such as SE and CBAM on actual data.

[0090] Example 2: AAM embedded in YOLOP

[0091] Autonomous driving perception task: The AAM of the present invention is embedded into YOLOP, and the comparison is made between the two systems with and without AAM:

[0092] (The data set is BDD100K)

[0093]

[0094] Table 2

[0095]

[0096] Table 3

[0097]

[0098] Table 4

[0099] AAM was embedded in the YOLOP model, and feature extraction and training models were established according to the methods provided by this invention. The training and test results are shown in Tables 2, 3, and 4. The results demonstrate that the autonomous driving perception algorithm based on the channel and spatial attention modules is highly effective. The results also indicate that the method employed by AAM can produce more robust feature description information, effectively improving model performance. AAM's adaptability enables the model to better adapt to diverse data distributions, thereby improving generalization performance and helping the model better understand the spatial structure in images and the correlations between different regions. This further enhances the model's ability to capture and express image features.

[0100] The analysis and application examples demonstrate that the autonomous driving perception algorithm based on the channel and spatial attention modules provided by this paper can enhance model accuracy and achieve accurate target detection. These characteristics make this method suitable for effective application in the autonomous driving perception field.

[0101] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A feature extraction method based on channel and spatial attention modules, characterized by: The following steps are involved: S1: intermediate special preprocessing feature map input to the neural network; S2: The channel attention module infers a one-dimensional channel attention feature map and multiplies the one-dimensional channel attention feature map with the input feature map to obtain the intermediate channel refined features; S3: The spatial attention module groups the intermediate channel refined features into multiple sub-features along the channel dimension, including: Refine the channel feature Divide into g groups along the channel dimension, that is Then process in parallel; S4: Generates spatial attention feature descriptors and refined sub-features along the channel dimension for each sub-feature, including: S4.1: The spatial attention module uses the sub-feature X along the channel dimension i ′ average pooling and maximum pooling operations to generate matrices and It is cascaded along the channel dimension; S4.2: Use a 1×1 kernel 2D convolution to perform inter-channel information interaction and information compression; In addition, f c (·) is used to enhance X i ″, that is, generating a spatial attention feature descriptor and a refined sub-feature along the channel dimension for each sub-feature, and the final output is obtained by the following equation: Where, f c (·) is a linear operation, W and b are Parameters, is a convolution operation, where the superscript represents the convolution kernel size, the subscript represents the convolution dimension, and F cat It is a channel cascade operation; S5: Aggregate the refined sub-features and use a channel shuffling mechanism to transfer information between different sub-features to generate a final refined feature with adaptive emphasis on important areas, including: Aggregate all refined sub-features and use the "channel shuffle" mechanism to achieve cross-group information flow along the channel dimension, where To generate the final refined features with adaptive emphasis on important areas; the final output X″ of the adaptive attention module AAM is the same size as the X input; in addition, W, b and the parameters of the two-dimensional convolution are only the parameters introduced into the spatial attention module; in a single spatial attention module, the number of channels of each branch is C / g, and the total number of parameters is 2(1+H×W)×C / g.

2. The feature extraction method based on channel and spatial attention module according to claim 1, characterized in that: The intermediate special preprocessing feature map is a feature map processed by the ResBlock module.

3. The feature extraction method based on channel and spatial attention module according to claim 1, characterized in that: The S2 includes: S2.1: The channel attention module uses average pooling and maximum pooling operations to aggregate spatial dimension information. These operations generate two one-dimensional vectors and They represent the features extracted by global average pooling and global maximum pooling respectively; S2.2: The two vectors are sent to the adaptation block to obtain the adaptive feature Z C , the adaptive attention module AAM has two trainable parameters α and β, which can participate in the joint training of the network; the internal reasoning process of AAM is: Where α and β are both floating point numbers greater than 0 and less than 1; S2.3: Use a 1D convolution with a convolution kernel of size k to capture the interaction information between channels. To determine the value of k, use the formula: Where, |t| odd represents the nearest odd number of t, C is the number of channels, γ and b are hyperparameters, which are set to 2 and 1 respectively; using the mapping, the kernel size k is adaptively determined by the number of channels C; S2.4: Z C After the one-dimensional convolution, a vector is generated, and then the Leaky ReLU function is used to activate this vector; the calculation process of the channel attention module is summarized as follows: S2.5: The channel attention module infers the one-dimensional channel attention feature map, which is Multiply it with the input feature map to obtain the channel-refined feature as 4. An autonomous driving perception method based on channel and spatial attention modules, characterized by: The following steps are involved: S1: using the feature extraction method based on the channel and spatial attention module according to any one of claims 1 to 3 to embed the neural network; S2: Select a data set and use the data set as a training set; S3: Applying the training set to the neural network to perform model training; S4: Input the image to be identified into the trained model for target detection and recognition.

5. The autonomous driving perception method based on channel and spatial attention modules according to claim 4, characterized in that: The data set adopts the BDD100K data set; the neural network is the YOLOP neural network.

Images