A dynamic attention scheduling method for cooperatively screening feature channels and spatial regions

Abstract

This invention discloses a dynamic attention scheduling method for collaboratively selecting feature channels and spatial regions, relating to the field of computer vision technology. The method includes constructing a collaborative attention mechanism for selecting feature channels and spatial regions, and setting a learnable parameter λ. c and λ s This method adjusts unimportant feature channels or unimportant spatial regions; it constructs a dynamically adjustable attention module, and trains the neural network's attention mechanism by setting a periodic parameter β, periodically capturing important image features. Therefore, this dynamic attention scheduling method, which collaboratively selects feature channels and spatial regions, not only enhances the weights of important features but also suppresses redundant features, improving the model's accuracy and robustness. Simultaneously, by periodically switching feature rules and adjusting parameters, it efficiently and dynamically manages the attention mechanism, improving the model's generalization ability.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a dynamic attention scheduling method for collaboratively selecting feature channels and spatial regions. Background Technology

[0002] With the continuous development of computer vision technology, deep learning models have been widely used in tasks such as image classification and object detection. However, existing deep learning models often face problems such as inaccurate feature extraction and insufficient generalization ability when processing complex images. To improve model performance, researchers have proposed various attention mechanisms to enhance the model's ability to perceive important features by dynamically adjusting the model's attention focus.

[0003] Currently, some studies use alternating collaborative attention networks for sentiment analysis. By alternating the modeling of attention at the target level and the context level, this method focuses on keywords related to the target, thereby learning more effective contextual representations. However, when modeling attention at the target level and the context level, this method uses a simple average vector, which cannot accurately reflect the actual importance of each target, and the interaction mechanism itself is also relatively simple.

[0004] Additionally, the Convolutional Block Attention Module (CBAM) enhances the model's perceptual capabilities by introducing channel attention and spatial attention into the convolutional neural network, thereby improving model performance without increasing network complexity. While CBAM can strengthen the weights of important features, it can generate redundant features in certain tasks, interfering with the model's decision-making process and reducing its accuracy.

[0005] Therefore, it is necessary to provide a dynamic attention scheduling method that collaboratively selects feature channels and spatial regions, which can periodically switch between different attention mechanisms to improve the performance and generalization ability of the model in tasks such as image classification and object detection. Summary of the Invention

[0006] The purpose of this invention is to provide a dynamic attention scheduling method for collaboratively selecting feature channels and spatial regions, which can periodically switch the attention mechanism and dynamically adjust the attention focus at different training stages to better capture important features in the image.

[0007] To achieve the above objectives, this invention provides a dynamic attention scheduling method for collaboratively filtering feature channels and spatial regions, comprising the following steps:

[0008] S1. Construct collaborative feature channel and spatial region attention, including a multi-scale fusion-residual channel attention module and a residual spatial attention module. Process the input feature map to generate corresponding channel attention and spatial attention, and set hyperparameters for each. and This is used to control the number of important feature channels and the proportion of elements in important spatial regions, and to set parameters. and Adjust the attention weights for unimportant feature channels and unimportant spatial regions;

[0009] S2. Construct a dynamically adjustable attention module, embedding collaborative feature channel selection and spatial region attention as the first attention mechanism into the neural network, and the neural network includes a second attention mechanism;

[0010] By setting the period parameter The neural network is dynamically adjusted to be trained using either a first attention mechanism or a second attention mechanism to capture important features of the image.

[0011] In one possible implementation, in S1, the multi-scale fusion-residual channel attention module performs global max pooling and global average pooling operations on each channel of the input feature map to obtain the first channel feature map and the second channel feature map. The two are then added together and passed through a fully connected layer, a sigmoid activation function and a softmax activation function to obtain the channel attention.

[0012] Global average pooling includes first average pooling and second average pooling. The first average pooling reduces the spatial dimension of each channel to 1. 1. The second average pooling reduces the spatial dimension of each channel to 2. 2. The two dimensions are obtained as follows: and The feature vectors are then subjected to dimensionality reduction and concatenation to obtain a feature vector with dimension . The feature vectors are then subjected to dimensionality upscaling to obtain a dimension of... The first feature map; correspondingly, global max pooling includes first max pooling and second max pooling, which, after dimensionality reduction, concatenation, and dimensionality increase, yields a dimension of The second feature map.

[0013] In one possible implementation, in S1, the residual spatial attention module calculates the maximum and average values ​​of each spatial location along the channel dimension, resulting in a first spatial feature map and a second spatial feature map. The two are then concatenated and subjected to channel dimensionality reduction and a Sigmoid activation function to obtain spatial attention.

[0014] In one possible implementation, S1 includes hyperparameters. and parameters After adjusting the channel attention, an output feature map is obtained by suppressing information from non-essential feature channels. Then, training coefficients are introduced. and , fusion output feature map and input feature map ,as follows:

[0015] ;

[0016] In the formula, The output feature map generated by the multi-scale fusion-residual channel attention module;

[0017] Through hyperparameters and parameters After adjusting the spatial attention, an output feature map is obtained that suppresses unimportant spatial information. And introduce training coefficients , fusion output feature map Output feature map generated by multi-scale fusion - residual channel attention module ,as follows:

[0018] ;

[0019] In the formula, This is the final output feature map generated by the residual space attention module.

[0020] In one possible implementation, in S2, the hyperparameters This parameter represents the proportion of important feature channels. The number of non-important feature channels can be controlled using this parameter, as follows:

[0021] ;

[0022] ;

[0023] In the formula, , The number of important feature channels and the number of unimportant feature channels, respectively. This represents the total number of feature channels.

[0024] In one possible implementation, hyperparameters This parameter represents the proportion of elements in important spatial regions. It can be used to control the number of elements in non-important spatial regions, as follows:

[0025] ;

[0026] ;

[0027] In the formula, , , representing the number of elements in important spatial regions and non-important spatial regions, respectively; S represents the total number of elements in the spatial regions. , The width of the feature map, The height of the feature map.

[0028] In one possible implementation, the neural networks available for S3 include YOLOv8, ResNet152, and ResNet18 networks.

[0029] In one possible implementation, in S3, an alternative second attention mechanism includes a compression-excitation module.

[0030] In one possible implementation, within each epoch, the weights of the selected first or second attention mechanism are trained, and the weights of the unselected second or first attention mechanism are frozen.

[0031] Therefore, the present invention employs the above-mentioned dynamic attention scheduling method for collaboratively selecting feature channels and spatial regions, which has the following technical effects:

[0032] (1) The present invention constructs a dynamic attention adjustment mechanism. By periodically switching the attention mechanism, the model can focus on different features at different training stages, thereby learning the features of the data more comprehensively, preventing model overfitting, effectively increasing the model's prediction accuracy for unknown data; and reducing unnecessary parameter adjustments, which is helpful for complex module design, reducing the consumption of computing resources, and making the model more efficient.

[0033] (2) This invention constructs a collaborative feature channel and spatial region attention. Through the multi-scale fusion-residual channel attention module and residual spatial attention module, it can more accurately screen important feature channels and spatial regions, suppress redundant features, and thus significantly improve the accuracy and generalization ability of the model. At the same time, the introduction of the dynamic adjustment attention mechanism enables the model to flexibly switch the appropriate attention mechanism at different training stages, further improving the adaptability of the model in complex tasks.

[0034] (3) The present invention does not require complex architecture adjustments to the entire network. The proposed method is easy to implement on the basis of existing models, thereby reducing the difficulty of engineering implementation and operation and increasing the practicality of the model. Collaborative screening of feature channels and spatial region attention can be embedded into the product to improve data processing speed and accuracy, while reducing manufacturing costs and reducing computing resource consumption, thereby enhancing the market competitiveness of the product.

[0035] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of feature extraction in an embodiment of a dynamic attention scheduling method for collaboratively filtering feature channels and spatial regions;

[0037] Figure 2 This is a schematic diagram of a multi-scale fusion-residual channel attention module in an embodiment of a dynamic attention scheduling method for collaboratively filtering feature channels and spatial regions;

[0038] Figure 3 This is a schematic diagram of the residual spatial attention module in an embodiment of a dynamic attention scheduling method for collaboratively filtering feature channels and spatial regions;

[0039] Figure 4 This is a schematic diagram of the dynamically adjusted attention module in an embodiment of a dynamic attention scheduling method for collaboratively filtering feature channels and spatial regions. Detailed Implementation

[0040] The present invention will be explained in more detail through the following embodiments. The purpose of disclosing the present invention is to protect all changes and modifications within the scope of the present invention. The present invention is not limited to the following embodiments.

[0041] Example 1

[0042] like Figure 1 As shown, the present invention provides a dynamic attention scheduling method for collaboratively filtering feature channels and spatial regions, the steps of which are as follows:

[0043] S1. First, construct a multi-scale fusion-residual channel attention module that collaboratively filters feature channels and spatial region attention, such as... Figure 2As shown, global max pooling and global average pooling operations are performed on the feature map corresponding to each channel of the input feature map x. The specific process of global average pooling in this embodiment is as follows: The input feature map x is subjected to two average pooling operations. The first average pooling reduces the spatial dimension of each channel of the output feature map to 1×1, resulting in an output dimension of [b,c,1,1]; the second average pooling reduces the spatial dimension of each channel to 2x2, resulting in an output dimension of [b,c,2,2]. The two feature maps obtained by the first and second average pooling operations are then dimensionality-reduced to obtain two feature vectors with dimensions of [b,c,1] and [b,c,4], respectively. These two feature vectors are concatenated to obtain a feature vector with dimension [b,c,5]. This feature vector is then dimensionality-increased to obtain a feature map with dimension [b,5c,1,1]. The same operation is then performed on the input feature map x using global max pooling to obtain a feature map with dimension [b,5c,1,1]. The two feature maps obtained through global max pooling and global average pooling operations are added together to obtain a feature map with dimensions [b, 5c, 1, 1]. This feature map is then passed through a fully connected layer to restore the channel dimensions, resulting in a feature map with dimensions [b, c, 1, 1]. This feature map is then passed through a sigmoid activation function and a softmax activation function to obtain the final output feature map generated by the residual space attention module.

[0044] Set hyperparameters As a percentage of important feature channels, the number of important feature channels As shown in the following formula:

[0045] ;

[0046] in, This represents the total number of feature channels.

[0047] Non-essential feature channel C ui The definition is as follows:

[0048] ;

[0049] Set a learnable parameter , It is a value The coefficient is applied to the non-essential feature channels to suppress information from these channels, resulting in an output feature map that suppresses information from the non-essential feature channels. .

[0050] In addition, two trainable coefficients were introduced. and The range of values ​​is The final output feature map generated by the multi-scale fusion-residual channel attention module As shown in the following formula:

[0051] .

[0052] Then, a residual spatial attention module is constructed to collaboratively filter feature channels and spatial region attention, such as... Figure 3 As shown, the maximum value (max pooling) is calculated at each spatial location along dimension 1 (i.e., channel dimension), and the average value (average pooling) is calculated at each spatial location along dimension 1 (i.e., channel dimension). The two tensors are concatenated to obtain an intermediate feature map. This feature map is then convolved (channel dimensionality reduction) and passed through a sigmoid activation function to obtain spatial attention.

[0053] Set hyperparameters As the proportion of elements in important spatial regions, this parameter can be used to control the number of elements in non-important spatial regions, as shown in the following formula:

[0054] ;

[0055] ;

[0056] In the formula, , , representing the number of elements in important spatial regions and non-important spatial regions, respectively; S represents the total number of elements in the spatial regions. , The width of the feature map, The height of the feature map.

[0057] Set a learnable parameter , It is a coefficient with a value of (0,1). This coefficient acts on the non-essential spatial channels to suppress information from these channels, resulting in an output feature map that suppresses information from the non-essential spatial channels. .

[0058] In addition, a trainable coefficient is introduced. The values ​​are all in the range of (0,1), and the final output feature map generated by the residual space attention module is... As shown in the following formula:

[0059] .

[0060] S2. Construct a dynamically adjustable attention module, such as Figure 4As shown, a YOLOv8 object detection model is defined that embeds collaboratively selected feature channels, spatial region attention, and a compression-excitation module. The YOLOv8 model is trained on large-scale data using a standard training procedure. The performance of each model on the validation set is recorded, and the weight file with the best performance is selected. In other embodiments, the YOLOv8 model can use other existing neural network models, such as ResNet152 and ResNet18; the compression-excitation module can also be replaced by other attention mechanisms suitable for computer vision.

[0061] Define a periodicity parameter β, for example, set to 5, indicating that the attention mechanism is switched and frozen every 5 training epochs. Load the weight file trained on the large-scale dataset in the above steps into the corresponding attention mechanism in the YOLOv8 model. Train on a small-scale dataset. During training, decide which attention mechanism to freeze based on the current epoch (e.g., use the collaborative selection of feature channels and spatial region attention module for the first 5 epochs, freeze the compression-activation module; use the compression-activation module for the next 5 epochs, freeze the collaborative selection of feature channels and spatial region attention module, and so on). Train according to the dynamic adjustment strategy. After each epoch, evaluate the model's performance on the validation set and record the best performance.

[0062] Example 2

[0063] This invention also provides a dynamic attention scheduling method for collaboratively selecting feature channels and spatial regions. It employs a structure of YOLOv8 + collaborative selection of feature channels and spatial regions attention + compression-activation module, trained on the COCO dataset (a large-scale dataset) with 100 iterations. Simultaneously, four sets of models were set up for comparative analysis: YOLOv8 (Model 1), YOLOv8 + collaborative selection of feature channels and spatial regions attention + compression-activation module (Model 2: the two modules are never frozen, and the weights of the two modules trained on the large dataset are loaded into this model), YOLOv8 + collaborative selection of feature channels and spatial regions attention + compression-activation module (Model 3: the two modules are always frozen, and the weights of the two modules trained on the large dataset are loaded into this model), and YOLOv8 + collaborative selection of feature channels and spatial regions attention + compression-activation module (Model 4: the two modules are alternately frozen, and the weights of the two modules trained on the large dataset are loaded into this model). On the GlobalWheat2020 dataset, the four pre-configured models were trained using the same random seed. In this embodiment, the model used was YOLOv8n, and the number of iterations was set to 100. The experimental results are shown in Table 1.

[0064] Table 1 Experimental results of Models 1 to 4

[0065]

[0066] As shown in Table 1, Model 4 performs best on both mAP50 and mAP50-95, reaching 97.45% and 64.35% respectively, significantly higher than other models. The dynamically adjusted attention mechanism (Model 4) outperforms the static configuration (Models 2 and 3) on small datasets, indicating that the dynamic adjustment strategy can better adapt to the characteristics of different datasets and improve the model's generalization ability. Model 2 also shows improved performance when both modules are never frozen, but not as much as Model 4, indicating that the attention mechanism can significantly improve model performance and is necessary during training. Model 3, with both modules always frozen, performs slightly better than the base model (Model 1), suggesting that parameters trained on large datasets have better performance.

[0067] Experimental results show that the YOLOv8 object detection model, which embeds collaboratively selected feature channels, spatial region attention, and compression-excitation modules, can significantly improve model performance through dynamic attention adjustment. Especially on small datasets, the dynamic adjustment strategy better adapts to data characteristics, improving the model's generalization ability and accuracy.

[0068] In another embodiment, a YOLOv8x model was used, and two sets of models were set up for comparative analysis to further illustrate the effectiveness of the proposed collaborative selection of feature channels and spatial region attention, as follows:

[0069] YOLOv8 (Model 5) and YOLOv8+ collaborative feature channel selection and spatial region attention (Model 6) were trained on the COCO dataset with 30 iterations. The experimental results are shown in Table 2.

[0070] Table 2 Experimental results for Model 5 and Model 6

[0071]

[0072] Model 6 outperforms Model 5 in both mAP50 and mAP50-95, improving them by 0.22% and 0.10%, respectively. This demonstrates that the collaborative feature channel selection and spatial region attention proposed in this invention can significantly improve the recognition accuracy and generalization ability of large models. This invention enhances the model's focus on important feature channels through multi-scale feature extraction and residual connections, while suppressing interference from unimportant feature channels. This mechanism enables the model to more accurately select features crucial to the task, thereby improving model performance.

[0073] Therefore, the present invention employs the above-mentioned dynamic attention scheduling method for collaboratively selecting feature channels and spatial regions, which can accurately identify and select the most important features for the decision-making process by utilizing collaboratively selecting feature channels and spatial regions for attention. At the same time, by dynamically adjusting the attention mechanism, the model can periodically switch its focus to different features, which helps to avoid overfitting, improve the model's generalization ability, and does not require complex adjustments to the entire network architecture.

[0074] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A dynamic attention scheduling method for collaboratively selecting feature channels and spatial regions, characterized in that, Includes the following steps: S1. Construct collaborative feature channel and spatial region attention, including a multi-scale fusion-residual channel attention module and a residual spatial attention module. Process the input feature map to generate corresponding channel attention and spatial attention, and set hyperparameters for each. and This is used to control the number of important feature channels and the proportion of elements in important spatial regions, and to set parameters. and Adjust the attention weights for unimportant feature channels and unimportant spatial regions; S2. Construct a dynamically adjustable attention module, embedding collaborative feature channel selection and spatial region attention as the first attention mechanism into the neural network, and the neural network includes a second attention mechanism; By setting the period parameter The neural network is dynamically adjusted to be trained using either a first attention mechanism or a second attention mechanism to capture important features of the image.

2. The dynamic attention scheduling method for collaboratively selecting feature channels and spatial regions according to claim 1, characterized in that, In S1, the multi-scale fusion-residual channel attention module performs global max pooling and global average pooling operations on each channel of the input feature map to obtain the first channel feature map and the second channel feature map. The two are added together and then passed through a fully connected layer, a sigmoid activation function and a softmax activation function to obtain the channel attention. Global average pooling includes first average pooling and second average pooling. The first average pooling reduces the spatial dimension of each channel to 1.

1. The second average pooling reduces the spatial dimension of each channel to 2.

2. The two dimensions are obtained as follows: and The feature vectors are then subjected to dimensionality reduction and concatenation to obtain a feature vector with dimension . The feature vectors are then subjected to dimensionality upscaling to obtain a dimension of... The first feature map; correspondingly, global max pooling includes first max pooling and second max pooling, which, after dimensionality reduction, concatenation, and dimensionality increase, yields a dimension of The second feature map.

3. The dynamic attention scheduling method for collaboratively selecting feature channels and spatial regions according to claim 1, characterized in that, In S1, the residual spatial attention module calculates the maximum and average values ​​of each spatial location along the channel dimension, resulting in a first spatial feature map and a second spatial feature map. After concatenating the two, the spatial attention is obtained by channel dimensionality reduction and the Sigmoid activation function.

4. The dynamic attention scheduling method for collaboratively selecting feature channels and spatial regions according to claim 1, characterized in that, In S1, this includes through hyperparameters and parameters After adjusting the channel attention, an output feature map is obtained by suppressing information from non-essential feature channels. ; Then, training coefficients are introduced. and , fusion output feature map and input feature map ,as follows: ； In the formula, The final output feature map generated by the multi-scale fusion-residual channel attention module; Through hyperparameters and parameters After adjusting the spatial attention, an output feature map is obtained that suppresses unimportant spatial information. And introduce training coefficients , fusion output feature map Output feature map generated by multi-scale fusion - residual channel attention module ,as follows: ； In the formula, This is the final output feature map generated by the residual space attention module.

5. The dynamic attention scheduling method for collaboratively selecting feature channels and spatial regions according to claim 1, characterized in that, In S2, hyperparameters The percentage of important feature channels is used to control the number of non-important feature channels, as follows: ； ； In the formula, , The number of important feature channels and the number of unimportant feature channels, respectively. This represents the total number of feature channels.

6. The dynamic attention scheduling method for collaboratively selecting feature channels and spatial regions according to claim 1, characterized in that, Hyperparameters The percentage of elements in important spatial regions is used to control the number of elements in non-important spatial regions, as follows: ； ； In the formula, , The number of elements in important spatial regions and non-important spatial regions, respectively; S represents the total number of elements in the spatial region. , The width of the feature map, The height of the feature map.

7. The dynamic attention scheduling method for collaboratively selecting feature channels and spatial regions according to claim 1, characterized in that, In S3, the available neural networks include YOLOv8, ResNet152, and ResNet18.

8. The dynamic attention scheduling method for collaboratively selecting feature channels and spatial regions according to claim 1, characterized in that, In S3, the alternative secondary attention mechanism includes a compression-excitation module.

9. The dynamic attention scheduling method for collaboratively selecting feature channels and spatial regions according to claim 1, characterized in that, Within each cycle, the weights of the selected first or second attention mechanism are trained, and the weights of the unselected second or first attention mechanism are frozen.

Images