An attention method and device for adaptively fusing multi-scale large nucleus features

Abstract

The application discloses an attention method and device for adaptively fusing multi-scale large kernel features. The method constructs multiple large convolution kernel feature extraction branches with different convolution kernel sizes, performs multi-scale feature extraction on the convolution features of an input image, and obtains multi-scale large kernel features. The multi-scale large kernel features are spliced along the channel dimension, adaptive spatial selection weights are generated through pooling, convolution and activation function operations, the adaptive spatial selection weights of each branch are multiplied element by element with the large kernel features of the corresponding branch, and are fused to obtain fused attention weights. The fused attention weights are multiplied element by element with input features to obtain enhanced features of the input image. The attention method adaptively fuses attention weights of different scales, effectively fuses multi-scale information, enhances the features of a detection target in the input image, suppresses irrelevant features, and improves the performance of target detection.

Description

Technical Field

[0001] This invention belongs to the field of deep learning technology, specifically relating to an adaptive attention method and apparatus for fusing multi-scale large kernel features. Background Technology

[0002] Convolutional neural networks (CNNs) have been widely applied in fields such as visual perception-based object detection and recognition. Attention mechanisms are a simple and efficient way to enhance the representation of convolutional features in images. Traditional attention mechanisms, by weighting convolutional features, can adaptively select and enhance the most relevant features while suppressing redundant or irrelevant features, but they struggle to effectively capture cross-scale and cross-regional feature information in images. Multi-scale attention mechanisms, by designing multi-scale path structures with different convolutional kernel sizes, generate multi-scale attention weights, enabling the simultaneous extraction of features from multiple scales and different receptive fields, capturing image context information from a larger area, thus improving the model's feature representation ability. However, treating attention weights of different scales equally during fusion limits the network's feature representation capability. Therefore, adaptively fusing attention weights of different scales can enhance the representation of convolutional features in images and improve the ability of convolutional neural networks to detect targets. Summary of the Invention

[0003] The main objective of this invention is to overcome the shortcomings and deficiencies of existing technologies and propose an adaptive attention method and apparatus for fusing multi-scale large kernel features. This method extracts multi-scale features from the input image's convolutional features by constructing multiple large convolutional kernel feature extraction branches with different kernel sizes, and adaptively fuses attention weights of different scales to generate fused attention weights. This technique can effectively fuse multi-scale information, suppress redundant features, enhance the image's convolutional feature representation capability, and improve the convolutional neural network's ability to detect targets.

[0004] To achieve the first objective mentioned above, this invention proposes an adaptive attention method that fuses multi-scale large kernel features. The attention method that fuses multi-scale large kernel features includes the following steps: S1. Obtain input feature P, wherein the input feature P is the convolutional feature obtained by passing the image through a convolutional neural network; S2. Perform a convolution operation on the input feature P, and then use N parallel convolution kernel feature extraction branches with different kernel sizes to perform multi-scale feature extraction on the features output by the convolution operation, obtaining N large kernel features M0, M1, ..., Mn at different scales. i ..., M N-1 , where N is the number of feature extraction branches of the large convolution kernel, N≥2; S3. Perform tensor concatenation of the N large-kernel features at different scales along the channel dimension to obtain the large-kernel multi-scale concatenated feature M.all Subsequently, pooling, convolution, and activation function operations are performed on the large-kernel multi-scale spliced ​​features to obtain adaptive spatial selection weights W0, W1, ..., Wn for N branches. i ..., W N-1 ; S4. Multiply the adaptive spatial selection weights of the N branches element-wise with all channels of the large kernel feature of the corresponding branch to obtain the attention weights WA0, WA1, ..., WA at N scales. i , ..., WA N-1 ; S5. Fuse the attention weights at N scales, and perform a convolution operation on the fused result to obtain the final fused attention weight W. Fuse ; S6, Integrate attention weights W Fuse The image enhancement feature F is obtained by multiplying the input feature P element by element. The image enhancement feature F is used to enhance the features of the detected target in the image and suppress irrelevant features.

[0005] Furthermore, in step S2, a convolution operation is performed on the input feature P. Then, using N parallel convolutional kernel feature extraction branches, multi-scale feature extraction is performed on the features output by the convolution operation to obtain N large kernel features M0, M1, ..., Mn at different scales. i ..., M N-1 Each of the N large convolutional kernel feature extraction branches performs a convolution operation with a large convolutional kernel. The kernel size of each branch is different, and the kernel size of the i-th branch is K. i ×K i Satisfying: K0 <K1<... K i <... <K N-1 Convolutional operations with different kernel sizes can extract features at different scales and capture contextual information of different ranges; convolutional operations with large kernels can capture a wider range of contextual information; N parallel convolutional kernel feature extraction branches can extract N features at different scales simultaneously. The large kernel features of the i-th large convolutional kernel feature extraction branch in step S2 ; in, ( ) represents the convolution operation of the i-th large convolution kernel feature extraction branch.

[0006] Furthermore, in step S3, the N large-kernel features of different scales are tensor-concatenated along the channel dimension to obtain the large-kernel multi-scale concatenated feature M. allThe large-kernel multi-scale splicing feature contains large-kernel features of different scales. Then, max pooling and average pooling operations are performed on the large-kernel multi-scale splicing feature along the channel dimension to generate max pooling and average pooling results. The max pooling and average pooling results are then tensor-concatenated along the channel dimension to obtain the large-kernel multi-scale pooling feature M. pool Pooling operations compress the channel dimension of the large-kernel multi-scale spliced ​​features; different pooling operations can ensure the diversity of compressed features; subsequently, convolution operations are performed on the large-kernel multi-scale pooled features. The process outputs the intermediate features of N branches; then, it performs non-linear activation on the intermediate features of the N branches using an activation function to obtain the adaptive spatial selection weights W0, W1, ..., Wn of the N branches. i ..., W N-1 The convolution operation The output channels are N, and each channel corresponds to the intermediate features of a branch. The activation function operation uses the Sigmoid activation function to generate weights in the [0, 1] interval, or uses the Softmax function to normalize along the channel dimension to generate weights in the [0, 1] interval. The activation function operation restricts the values ​​of the adaptive spatial selection weights to the interval between 0 and 1. Through the above operation, the adaptive spatial selection weights of N branches are adaptively calculated according to the large kernel features of different scales. Large-kernel multi-scale splicing features ; Large kernel multi-scale pooling features ; Adaptive spatial selection weights ; in, ( ) represents the activation function operation. [ ] indicates tensor concatenation along the channel dimension. ( ) indicates a max pooling operation. ( ) indicates the average pooling operation.

[0007] Furthermore, the attention weights at the i-th scale in step S4 ; The operator sign indicates that all channels are multiplied element-wise; by using an adaptive space of different scales to select weights, the kernel features of different scales are weighted separately, and attention weights of different scales are calculated. Attention weights of different scales can be adaptively generated according to the input features.

[0008] Furthermore, in step S5, the attention weights of N scales are fused using an element-wise addition method, and the fusion result is then subjected to a convolution operation. The final fusion attention weights are obtained. Integrating attention weights The calculation formula is as follows: Fusion attention weights By fusing N attention weights at different scales, the fused attention weights can adaptively utilize features at different scales while capturing contextual information of different ranges.

[0009] Furthermore, the image enhancement features output in step S6 ; Among them, the operator This represents element-wise multiplication; the fusion of attention weights applies different weights to different parts of the input feature P, enhancing the feature information of the target, suppressing redundant or irrelevant features, and improving the accuracy and performance of target detection.

[0010] Furthermore, the output image enhancement feature F has the same dimension as the input feature P; the adaptive attention method for fusing multi-scale large kernel features can be integrated as an independent module into various convolutional neural network structures. The input feature P has a dimension of C×H×W, and the large kernel features M0, M1, ..., M of N branches are... i ..., M N-1 All dimensions are C×H×W, with large-kernel multi-scale splicing features M. all The dimensions are NC×H×W, and the large-kernel multi-scale pooling feature M pool The dimension is 2×H×W, and the adaptive spatial selection weights W0, W1, ..., W of the N branches are... i ..., W N-1 The dimension is 1×H×W, and the attention weights are WA0, WA1, ..., WA2. i , ..., WA N-1 The dimensions are C×H×W, and the attention weights W are fused. Fuse The dimensions of the input feature are C×H×W, and the dimensions of the output feature F are C×H×W; where C is the number of input feature channels, H is the height of the input feature, and W is the width of the input feature; the output image enhancement feature F has the same dimensions C×H×W as the input feature P.

[0011] To achieve the second objective mentioned above, this invention proposes an adaptive attention device for fusing multi-scale large kernel features, used to execute the aforementioned adaptive attention method for fusing multi-scale large kernel features. The attention device for fusing multi-scale large kernel features includes: The feature input module is used to obtain input features P, wherein the input features P are convolutional features obtained by passing the image through a convolutional neural network; The multi-scale feature extraction module performs convolution operations on the input feature P, and then uses N parallel convolution kernel feature extraction branches with different kernel sizes to perform multi-scale feature extraction on the features output by the convolution operation, obtaining N large kernel features M0, M1, ..., Mn at different scales. i ..., M N-1 , where N is the number of feature extraction branches of the large convolution kernel, N≥2; The adaptive spatial selection weight calculation module is used to perform tensor concatenation of the N large-kernel features at different scales along the channel dimension to obtain the large-kernel multi-scale concatenated feature M. all Subsequently, pooling, convolution, and activation function operations are performed on the large-kernel multi-scale spliced ​​features to obtain adaptive spatial selection weights W0, W1, ..., Wn for N branches. i ..., W N-1 ; The attention weight calculation module is used to perform element-wise multiplication of the adaptive spatial selection weights of N branches with all channels of the large kernel feature of the corresponding branch to obtain attention weights WA0, WA1, ..., WA at N scales. i , ..., WA N-1 ; The attention weight fusion calculation module is used to fuse attention weights at N scales, perform a convolution operation on the fusion result, and obtain the final fused attention weight W. Fuse ; The image enhancement feature calculation module is used to fuse attention weights W. Fuse The image enhancement feature F is obtained by multiplying the input feature P element by element. The image enhancement feature F is used to enhance the features of the detected target in the image and suppress irrelevant features.

[0012] Compared with the prior art, the present invention has the following advantages and beneficial effects: (1) This invention proposes an adaptive attention method for fusing multi-scale large kernel features. By calculating the adaptive spatial selection weights of large kernel features at different scales, the fusion attention weights are generated. The feature information at different scales is adaptively fused, while capturing context information at different ranges. Different parts of the input image convolution features are weighted to enhance the features of the target, effectively enhance the representation ability of the image convolution features, and improve the accuracy and performance of target detection.

[0013] (2) The adaptive attention device for fusing multi-scale large kernel features proposed in this invention can be used as an independent module. The input and output of the independent module have the same dimension, and it can be seamlessly integrated into various convolutional neural network structures, thus having wide applicability. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a schematic diagram of the adaptive attention method for fusing multi-scale large kernel features disclosed in this invention; Figure 2 This is a schematic diagram of the lane line detection network model structure according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of the large convolutional kernel feature extraction branch in an embodiment of the present invention; Figure 4 This is a schematic diagram of lane line detection results in an embodiment of the present invention where the method of the present invention is not used; Figure 5 This is a schematic diagram of the lane line detection results using the method of the present invention in Embodiment 2 of the present invention; Figure 6 This is a schematic diagram of an attention device structure for adaptive fusion of multi-scale large kernel features disclosed in Embodiment 3 of the present invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Example 1 This embodiment discloses an adaptive attention method for fusing multi-scale large kernel features for lane line detection; in lane line detection, the detection target is the lane lines in the image; the lane line detection network is as follows: Figure 2 As shown, the lane detection network uses the CLRNet network framework, including a backbone network, a neck network, and a detection head network connected sequentially. The detection head network includes parallel-connected classification and regression branches. The backbone network, neck network, and detection head network all adopt the CLRNet network structure, with the backbone network using a ResNet18 network and the neck network using a Feature Pyramid Network (FPN). The attention module structure, implemented based on an adaptive multi-scale large-kernel feature fusion attention method, is shown below. Figure 1As shown, the attention module is inserted between the backbone and neck network of the CLRNet network in the form of a residual structure. The output features of the attention module are added to the input features of the attention module to form the output features of the inserted module, thus constituting the attention module residual structure and solving the network degradation problem during network training. The backbone network extracts convolutional features from the input lane line image, enhances the features through the inserted attention module residual structure, and then completes the lane line detection task through the neck network and the detection head network. Specifically, it includes the following steps: S1. Obtain the input feature P of the attention module. The input feature P is the convolutional feature obtained by passing the lane line image through the backbone network ResNet18. The dimension of the input feature P is C×H×W, where C is the number of input feature channels, H is the height of the input feature, and W is the width of the input feature. S2. Perform a convolution operation on the input feature P. Then, using N parallel convolutional kernel feature extraction branches, multi-scale feature extraction is performed on the features output by the convolution operation to obtain N large kernel features M0, M1, ..., Mn at different scales. i ..., M N-1 Each of the N large convolutional kernel feature extraction branches performs a convolution operation with a large convolutional kernel. The kernel size of each branch is different, and the kernel size of the i-th branch is K. i ×K i To reduce computational complexity and the number of parameters, in this embodiment, the size of the convolution kernel for the i-th branch is K. i ×K i The convolution operation consists of two concatenated, mutually perpendicular strip depthwise convolution operations. , and pointwise convolution operation Implement the strip depthwise convolution operation for the i-th branch. and The kernel sizes are K respectively. i ×1 and 1×K i Pointwise convolution operation The convolution kernel size is 1×1. In this embodiment, N=3, K0=1, K1=7, K2=11, and the convolution operation... This is a depthwise convolution operation with a kernel size of 5×5; Among them, the large kernel features of the i-th large convolution kernel feature extraction branch ; in, ( ) represents the convolution operation of the i-th large convolution kernel feature extraction branch, where the large kernel scale features of the N branches are M0, M1, ..., M i ..., M N-1The dimensions are all C×H×W; S3. Perform tensor concatenation of the N large-kernel features at different scales along the channel dimension to obtain the large-kernel multi-scale concatenated feature M. all M all The dimensions are NC×H×W. Then, max pooling and average pooling operations are performed on the large-kernel multi-scale concatenated feature along the channel dimension to generate max pooling and average pooling results. The max pooling and average pooling results are then tensor-concatenated along the channel dimension to obtain the large-kernel multi-scale pooling feature M. pool M pool The dimension is 2×H×W; then convolution operations are performed on the large kernel multi-scale pooling features. The process outputs the intermediate features of N branches; then, it performs non-linear activation on the intermediate features of the N branches using an activation function to obtain the adaptive spatial selection weights W0, W1, ..., Wn of the N branches. i ..., W N-1 The adaptive spatial selection weights for the N branches all have a dimension of 1×H×W; the convolution operation The output channels are N, and each channel corresponds to the intermediate feature of a branch; in this embodiment, the convolution operation... For a depthwise convolution operation with a kernel size of 7×7, the activation function operation uses the Sigmoid activation function to generate weights in the [0, 1] interval; Large-kernel multi-scale splicing features ; Large kernel multi-scale pooling features ; Adaptive spatial selection weights ; in, This represents the Sigmoid activation function operation. Tensor splicing representing the channel dimension. This indicates a max pooling operation. This indicates an average pooling operation; S4. Multiply the adaptive spatial selection weights of the N branches element-wise with all channels of the large kernel feature of the corresponding branch to obtain the attention weights WA0, WA1, ..., WA at N scales. i , ..., WA N-1 The dimensions of the attention weights at N scales are all C×H×W; Attention weights at the i-th scale ; The operator symbol indicates that all channels are multiplied element-wise; S5. Fuse the attention weights of N scales using an element-wise addition method, and then perform a convolution operation on the fused result. The final fusion attention weights are obtained. Integrate attention weights W Fuse The dimensions are C×H×W; in this embodiment, the convolution operation... This is a pointwise convolution operation with a kernel size of 1×1; Fusion attention weights ; S6, Integrate attention weights W Fuse The image enhancement feature F is obtained by multiplying the input feature P element-wise. The dimension of the enhancement feature F is C×H×W. The enhancement feature is used to enhance the lane line features in the image and suppress irrelevant features. Enhanced features output by the attention module ; The operator ⊙ represents element-wise multiplication. In this embodiment, the enhanced feature F is added to the input feature P of the attention module and used as the input of the neck network. The final lane detection result is obtained through the classification and regression branches of the neck network and the detection head network. Example 2 Based on Example 1, this example further discloses an adaptive attention method for fusing multi-scale large kernel features for lane detection; the lane detection network is as follows: Figure 2 As shown, the lane detection network uses the CLRNet network framework, including a backbone network, a neck network, and a detection head network connected sequentially. The detection head network includes parallel-connected classification and regression branches. The backbone network, neck network, and detection head network all adopt the CLRNet network structure, with the backbone network using a ResNet18 network and the neck network using a Feature Pyramid Network (FPN). The attention module structure, implemented based on an adaptive multi-scale large-kernel feature fusion attention method, is shown below. Figure 1 As shown, the attention module is inserted between the backbone and neck network of the CLRNet network in the form of a residual structure. The output features of the attention module are added to the input features of the attention module to form the output features of the inserted module, thus constituting the attention module residual structure and solving the network degradation problem during network training. The backbone network extracts convolutional features from the input lane line image, enhances the features through the inserted attention module residual structure, and then completes the lane line detection task through the neck network and the detection head network. Specifically, it includes the following steps: S1. Obtain the input feature P of the attention module. The dimension of the input feature P is C×H×W, where C is the number of input feature channels, H is the height of the input feature, and W is the width of the input feature. For details, please refer to step S1 in Example 1. S2. Perform a convolution operation on the input feature P. Then, using N parallel convolutional kernel feature extraction branches, multi-scale feature extraction is performed on the features output by the convolution operation to obtain N large kernel features M0, M1, ..., Mn at different scales. i ..., M N-1 Each of the N large convolutional kernel feature extraction branches performs a convolution operation with a large convolutional kernel. The kernel size of each branch is different, and the kernel size of the i-th branch is K. i ×K i To reduce computational complexity and the number of parameters, in this embodiment, the size of the convolution kernel for the i-th branch is K. i ×K i The convolution operation consists of two concatenated, mutually perpendicular strip depthwise convolution operations. , and pointwise convolution operation Implement the strip depthwise convolution operation for the i-th branch. and The kernel sizes are K respectively. i ×1 and 1×K i Pointwise convolution operation The convolution kernel size is 1×1. In this embodiment, N=4, K0=1, K1=7, K2=11, K3=21. The convolution operation... This is a depthwise convolution operation with a kernel size of 5×5; Among them, the large kernel features of the i-th large convolution kernel feature extraction branch ; in, ( ) represents the convolution operation of the i-th large convolution kernel feature extraction branch, where the large kernel scale features of the N branches are M0, M1, ..., M i ..., M N-1 The dimensions are all C×H×W; S3. Perform tensor concatenation of the N large-kernel features at different scales along the channel dimension to obtain the large-kernel multi-scale concatenated feature M. all Subsequently, pooling, convolution, and sigmoid activation functions are applied to the large-kernel multi-scale concatenated features to obtain adaptive spatial selection weights W0, W1, ..., Wn for N branches. i ..., W N-1 For details, please refer to step S3 in Example 1; S4. Multiply the adaptive spatial selection weights of the N branches element-wise with all channels of the large kernel feature of the corresponding branch to obtain the attention weights WA0, WA1, ..., WA at N scales. i , ..., WA N-1 For details, please refer to step S4 in Example 1; S5. Fuse the attention weights of N scales using an element-wise addition method, and perform a convolution operation on the fused result to obtain the final fused attention weight W. Fuse For details, please refer to step S5 in Example 1; S6, Integrate attention weights W Fuse The image enhancement feature F is obtained by multiplying the input feature P element by element. The image enhancement feature F is used to enhance the lane line features in the image and suppress irrelevant features. For details, please refer to step S6 in Example 1. In this embodiment, the enhanced feature F is added to the input feature P of the attention module and used as the input of the neck network. The final lane detection result is obtained through the classification and regression branches of the neck network and the detection head network. Lane line detection results for images in the CULane public lane line dataset are as follows: Figure 4 , Figure 5 As shown, Figure 4 The detection results are from the CLRNet network. Figure 5 The detection results after inserting the attention module residual structure from Embodiment 2 of this invention into the CLRNet network show that... Figure 4 Lane markings that were not detected in Figure 5 The results show that the method of this invention can enhance the convolutional features of lane line images and improve the performance of lane line detection. Table 1 compares the experimental results of different methods on the public dataset CULane, with the performance evaluation index being the F1 score. The experimental results show that inserting the attention module residual structure of Embodiment 1 or Embodiment 2 of this invention into the CLRNet network can improve the F1 score of lane line detection. Compared with Embodiment 1, Embodiment 2 adds one more large convolutional kernel feature extraction branch and uses a larger convolutional kernel, which can capture a wider range of contextual information, so its performance is better.

[0018] Table 1. Comparison of experimental results from the examples

[0019] Example 3 like Figure 6 As shown, this embodiment provides an attention device for adaptively fusing multi-scale large kernel features. The attention device includes: a feature input module 601, a multi-scale feature extraction module 602, an adaptive spatial selection weight calculation module 603, an attention weight calculation module 604, a fusion attention weight calculation module 605, and an image enhancement feature calculation module 606. The specific functions of each module are as follows: The feature input module 601 is used to obtain input feature P, wherein the input feature P is the convolutional feature obtained by passing the image through a convolutional neural network; The multi-scale feature extraction module 602 performs convolution operations on the input feature P, and then uses N parallel convolution kernel feature extraction branches with different kernel sizes to perform multi-scale feature extraction on the features output by the convolution operation, obtaining N large kernel features M0, M1, ..., Mn at different scales. i ..., M N-1 , where N is the number of feature extraction branches of the large convolution kernel, N≥2; The adaptive spatial selection weight calculation module 603 is used to perform tensor concatenation of the N large kernel features of different scales along the channel dimension to obtain the large kernel multi-scale concatenated feature M. all Subsequently, pooling, convolution, and activation function operations are performed on the large-kernel multi-scale spliced ​​features to obtain adaptive spatial selection weights W0, W1, ..., Wn for N branches. i ..., W N-1 ; Attention weight calculation module 604 is used to perform element-wise multiplication of the adaptive spatial selection weights of N branches with all channels of the large kernel feature of the corresponding branch to obtain attention weights WA0, WA1, ..., WA at N scales. i , ..., WA N-1 ; The attention weight calculation module 605 is used to fuse attention weights at N scales, perform a convolution operation on the fused result, and obtain the final fused attention weight W. Fuse ; Image enhancement feature calculation module 606 is used to calculate the fused attention weights W. Fuse The image enhancement feature F is obtained by multiplying the input feature P element by element. The image enhancement feature F is used to enhance the features of the detected target in the image and suppress irrelevant features.

[0020] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0021] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. An adaptive attention method for fusing multi-scale large kernel features, characterized in that, The adaptive attention method for fusing multi-scale large kernel features includes the following steps: S1. Obtain input feature P, wherein the input feature P is the convolutional feature obtained by passing the image through a convolutional neural network; S2. Perform a convolution operation on the input feature P, and then use N parallel convolution kernel feature extraction branches with different kernel sizes to perform multi-scale feature extraction on the features output by the convolution operation, obtaining N large kernel features M0, M1, ..., Mn at different scales. i ..., M N-1 , where N is the number of feature extraction branches of the large convolution kernel, N≥2; S3. Perform tensor concatenation of the N large-kernel features at different scales along the channel dimension to obtain the large-kernel multi-scale concatenated feature M. all Subsequently, pooling, convolution, and activation function operations are performed on the large-kernel multi-scale spliced ​​features to obtain adaptive spatial selection weights W0, W1, ..., Wn for N branches. i ..., W N-1 ; Specifically, the N large-kernel features of different scales are tensor-concatenated along the channel dimension to obtain the large-kernel multi-scale concatenated feature M. all Subsequently, max pooling and average pooling operations are performed on the large-kernel multi-scale spliced ​​features along the channel dimension to generate max pooling and average pooling results. The max pooling and average pooling results are then tensor-concatenated along the channel dimension to obtain the large-kernel multi-scale pooled feature M. pool Subsequently, convolution operations are performed on the large-kernel multi-scale pooling features. The process outputs the intermediate features of N branches; then, it performs non-linear activation on the intermediate features of the N branches using an activation function to obtain the adaptive spatial selection weights W0, W1, ..., Wn of the N branches. i ..., W N-1 The convolution operation The output channels are N, and each channel corresponds to the intermediate features of a branch; the activation function operation uses the Sigmoid activation function to generate weights in the [0, 1] interval, or uses the Softmax function to normalize along the channel dimension to generate weights in the [0, 1] interval. Large kernel multi-scale splicing features ; Large kernel multi-scale pooling features ; Adaptive spatial selection weights ; in, ( ) represents the activation function operation. [ ] indicates tensor concatenation along the channel dimension. ( ) indicates a max pooling operation. ( ) indicates average pooling operation; S4. Multiply the adaptive spatial selection weights of the N branches element-wise with all channels of the large kernel feature of the corresponding branch to obtain the attention weights WA0, WA1, ..., WA at N scales. i , ..., WA N-1 ; S5. Fuse the attention weights at N scales, and perform a convolution operation on the fused result to obtain the final fused attention weight W. Fuse The attention weights at N scales are fused using an element-wise addition method, and the fusion result is then subjected to a convolution operation. The final fusion attention weights are obtained. Integrating attention weights The calculation formula is as follows: Fusion attention weights ; S6, Integrate attention weights W Fuse The image enhancement feature F is obtained by element-wise multiplication with the input feature P, where the image enhancement feature is... Operators This represents element-wise multiplication. The image enhancement feature F is used to enhance the features of the detected target in the image and suppress irrelevant features.

2. The attention method for adaptively fusing multi-scale large kernel features according to claim 1, characterized in that, In step S2, a convolution operation is performed on the input feature P. Then, using N parallel convolutional kernel feature extraction branches, multi-scale feature extraction is performed on the features output by the convolution operation to obtain N large kernel features M0, M1, ..., Mn at different scales. i ..., M N-1 Each of the N large convolutional kernel feature extraction branches performs a convolution operation with a large convolutional kernel. The kernel size of each branch is different, and the kernel size of the i-th branch is K. i ×K i Satisfying: K0 <K1<... K i <... <K N-1 ; The large kernel features of the i-th large convolutional kernel feature extraction branch in step S2 ; in, ( ) represents the convolution operation of the i-th large convolution kernel feature extraction branch.

3. The attention method for adaptively fusing multi-scale large kernel features according to claim 1, characterized in that, The attention weights at the i-th scale in step S4 ; The operator symbol indicates that all channels are multiplied element-wise.

4. The attention method for adaptively fusing multi-scale large kernel features according to claim 1, characterized in that, The output image enhancement feature F has the same dimension as the input feature P; The input feature P has a dimension of C×H×W, and the large kernel features M0, M1, ..., M of N branches are... i ..., M N-1 All dimensions are C×H×W, with large-kernel multi-scale splicing features M. all The dimensions are NC×H×W, and the large-kernel multi-scale pooling feature M pool The dimension is 2×H×W, and the adaptive spatial selection weights W0, W1, ..., W of the N branches are... i ..., W N-1 The dimension is 1×H×W, and the attention weights are WA0, WA1, ..., WA2. i , ..., WA N-1 The dimensions are C×H×W, and the attention weights W are fused. Fuse The dimensions of the input feature are C×H×W, and the dimensions of the output feature F are C×H×W; where C is the number of input feature channels, H is the height of the input feature, and W is the width of the input feature.

5. An adaptive attention device for fusing multi-scale large kernel features, used to execute the adaptive attention method for fusing multi-scale large kernel features as described in any one of claims 1 to 4, characterized in that, The adaptive attention device for fusing multi-scale large kernel features includes: The feature input module is used to obtain input features P, which are convolutional features obtained by passing the image through a convolutional neural network. The multi-scale feature extraction module performs convolution operations on the input feature P, and then uses N parallel convolution kernel feature extraction branches with different kernel sizes to perform multi-scale feature extraction on the features output by the convolution operation, obtaining N large kernel features M0, M1, ..., Mn at different scales. i ..., M N-1 , where N is the number of feature extraction branches of the large convolution kernel, N≥2; The adaptive spatial selection weight calculation module is used to perform tensor concatenation of the N large-kernel features at different scales along the channel dimension to obtain the large-kernel multi-scale concatenated feature M. all Subsequently, pooling, convolution, and activation function operations are performed on the large-kernel multi-scale spliced ​​features to obtain adaptive spatial selection weights W0, W1, ..., Wn for N branches. i ..., W N-1 ; The attention weight calculation module is used to perform element-wise multiplication of the adaptive spatial selection weights of N branches with all channels of the large kernel feature of the corresponding branch to obtain attention weights WA0, WA1, ..., WA at N scales. i , ..., WA N-1 ; The attention weight fusion calculation module is used to fuse attention weights at N scales, perform a convolution operation on the fusion result, and obtain the final fused attention weight W. Fuse ; The image enhancement feature calculation module is used to fuse attention weights W. Fuse The image enhancement feature F is obtained by multiplying the input feature P element by element. The image enhancement feature F is used to enhance the features of the detected target in the image and suppress irrelevant features.

Images