Hyperspectral image classification method based on spectral-spatial attention mechanism and residual network

Abstract

The present invention relates to a hyperspectral image classification method based on spectral-spatial attention mechanism and residual network, comprising: inputting a hyperspectral image; using principal component analysis to reduce dimension to obtain a hyperspectral image after dimension reduction; performing preliminary feature extraction to obtain an image after feature extraction; obtaining an enhanced image; obtaining an image for further mining spatial features; obtaining an image after adaptive calibration; subjecting the image after adaptive calibration to global pooling, flattening, batch normalization and a fully connected layer to obtain a final classification result. The present invention uses a depth-separable convolution block to achieve lightweight and efficient feature extraction; the residual SEA attention module obtains global information through longitudinal and lateral squeezing, and obtains local information through enhancement; the residual EMA attention module can obtain paired relationships between pixels to better mine spatial features; the residual SE attention module can adaptively calibrate channel information to better obtain channel information.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and image processing, and in particular to a hyperspectral image classification method based on a spectral-spatial attention mechanism and a residual network. Background Art

[0002] Each pixel in a hyperspectral remote sensing image has hundreds of narrow spectral bands from ultraviolet to infrared, which contains the spectral and spatial information of the imaging target. It has been widely used in urban construction, ocean observation, precision agriculture, disaster prevention and mitigation, resource exploration and other fields.

[0003] In the early days, hyperspectral image classification was mainly based on machine learning, such as random minimum spanning forest, support vector machine, local covariance matrix representation and linear discriminant method. However, these classification methods have the problem of insufficient feature extraction ability and poor applicability to hyperspectral image scenes. In recent years, deep learning technology has shown strong discrimination and learning capabilities in hyperspectral image classification tasks. In particular, the attention mechanism can help the model accurately focus on the key areas in the image, and has achieved good performance in image classification, target detection, semantic segmentation and other fields. In order to suppress the interference of non-targets in hyperspectral images on target feature learning and enable the network model to learn spatial spectral joint features from the target area, some scholars have found that the introduction of attention mechanism can selectively focus on significant parts instead of treating each part equally. Therefore, CNN is combined with the attention mechanism and applied in the field of hyperspectral, and significant improvement has been achieved.

[0004] However, hyperspectral images have redundancy due to the excessive number of bands. At the same time, the spatial pixels of hyperspectral images have differences in importance and there are correlations between pixels, which further affect the accuracy of classification. In addition, deep networks also face the problem of gradient vanishing and explosion when processing data, which will also have an adverse effect on the classification results. Summary of the invention

[0005] In order to solve the problem that spectral bands have large redundancy and each band and each pixel contributes differently to classification, the purpose of the present invention is to provide a hyperspectral image classification method based on spectral-spatial attention mechanism and residual network, which can adaptively select bands, mine the pairwise correlation of spatial pixels and has low computational complexity.

[0006] To achieve the above object, the present invention adopts the following technical solution: a hyperspectral image classification method based on spectral-spatial attention mechanism and residual network, the method comprising the following steps in order:

[0007] (1) Input hyperspectral image H∈R h×w×l, where h, w, and l represent the length, width, and number of bands of the hyperspectral image, respectively;

[0008] (2) For the input hyperspectral image H∈R h×w×l Use principal component analysis to reduce the dimension and obtain the hyperspectral image H after dimension reduction. PCA ∈R h×w×b , reduce the number of bands from l to b;

[0009] (3) The hyperspectral image H after dimensionality reduction PCA ∈R h×w×b Through two 3×3 convolution blocks, preliminary feature extraction is performed to obtain the image H after feature extraction. 3×3 ; The 3×3 convolution block includes a two-dimensional 3×3 convolution, a batch normalization layer and a Relu activation function;

[0010] (4) The image H after feature extraction 3×3 Through two residual SEA attention modules, axial squeezing is used to extract global semantic information, and then detail enhancement is used to extract local information to obtain the enhanced image H SEA ;

[0011] (5) The enhanced image H SEA Through a residual EMA attention module, the channel weights are recalibrated and the spatial pixel pairwise relationship is captured by using cross-dimensional interactions to obtain the image H that further mines spatial features. EMA ;

[0012] (6) Further mining the spatial features of the image H EMA Through a residual SE attention module, the adaptively calibrated image H is obtained SE ;

[0013] (7) The adaptively calibrated image H SE After global pooling, flattening, batch normalization and fully connected layers, the final classification result is obtained.

[0014] The step (4) specifically means that: the two residual SEA attention modules include a first residual SEA attention module and a second residual SEA attention module, and the first residual SEA attention module and the second residual SEA attention module have the same structure;

[0015] The first residual SEA attention module includes a first depth-separable convolution block and a SEA attention mechanism, wherein the first depth-separable convolution block includes a first 1×1 point-by-point convolution, a first 3×3 channel-by-channel convolution, a first batch of normalization layers, a second batch of normalization layers, a first LeakyRelu activation function, and a second LeakyRelu activation function; the image H after feature extraction 3×3First, through the first batch of normalization layers, 1×1 point-by-point convolution, the first LeakyRelu activation function, and then through the second batch of normalization layers, the first 3×3 channel-by-channel convolution, and the second LeakyRelu activation function, the output X∈R of the first depth-separable convolution block is obtained. H×W×C , where H×W is the space size and C is the number of channels;

[0016] The SEA attention mechanism includes the first 1×1 two-dimensional convolution, the second 1×1 two-dimensional convolution, the third 1×1 two-dimensional convolution, the second 3×3 channel-by-channel convolution, the third batch normalization layer, the fourth batch normalization layer, the Relu6 activation function, and multi-head attention;

[0017] First, perform a squeeze operation: the output of the first depth-separable convolutional block X∈R H×W×C After the first 1×1 two-dimensional convolution, we get First, Q, K, and V are horizontally extruded to obtain At the same time, Q, K, and V are also longitudinally extruded to obtain Q h , K h 、V h Take multi-head attention and also Q v , K v 、V v Taking multiple attentions and then adding them together, we get:

[0018]

[0019] In the formula, is a scaling factor, Y1 1 Through the second 1×1 two-dimensional convolution, the output Y1 of the squeezing operation is obtained;

[0020] Then perform detail enhancement operation: connect Q, K, and V in the channel dimension, and then pass through the second 3×3 channel-by-channel convolution, the third batch of normalization layers, the Relu6 activation function, the third 1×1 two-dimensional convolution, and the fourth batch of normalization layers to obtain the output Y2 of the detail enhancement operation. Multiply Y1 and Y2 to get the output Y of the SEA attention mechanism;

[0021] The output X of the first depth-separable convolutional block and the output Y of the SEA attention mechanism are residually connected to obtain the output of the first residual SEA attention module Similarly, the output of the first residual SEA attention module Input the second residual SEA attention module, and the second residual SEA attention module outputs the enhanced image H SEA .

[0022] The step (5) specifically refers to: the residual EMA attention module includes a second depth-separable convolution block and an EMA attention mechanism, the second depth-separable convolution block includes a second 1×1 point-by-point convolution, a third 3×3 channel-by-channel convolution, a fifth batch normalization layer, a sixth batch normalization layer, a third LeakyRelu activation function, and a fourth LeakyRelu activation function; the enhanced image H SEA First, through the fifth batch of normalization layers, the second 1×1 point-by-point convolution, the third LeakyRelu activation function, and then through the sixth batch of normalization layers, the third 3×3 channel-by-channel convolution, and the fourth LeakyRelu activation function, the output of the second depth-separable convolution block I∈R C×H×W , where H×W is the spatial size and C is the number of channels. The size of I is reshaped to C / / G×H×W, and I1 is obtained as the input of the EMA attention mechanism;

[0023] The EMA attention mechanism is processed in three branches, including the first branch, the second branch and the third branch. The first branch performs one-dimensional global average pooling in the horizontal direction, and the second branch performs one-dimensional global average pooling in the vertical direction. The results of the one-dimensional global average pooling of the first branch and the second branch are connected along the image height and transmitted to the 1×1 convolution. The output result of the 1×1 convolution is decomposed into two vectors and passed through two sigmoid activation functions respectively. Then, the attention graphs of the two branches are aggregated by a multiplication to obtain R 1&2 ; The third branch performs 3×3 convolution to capture multi-scale feature representation to obtain R3;

[0024] R 1&2 Through group normalization, two-dimensional global average pooling and softmax function, we get The original dimension of R3 C / / G×H×W Change to new dimension C / / G×HW ,get Will and Multiply them together to get the first spatial attention weight, which is Similarly, the output R3 of the third branch is obtained by two-dimensional global average pooling and softmax function R 1&2 The original dimension C / / G×H×W Change to new dimension C / / G×HW ,get Will and Multiply them together to get the second spatial attention weight, Finally, the first spatial attention weight Second spatial attention weight The value of is added, and then the sigmoid activation function is used and multiplied with the input I1 to obtain the output I of the EMA attention mechanism. output ;

[0025] The output I of the second depth-wise separable convolutional block and the output I of the EMA attention mechanism are combined output Perform residual connection to obtain image H that further mines spatial features EMA .

[0026] The step (6) specifically refers to: the residual SE attention module includes a first two-dimensional convolution block, a second two-dimensional convolution block, and an SE attention mechanism; the first two-dimensional convolution block and the second two-dimensional convolution block have the same structure, and the first two-dimensional convolution block consists of a batch normalization layer, a 3×3 convolution, and a Relu activation function; the SE attention mechanism includes a global average pooling layer, a fully connected layer, and a Relu activation function;

[0027] Further mining the spatial features of image H EMA After the first two-dimensional convolution block, feature C1 is obtained, feature C1 is input into the second two-dimensional convolution block to obtain feature C2, and feature C1 and feature C2 are residually connected to obtain feature C3;

[0028] Input C3 into the SE attention mechanism, first use the global average pooling layer to compress the spatial information, then use the fully connected layer and Relu activation function to excite the channel to obtain the channel weight, multiply the channel weight and input C3 to obtain feature C4, and perform residual connection between feature C3 and feature C4 to obtain the adaptively calibrated image H SE .

[0029] It can be seen from the above technical scheme that the beneficial effects of the present invention are: first, the present invention uses a depth-separable convolution block with low computational complexity, thereby realizing lightweight and efficient feature extraction; second, the residual SEA attention module proposed in the present invention obtains global information through vertical and horizontal squeezing, and obtains local information through enhancement; third, the residual EMA attention module proposed in the present invention can obtain the paired relationship between pixels and better mine spatial features; fourth, the residual SE attention module proposed in the present invention can adaptively calibrate channel information, adaptively select bands, and better obtain channel information. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 Schematic diagram of the method of the present invention

[0031] Figure 2 Schematic diagram of the composition of the SEA attention mechanism in the present invention;

[0032] Figure 3 It is a schematic diagram of the composition of the EMA attention mechanism in the present invention;

[0033] Figure 4 Schematic diagram of the composition of the SE attention mechanism in the present invention. DETAILED DESCRIPTION

[0034] like Figure 1 As shown, a hyperspectral image classification method based on spectral-spatial attention mechanism and residual network includes the following steps in order:

[0035] (1) Input hyperspectral image H∈R h×w×l , where h, w, and l represent the length, width, and number of bands of the hyperspectral image, respectively;

[0036] (2) For the input hyperspectral image H∈R h×w×l Use principal component analysis to reduce the dimension and obtain the hyperspectral image H after dimension reduction. PCA ∈R h×w×b , reduce the number of bands from l to b;

[0037] (3) The hyperspectral image H after dimensionality reduction PCA ∈R h×w×b Through two 3×3 convolution blocks, preliminary feature extraction is performed to obtain the image H after feature extraction. 3×3 ; The 3×3 convolution block includes a two-dimensional 3×3 convolution, a batch normalization layer and a Relu activation function;

[0038] (4) The image H after feature extraction 3×3 Through two residual SEA attention modules, axial squeezing is used to extract global semantic information, and then detail enhancement is used to extract local information to obtain the enhanced image H SEA ;

[0039] (5) The enhanced image H SEA Through a residual EMA attention module, the channel weights are recalibrated and the spatial pixel pairwise relationship is captured by using cross-dimensional interactions to obtain the image H that further mines spatial features. EMA ;

[0040] (6) Further mining the spatial features of the image H EMA Through a residual SE attention module, the adaptively calibrated image H is obtained SE ;

[0041] (7) The adaptively calibrated image H SE After global pooling, flattening, batch normalization and fully connected layers, the final classification result is obtained.

[0042] The step (4) specifically means that: the two residual SEA attention modules include a first residual SEA attention module and a second residual SEA attention module, and the first residual SEA attention module and the second residual SEA attention module have the same structure;

[0043] The first residual SEA attention module includes a first depth-separable convolution block and a SEA attention mechanism, wherein the first depth-separable convolution block includes a first 1×1 point-by-point convolution, a first 3×3 channel-by-channel convolution, a first batch of normalization layers, a second batch of normalization layers, a first LeakyRelu activation function, and a second LeakyRelu activation function; the image H after feature extraction 3×3 First, through the first batch of normalization layers, 1×1 point-by-point convolution, the first LeakyRelu activation function, and then through the second batch of normalization layers, the first 3×3 channel-by-channel convolution, and the second LeakyRelu activation function, the output X∈R of the first depth-separable convolution block is obtained. H×W×C , where H×W is the spatial size and C is the number of channels; the operation of the first depth-wise separable convolutional block effectively reduces the number of parameters, improves the computing speed, and enhances the generalization ability of the network.

[0044] like Figure 2 As shown, the SEA attention mechanism is added after the first depth-separable convolution block to realize global feature extraction and local detail enhancement. The SEA attention mechanism includes the first 1×1 two-dimensional convolution, the second 1×1 two-dimensional convolution, the third 1×1 two-dimensional convolution, the second 3×3 channel-by-channel convolution, the third batch normalization layer, the fourth batch normalization layer, the Relu6 activation function and multi-head attention;

[0045] First, perform a squeeze operation: the output of the first depth-separable convolutional block X∈R H×W×C After the first 1×1 two-dimensional convolution, we get First, Q, K, and V are horizontally extruded to obtain At the same time, Q, K, and V are also longitudinally extruded to obtain Q h , K h 、V h Take multi-head attention and also Q v , K v 、V v Taking multiple attentions and then adding them together, we get:

[0046]

[0047] In the formula, is a scaling factor, Y1 1Through the second 1×1 two-dimensional convolution, the output Y1 of the squeezing operation is obtained;

[0048] Then perform detail enhancement operation: connect Q, K, and V in the channel dimension, and then pass through the second 3×3 channel-by-channel convolution, the third batch of normalization layers, the Relu6 activation function, the third 1×1 two-dimensional convolution, and the fourth batch of normalization layers to obtain the output Y2 of the detail enhancement operation. Multiply Y1 and Y2 to get the output Y of the SEA attention mechanism;

[0049] The output X of the first depth-separable convolutional block and the output Y of the SEA attention mechanism are residually connected to obtain the output of the first residual SEA attention module Similarly, the output of the first residual SEA attention module Input the second residual SEA attention module, and the second residual SEA attention module outputs the enhanced image H SEA .

[0050] The step (5) specifically refers to: the residual EMA attention module includes a second depth-separable convolution block and an EMA attention mechanism, the second depth-separable convolution block includes a second 1×1 point-by-point convolution, a third 3×3 channel-by-channel convolution, a fifth batch normalization layer, a sixth batch normalization layer, a third LeakyRelu activation function, and a fourth LeakyRelu activation function; the enhanced image H SEA First, through the fifth batch of normalization layers, the second 1×1 point-by-point convolution, the third LeakyRelu activation function, and then through the sixth batch of normalization layers, the third 3×3 channel-by-channel convolution, and the fourth LeakyRelu activation function, the output of the second depth-separable convolution block I∈R C×H×W , where H×W is the spatial size and C is the number of channels. The size of I is reshaped to C / / G×H×W, and I1 is obtained as the input of the EMA attention mechanism. The operation of the second depth-separable convolutional block effectively reduces the number of parameters, improves the operation speed, and enhances the generalization ability of the network.

[0051] like Figure 3As shown in the figure, adding the EMA attention mechanism after the second depth-separable convolution block can not only recalibrate the channel weights in each parallel branch, but also capture the spatial pixel-level pairwise relationship. The EMA attention mechanism is processed in three branches, including the first branch, the second branch and the third branch. The first branch performs one-dimensional global average pooling in the horizontal direction, and the second branch performs one-dimensional global average pooling in the vertical direction. The results of the one-dimensional global average pooling of the first branch and the second branch are connected along the image height and then transmitted to the 1×1 convolution. The output result of the 1×1 convolution is decomposed into two vectors and respectively passed through two sigmoid activation functions. Then, the attention maps of the two branches are aggregated by a multiplication to obtain R. 1&2 ; The third branch performs 3×3 convolution to capture multi-scale feature representation to obtain R3;

[0052] R 1&2 Through group normalization, two-dimensional global average pooling and softmax function, we get The original dimension of R3 C / / G×H×W Change to new dimension C / / G×HW ,get Will and Multiply them together to get the first spatial attention weight, which is Similarly, the output R3 of the third branch is obtained by two-dimensional global average pooling and softmax function R 1&2 The original dimension C / / G×H×W Change to new dimension C / / G×HW ,get Will and Multiply them together to get the second spatial attention weight, Finally, the first spatial attention weight Second spatial attention weight The value of is added, and then the sigmoid activation function is used and multiplied with the input I1 to obtain the output I of the EMA attention mechanism. output ;

[0053] In order to effectively deal with the overfitting problem, the output I of the second depth-wise separable convolutional block and the output I of the EMA attention mechanism are combined. output Perform residual connection to obtain image H that further mines spatial features EMA .

[0054] The step (6) specifically refers to: the residual SE attention module includes a first two-dimensional convolution block, a second two-dimensional convolution block, and an SE attention mechanism; the first two-dimensional convolution block and the second two-dimensional convolution block have the same structure, and the first two-dimensional convolution block consists of a batch normalization layer, a 3×3 convolution, and a Relu activation function; the SE attention mechanism includes a global average pooling layer, a fully connected layer, and a Relu activation function;

[0055] Further mining the spatial features of image H EMA After the first two-dimensional convolution block, feature C1 is obtained, feature C1 is input into the second two-dimensional convolution block to obtain feature C2, and feature C1 and feature C2 are residually connected to obtain feature C3;

[0056] like Figure 4 As shown in Figure 1, C3 is input into the SE attention mechanism, the global average pooling layer is used to compress the spatial information, and then the fully connected layer and Relu activation function are used to excite the channel to obtain the channel weight. The channel weight and input C3 are multiplied to obtain feature C4. Feature C3 and feature C4 are residually connected to obtain the adaptively calibrated image H SE .

[0057] In summary, the present invention takes hyperspectral images as the research object and utilizes three different attention mechanisms to emphasize the importance of different bands for hyperspectral image classification and capture the paired relationship between spatial feature pixels, ultimately achieving efficient extraction of spectral-spatial features. At the same time, the residual network effectively alleviates the gradient vanishing and explosion problems in the deep network, ensuring the stability and efficiency of model training.

Claims

1. A hyperspectral image classification method based on spectral-spatial attention mechanism and residual network, characterized by: The method comprises the following steps in the following order: (1) Input hyperspectral image H∈R h×w×l , where h, w, and l represent the length, width, and number of bands of the hyperspectral image, respectively; (2) For the input hyperspectral image H∈R h×w×l Use principal component analysis to reduce the dimension and obtain the hyperspectral image H after dimension reduction. PCA ∈R h×w×b , reduce the number of bands from l to b; (3) The hyperspectral image H after dimensionality reduction PCA ∈R h×w×b Through two 3×3 convolution blocks, preliminary feature extraction is performed to obtain the image H after feature extraction. 3×3 ; The 3×3 convolution block includes a two-dimensional 3×3 convolution, a batch normalization layer and a Relu activation function; (4) The image H after feature extraction 3×3 Through two residual SEA attention modules, axial squeezing is used to extract global semantic information, and then detail enhancement is used to extract local information to obtain the enhanced image H SEA ; (5) The enhanced image H SEA Through a residual EMA attention module, the channel weights are recalibrated and the spatial pixel pairwise relationship is captured by using cross-dimensional interactions to obtain the image H that further mines spatial features. EMA ; (6) Further mining the spatial features of the image H EMA Through a residual SE attention module, the adaptively calibrated image H is obtained SE ; (7) The adaptively calibrated image H SE After global pooling, flattening, batch normalization and fully connected layers, the final classification result is obtained; The step (6) specifically refers to: the residual SE attention module includes a first two-dimensional convolution block, a second two-dimensional convolution block, and an SE attention mechanism; the first two-dimensional convolution block and the second two-dimensional convolution block have the same structure, and the first two-dimensional convolution block consists of a batch normalization layer, a 3×3 convolution, and a Relu activation function; the SE attention mechanism includes a global average pooling layer, a fully connected layer, and a Relu activation function; Further mining the spatial features of image H EMA After the first two-dimensional convolution block, feature C1 is obtained, feature C1 is input into the second two-dimensional convolution block to obtain feature C2, and feature C1 and feature C2 are residually connected to obtain feature C3; Input C3 into the SE attention mechanism, first use the global average pooling layer to compress the spatial information, then use the fully connected layer and Relu activation function to excite the channel to obtain the channel weight, multiply the channel weight and input C3 to obtain feature C4, and perform residual connection between feature C3 and feature C4 to obtain the adaptively calibrated image H SE .

2. The hyperspectral image classification method based on spectral-spatial attention mechanism and residual network according to claim 1, characterized in that: The step (4) specifically means that: the two residual SEA attention modules include a first residual SEA attention module and a second residual SEA attention module, and the first residual SEA attention module and the second residual SEA attention module have the same structure; The first residual SEA attention module includes a first depth-separable convolution block and a SEA attention mechanism, wherein the first depth-separable convolution block includes a first 1×1 point-by-point convolution, a first 3×3 channel-by-channel convolution, a first batch of normalization layers, a second batch of normalization layers, a first LeakyRelu activation function, and a second LeakyRelu activation function; Image H after feature extraction 3×3 First, through the first batch of normalization layers, 1×1 point-by-point convolution, the first LeakyRelu activation function, and then through the second batch of normalization layers, the first 3×3 channel-by-channel convolution, and the second LeakyRelu activation function, the output X∈R of the first depth-separable convolution block is obtained. H×W×C , where H×W is the space size and C is the number of channels; The SEA attention mechanism includes the first 1×1 two-dimensional convolution, the second 1×1 two-dimensional convolution, the third 1×1 two-dimensional convolution, the second 3×3 channel-by-channel convolution, the third batch normalization layer, the fourth batch normalization layer, the Relu6 activation function, and multi-head attention; First, perform a squeeze operation: the output of the first depth-separable convolutional block X∈R H×W×C After the first 1×1 two-dimensional convolution, we get First, Q, K, and V are horizontally extruded to obtain At the same time, Q, K, and V are also longitudinally extruded to obtain Q h , K, h V h Take multi-head attention and also Q v , K, v V v Taking multiple attentions and then adding them together, we get: In the formula, is a scaling factor, Y1 1 Through the second 1×1 two-dimensional convolution, the output Y1 of the squeezing operation is obtained; Then perform detail enhancement operation: connect Q, K, and V in the channel dimension, and then pass through the second 3×3 channel-by-channel convolution, the third batch of normalization layers, the Relu6 activation function, the third 1×1 two-dimensional convolution, and the fourth batch of normalization layers to obtain the output Y2 of the detail enhancement operation. Multiply Y1 and Y2 to get the output Y of the SEA attention mechanism; The output X of the first depth-separable convolutional block and the output Y of the SEA attention mechanism are residually connected to obtain the output of the first residual SEA attention module Similarly, the output of the first residual SEA attention module Input the second residual SEA attention module, and the second residual SEA attention module outputs the enhanced image H SEA .

3. The hyperspectral image classification method based on spectral-spatial attention mechanism and residual network according to claim 1, characterized in that: The step (5) specifically refers to: the residual EMA attention module includes a second depth-separable convolution block and an EMA attention mechanism, the second depth-separable convolution block includes a second 1×1 point-by-point convolution, a third 3×3 channel-by-channel convolution, a fifth batch normalization layer, a sixth batch normalization layer, a third LeakyRelu activation function, and a fourth LeakyRelu activation function; the enhanced image H SEA First, through the fifth batch of normalization layers, the second 1×1 point-by-point convolution, the third LeakyRelu activation function, and then through the sixth batch of normalization layers, the third 3×3 channel-by-channel convolution, and the fourth LeakyRelu activation function, the output of the second depth-separable convolution block I∈R C×H×W , where H×W is the spatial size and C is the number of channels. The size of I is reshaped to C / / G×H×W, and I1 is obtained as the input of the EMA attention mechanism; The EMA attention mechanism is processed in three branches, including the first branch, the second branch and the third branch. The first branch performs one-dimensional global average pooling in the horizontal direction, and the second branch performs one-dimensional global average pooling in the vertical direction. The results of the one-dimensional global average pooling of the first branch and the second branch are connected along the image height and transmitted to the 1×1 convolution. The output result of the 1×1 convolution is decomposed into two vectors and passed through two sigmoid activation functions respectively. Then, the attention graphs of the two branches are aggregated by a multiplication to obtain R 1&2 ; The third branch performs 3×3 convolution to capture multi-scale feature representation to obtain R3; R 1&2 Through group normalization, two-dimensional global average pooling and softmax function, we get The original dimension of R3 C / / G×H×W Change to new dimension C / / G×HW ,get Will and Multiply them together to get the first spatial attention weight, which is Similarly, the output R3 of the third branch is obtained by two-dimensional global average pooling and softmax function R 1&2 The original dimension C / / G×H×W Change to new dimension C / / G×HW ,get Will and Multiply them together to get the second spatial attention weight, Finally, the first spatial attention weight Second spatial attention weight The value of is added, and then the sigmoid activation function is used and multiplied with the input I1 to obtain the output I of the EMA attention mechanism. output ; The output I of the second depth-wise separable convolutional block and the output I of the EMA attention mechanism are combined output Perform residual connection to obtain image H that further mines spatial features EMA .

Images