Image classification method based on physical global filtering attention convolutional neural network

Abstract

The application discloses an image classification method based on a physical global filtering attention convolutional neural network, mainly solves the problem that the prior art has insufficient recognition ability for complex patterns and subtle features in an image classification task, and an implementation scheme is as follows: an image classification dataset is acquired, and a training set and a test set are divided; a global filtering attention module including an encoder, a global filter layer, a feature fusioner, an expander and a global attention fusioner is constructed to fuse one-dimensional channel weights and two-dimensional spatial weights into three-dimensional attention weights; the global filtering attention module is added to a residual block in an existing convolutional neural network architecture; the training set is used to train the image classification network; and the test set is input into the trained image classification network based on the physical global filter attention convolutional neural network to obtain an image classification result of the test set. The application improves the recognition ability of the network for complex patterns and subtle features in the image classification task, improves the image classification accuracy, and can be used for automatic driving and medical image analysis.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of deep learning neural networks, and particularly relates to an image classification method based on a convolutional neural network, which can be used for automatic driving and medical image analysis. BACKGROUND

[0002] Convolutional neural networks (CNNs) have shown excellent performance in various vision tasks, such as image classification, object detection, and video understanding. Numerous studies have shown that the key to improving performance lies in designing a better CNN architecture. Fourier transform has been an important tool for digital signal and image processing for decades, and with the breakthrough of CNN in vision, various works have begun to incorporate Fourier transform into deep learning methods. Some of these studies use discrete Fourier transform to convert images to the frequency domain and use frequency information to improve the performance of certain tasks, and some use the convolution theorem of Fourier transform to accelerate CNN.

[0003] Currently, image classification methods based on CNN architecture design can be roughly divided into two categories: manual design (MD) and neural architecture search (NAS). Regardless of the method used, the structure of CNN is usually composed of basic convolution, pooling, and activation functions, etc. For manual design methods, the most direct strategy is to increase the depth of CNN to utilize the ability of deep networks to approximate high-dimensional functions, thereby improving the final performance. The design philosophy of classic convolutional neural network models such as AlexNet, ZFNet, and VGGNet follows this principle. Another design philosophy is to build convolutional modules with stronger feature representation capabilities, such as the initial block used in the common convolutional neural network GoogLeNet, the dense connection used in DenseNet, and the full convolution used in UNet. These network structures have been widely used in various tasks in different fields, however, the CNN structure constructed by the convolutional module has a high representation capability, but usually requires rich professional knowledge. For neural architecture search methods, the network structure is automatically constructed through search strategies such as evolutionary algorithms, reinforcement learning, gradient optimization, and Bayesian optimization, to avoid the shortcomings of manual design.

[0004] In addition, in addition to designing more complex CNN structures, another research hotspot for image classification methods is the design of plug-and-play attention modules. These modules help the network pay more attention to important features by enhancing and suppressing feature maps.

[0005] Hu et al. in the article Squeeze-and-excitation networks proposed a Squeeze-and-Excitation attention mechanism module, which improves the representation ability of the network by modeling the interdependence between convolutional feature channels, and this method enables the network to achieve feature recalibration and learn to use global information to selectively emphasize information features and suppress less useful features to enhance image classification performance.

[0006] Park et al. in the article Bam: Bottleneck attention module proposed a BAM attention mechanism module, which obtains an Attention Map through two separate paths channel and spatial to achieve 3D attention mechanism, and this image classification method helps to effectively improve the representation ability of the network, which performs well in large models.

[0007] Woo et al. in the article Cbam: Convolutional block attention module proposed a CBAM attention module, which can be seamlessly integrated into any CNN architecture and can be trained end-to-end with basic CNN. By intermediate feature maps, the attention weights are inferred along the spatial and channel dimensions in turn, and then multiplied with the original feature maps to adaptively adjust the features. After adding CBAM to different models using this method on different image classification datasets, the image classification performance of the network has been consistently improved, demonstrating its wide applicability.

[0008] However, the above existing image classification methods have the following two problems:

[0009] First, since they use highly complex local operators such as fully connected and convolution in the attention module, not only do they limit the information they can obtain from global feature maps, but they also require a large amount of matrix calculations, affecting the computational efficiency of the network and increasing the training time and cost.

[0010] Second, since they mainly focus on spatial information, they ignore the role of frequency information in the network, which leads to insufficient recognition of complex patterns and subtle features in the network in image classification tasks, thereby affecting the network's image classification ability. SUMMARY

[0011] The purpose of the present application is to reduce the matrix operation amount, enhance the feature extraction of the network on the image frequency information, thereby improving the calculation efficiency of image classification, improving the image classification accuracy of the convolutional neural network, and improving the recognition ability of the network on complex patterns and subtle features in the image classification task.

[0012] The technical idea for achieving the purpose of the present application is: learning the long-term interaction and global information of channels and spaces through global learnable filters to obtain one-dimensional global channel weights and two-dimensional global space weights; generating three-dimensional global filter weights by aggregating one-dimensional global channel weights and two-dimensional global space weights to realize physical global filter attention, thereby improving the image classification accuracy and calculation efficiency of the convolutional neural network.

[0013] According to the above technical idea, the technical scheme of the present application includes the following:

[0014] (1) Obtain an image classification dataset, and divide it into a training set and a test set according to an 8:2 ratio;

[0015] (2) Construct a global filter attention module GFA:

[0016] (2a) Construct a first encoder D1 composed of a spatial dimension maximum pooling layer and an average pooling layer, which is used for soft coding of global channel information to generate a one-dimensional channel vector and Construct a second encoder D2 composed of a channel dimension maximum pooling layer and an average pooling layer, which is used for soft coding of global space information to generate a two-dimensional space map and

[0017] (2b) Construct a first global filter layer G1 including a one-dimensional fast Fourier transform, a first global learnable filter L1, and a one-dimensional inverse fast Fourier transform, which is used for filtering the one-dimensional channel vector and to generate a global channel feature vector and Construct a second global filter layer G2 including a two-dimensional fast Fourier transform, a second global learnable filter L2, and a two-dimensional inverse fast Fourier transform, which is used for filtering the two-dimensional space map and to generate a global space feature map and

[0018] (2c) Construct a first feature fusioner S1 including a first element-wise adder and a sigmoid activation function, which is used for fusing the global channel feature vector and fuse into one-dimensional global channel weight M gc ; a second feature fusioner S2 including a second element-wise adder and a sigmoid activation function is constructed to fuse the global spatial feature map and fuse into two-dimensional global spatial weight M gs ;

[0019] (2d) an expander E is set to expand the one-dimensional global channel weight M gc into a global channel weight M' of size CxHxW; gc , and expand the two-dimensional global spatial weight M gs into a global spatial weight M' of size CxHxW; gs , where C is the channel dimension, i.e., the number of channels, and H and W are the height and width of the spatial dimension, respectively;

[0020] (2e) a global attention fusioner M including a third element-wise adder and a sigmoid activation function is constructed to element-wise add the global channel weight M' gc and the global spatial weight M' gs , and aggregate the added result into a three-dimensional global filter weight M gf through the sigmoid activation function;

[0021] (2f) the first encoder D1, the first global filter layer G1, and the first feature fusioner S1 are cascaded to form a first filter attention branch; the second encoder D2, the second global filter layer G2, and the second feature fusioner S2 are cascaded to form a second filter attention branch; the expander E and the global attention fusioner M are cascaded to form a fusion branch; the expander E is connected to the first feature fusioner S1 in the first branch and the second feature fusioner S2 in the second branch, respectively, to form a global filter attention module GFA;

[0022] (3) the global filter attention module GFA is added to the convolutional neural block in the existing convolutional neural network Net1 architecture to obtain a physical global filter attention convolutional neural network Net2;

[0023] (4) the training set is input to the physical global filter attention convolutional neural network Net2, which is trained by the back propagation method to obtain a trained physical global filter attention convolutional neural network Net3;

[0024] (5) the test set is input to the trained physical global filter attention convolutional neural network Net3 to obtain the image classification result of the test set.

[0025] Compared with the prior art, the present application has the following advantages:

[0026] 1、The present application sets the first global filtering layer G1 and the second global filtering layer G2, which can convert the one-dimensional channel vector and the two-dimensional spatial graph output by the encoder into the frequency domain for learnable filtering, learn the long-term interaction and global information of the channel and the space in the frequency domain, enhance the feature extraction of the network on the image frequency information, and improve the recognition ability of the network on complex patterns and subtle features in the image classification task.

[0027] 2、The present application uses one-dimensional fast Fourier transform and two-dimensional fast Fourier transform, which has lower complexity than the full connection layer and the convolution layer used in the prior art, and avoids a large amount of matrix calculation, so that the network is more efficient in the image classification task.

[0028] 3、The present application sets the expander E and the global attention fusion device M, which can expand the one-dimensional global channel weight M gc and the two-dimensional global spatial weight M gs to CxHxW size, and then add them element by element, and finally aggregate them into the final three-dimensional global filtering weight M gf through the sigmoid activation function, so that the three-dimensional global filtering attention aggregated by the one-dimensional global channel attention and the two-dimensional global spatial attention has strong global attention on the channel and the space, and enhances the extraction ability of the network on the important information features of the image. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 is the implementation flowchart of the present application;

[0030] Figure 2 is the structure diagram of the global filtering attention module constructed in the present application;

[0031] Figure 3 is the structure diagram of the global attention fusion device M in the global filtering attention module constructed in the present application;

[0032] Figure 4 is the structure diagram of the global filtering attention convolutional neural network ResNet-34 constructed in the present application;

[0033] Figure 5 is the Top-1 accuracy rate curve comparison diagram obtained by respectively using the existing convolutional neural network ResNet-34, the existing convolutional neural network ResNet-34 using the CBAM attention module and the network of the present application to perform image classification on the data set CIFAR-10;

[0034] Figure 6A Top-1 accuracy curve comparison diagram obtained by performing image classification calculation on a dataset CIFAR-100 by using an existing convolutional neural network ResNet-34, an existing convolutional neural network ResNet-34 using a CBAM attention module, and the network of the present application, respectively. DETAILED DESCRIPTION

[0035] The embodiments and effects of the present application are described in further detail below with reference to the accompanying drawings.

[0036] Reference Figure 1 The implementation steps of the present example are as follows:

[0037] Step 1: Obtain an image classification dataset and divide it into a training set and a test set.

[0038] Obtain a ten-class CIFAR-10 dataset containing 60,000 color images, a hundred-class CIFAR-100 dataset containing 60,000 color images, and a thousand-class ImageNet-1K dataset containing more than 1 million images from a public website;

[0039] Divide the obtained dataset into a training set and a test set in a ratio of 8:2.

[0040] Step 2, construct a global filtering attention module GFA.

[0041] Reference Figure 2 The specific steps are as follows:

[0042] (2.1) Construct a first encoder D1 composed of a spatial dimension max-pooling layer and an average pooling layer, for soft encoding of global channel information, that is, for a given feature F of size CxHxW, two one-dimensional channel max-pooling vectors of size Cx1x1 are obtained by a spatial dimension global max-pooling and a spatial dimension average pooling, respectively and average pooling vector

[0043] (2.2) Construct a second encoder D2 composed of a channel dimension max-pooling layer and an average pooling layer, for soft encoding of global spatial information, that is, for a given feature F of size CxHxW, two two-dimensional spatial max-pooling maps of size 1xHxW are obtained by a channel dimension max-pooling and a channel dimension average pooling, respectively and average pooling map

[0044] (2.3) Construct a first global filtering layer G1 including a one-dimensional fast Fourier transform, a first global learnable filter L1, and a one-dimensional inverse fast Fourier transform, for one-dimensional channel vectors and Perform filtering:

[0045] (2.3.1) The one-dimensional channel vector output by the upper layer after pooling and Perform a one-dimensional FFT transformation to the frequency domain:

[0046]

[0047] In the formula, F1 represents a one-dimensional channel vector. and It is a complex tensor, denoted as the spectrum of F1. This represents a one-dimensional FFT, for a given signal sequence. The formula for the one-dimensional discrete Fourier transform is as follows:

[0048]

[0049] Where, x n This represents the value of the input signal at the nth sample point, where N represents the total length of the signal, i.e., the number of samples, and n is the time-domain signal sample index from 0 to N-1. is the rotation factor, which represents the periodic frequency component, and j is the imaginary unit;

[0050] (2.3.2) Multiplied by the first globally learnable filter L1 To control the spectrum and obtain the filtered spectrum

[0051]

[0052] in It is element-wise multiplication, where L1 is the input one-dimensional channel vector. and The first globally learnable filter with the same dimension;

[0053] (2.3.3) Use IFFT to filter the spectrum Transforming back to the original space, we obtain the one-dimensional feature F'1:

[0054]

[0055] This one-dimensional feature F'1 is the one-dimensional global filter channel feature vector. and The formula for the one-dimensional IFFT and the one-dimensional discrete Fourier inverse transform is as follows:

[0056]

[0057] where X[k] represents the value of the k-th frequency component, k is the frequency index of the frequency domain signal from 0 to N-1, represents the value of the signal after inverse transformation at the n-th sample point.

[0058] (2.4) A first feature fusioner S1 including a first element-wise adder and a sigmoid activation function is constructed to fuse the one-dimensional global filter channel feature vector and element-wise addition, and then input into the sigmoid activation function to fuse into a one-dimensional Cx1x1 global channel weight M gc :

[0059]

[0060] where σ(·) represents the activation function, ⊕ is the element-wise addition, M gc is fused into a one-dimensional global channel weight from the global channel feature vector and .

[0061] (2.5) A second global filter layer G2 including a two-dimensional fast Fourier transform, a second global learnable filter L2, and a two-dimensional inverse fast Fourier transform is constructed to filter the two-dimensional spatial map and :

[0062] (2.5.1) The two-dimensional spatial map and outputted after the pooling of the upper layer is converted to the frequency domain by two-dimensional FFT:

[0063]

[0064] where F2 represents the two-dimensional spatial map and is a complex tensor, denoted as the frequency spectrum of F2, represents the two-dimensional FFT, and for a given two-dimensional signal {x[z,m]:0≤z≤Z-1,0≤m≤M-1}, the two-dimensional discrete Fourier transform formula is as follows:

[0065]

[0066] where x[z,m] represents the value of the two-dimensional signal at position [z,m], Z and M represent the number of rows and columns respectively, and u and v represent the horizontal frequency index and the vertical frequency index of the two-dimensional frequency domain signal respectively; is the positive transform rotation factor,

[0067] for converting the signal to the frequency domain, and j is the imaginary unit;

[0068] (2.5.2) multiplying by a second globally learnable filter L2 to control the spectrum, obtaining a filtered spectrum

[0069]

[0070] where the dimension of the second globally learnable filter L2 is the same as the two-dimensional spatial map output by the second encoder D2 and

[0071] (2.5.3) converting the filtered spectrum back to the original space using a two-dimensional IFFT, obtaining a two-dimensional feature F'2:

[0072]

[0073] The two-dimensional feature F'2 is the two-dimensional globally filtered spatial feature map and represents a two-dimensional IFFT, and the formula for the two-dimensional inverse discrete Fourier transform is as follows:

[0074]

[0075] where X[u,v] represents the value of the two-dimensional signal at frequency [u,v], z and m represent the spatial indices of the two-dimensional signal, represents the value of the signal after inverse transformation at position [z,m], is the normalization factor in the inverse transformation; is the inverse transformation rotation factor, used to convert the signal back to the original space.

[0076] (2.6) constructing a second feature fusioner S2 including a second element-wise adder and a sigmoid activation function, for element-wise adding the two-dimensional globally filtered spatial feature map and and then inputting the activation function to fuse into a two-dimensional 1xHxW global spatial weight M gs :

[0077]

[0078] where σ(·) represents the activation function, ⊕ is element-wise addition, and M gs is the two-dimensional global spatial weight fused from the global spatial feature map and .

[0079] (2.7) setting an expander E for expanding the one-dimensional global channel weight M gc ​The vector is expanded to a global channel weight M' of size C×H×W. gc The expander E will use the two-dimensional global spatial weights M gs Expanded to a global spatial weight M' of size C×H×W gs The formulas are as follows:

[0080] M' gc =expand(M gc )

[0081] M' gs =expand(M gs )

[0082] Where expand(·) represents the dimension expansion operation, M gc Let M be a one-dimensional global channel weight of size C×1×1. gs The weights are two-dimensional global spatial weights of size 1×H×W.

[0083] (2.8) Construct a global attention fusion unit M that includes a third element-wise adder and a sigmoid activation function, such as Figure 3 As shown, it consists of a third element-wise adder and a sigmoid activation function, used to expand the global channel weights M of size C×H×W. gc and global space weight M gs The elements are added one by one and aggregated through the sigmoid activation function to form the final three-dimensional global filter weights M. gf :

[0084] M gf =σ(M' gc ⊕M' gs )

[0085] Where M gf It is the three-dimensional global filter weight, M' gc It is the global channel weight, M' gs σ represents the global space weights, σ(·) represents the activation function, and ⊕ represents element-wise addition.

[0086] (2.9) Connect the various parts to form the Global Filter Attention Module (GFA):

[0087] (2.9.1) The first encoder D1, the first global filter layer G1, and the first feature fusion unit S1 are cascaded to form the first filter attention branch;

[0088] (2.9.2) The second encoder D2, the second global filter layer G2, and the second feature fusion unit S2 are cascaded to form the second filter attention branch;

[0089] (2.9.3) The expander E is cascaded with the global attention fusion M to form a fusion branch;

[0090] (2.9.4) The expander E is connected with the first feature fusion S1 in the first branch and the second feature fusion S2 in the second branch respectively to form a global filtering attention module GFA.

[0091] Step 3, add the global filtering attention module GFA in the convolutional neural block in the existing convolutional neural network Net1 architecture to obtain a physical global filtering attention convolutional neural network Net2.

[0092] The physical global filtering attention convolutional neural network Net2 includes an initial convolutional layer, a maximum pooling layer, a plurality of convolutional neural blocks, a global average pooling layer, a fully connected layer and the global filtering attention module GFA constructed in step 2 in the convolutional neural network Net1, each convolutional neural block includes a split convolutional layer, a normalization layer, an activation function and an adder, and the global filtering attention module is connected between the convolutional layer and the adder, and the structural relationship is as follows:

[0093] Initial convolutional layer—>maximum pooling layer—>a plurality of convolutional neural blocks—>global average pooling layer—>fully connected layer.

[0094] The following uses a convolutional neural network ResNet-34 network as an example:

[0095] The present example uses but is not limited to a ResNet-34 convolutional neural network, and a physical global filtering attention convolutional neural network ResNet-34 composed of the same, as shown in Figure 4 The network includes an initial convolutional layer, a maximum pooling layer, 32 convolutional neural blocks, a global average pooling layer and a fully connected layer. The convolutional neural block includes two split convolutional layers, two normalization layers, two activation functions, an adder and a global filtering attention module GFA. Each convolutional neural block includes two parts, wherein the first part is composed of a split convolutional layer with a convolution kernel of 3x3, a normalization layer and an activation function; the second part is composed of a split convolutional layer with a convolution kernel of 3x3 and a normalization layer. The output end of the first part is connected to the input end of the second part, the output of the second part is connected to the adder, the global filtering attention module GFA is connected between the input end of the first part and the adder, and the output of the adder is connected to an activation function to form a physical global filtering attention convolutional neural network ResNet-34.

[0096] Step 4, train the physical global filtering attention convolutional neural network Net2 by the back propagation method.

[0097] (4.1) Define the existing cross-entropy loss function as the global loss function L of the physical-based global filter attention convolutional neural network Net2:

[0098]

[0099] Where C is the total number of categories, y i is the true label, y i = 1 if category i is correct, otherwise y i = 0, p i is the probability of the model predicting category i, log(·) is the logarithm operation with base e;

[0100] (4.2) Input a batch of data in the training set to the network Net2, and calculate the global loss value of the training set according to the predicted category probability obtained by inputting the network with the training set;

[0101] (4.3) Update the network parameters by backpropagation iteration of the training set global loss;

[0102] (4.4) Calculate the global loss of the validation set according to the predicted category probability obtained by inputting the network with a batch of data in the validation set, which is used to evaluate the performance of the current convolutional neural network;

[0103] (4.5) Repeat (4.2) to (4.4) until the global loss of the training set converges, and obtain the trained physical-based global filter attention convolutional neural network Net3.

[0104] Step 5, input the test set to the trained physical-based global filter attention convolutional neural network Net3 to obtain the image classification result of the test set.

[0105] The effects of the present application can be further illustrated by the following simulation results.

[0106] I. Simulation conditions

[0107] 1. The simulation platform is a high-performance server computing platform with 56 CPUs and 502 GB of memory, the operating system is Linux 4.4.0, Python 3.8 is used, Pytorch 1.1 is used to build the neural network model, and it is trained, 5 different models of NVIDIA graphics cards and CUDA 10.1 are used for acceleration, the NVIDIA graphics card model is 2 RTX2080Ti, 2 RTX2080Ti Rev and 1 TITAN V.

[0108] 2. The evaluation index used is Top-1 accuracy and Top-5 accuracy (%), the Top-A accuracy calculation formula is:

[0109]

[0110] where P is the total number of samples, y A is the true label of the a-th sample, is the first A classes predicted by the model for the a-th sample, 1(·) is an indicator function, which is 1 when the condition in the bracket is true, and 0 otherwise.

[0111] II. Simulation content

[0112] Simulation 1, under the above simulation conditions, the existing convolutional neural network ResNet-34 image classification method, the existing CBAM attention module based convolutional neural network image classification method, and the image classification method based on the physical global filtering attention convolutional neural network of the present application are used respectively to perform image classification on the CIFAR-10 dataset, and the Top-1 verification accuracy of each is calculated, and the results are shown in Figure 5 .

[0113] It can be seen from Figure 5 that the initial image classification accuracy of the image classification method based on the physical global filtering attention convolutional neural network of the present application is improved by 25.73% compared with the existing convolutional neural network ResNet-34 image classification method, and by 11.34% compared with the existing CBAM attention module based convolutional neural network image classification method. At the same time, it can also be seen that the accuracy curve of the image classification method based on the physical global filtering attention convolutional neural network of the present application reaches the convergence state 20 epochs earlier than the accuracy curve of the existing convolutional neural network ResNet-34 image classification method.

[0114] The simulation results show that the image classification method based on the physical global filtering attention convolutional neural network of the present application can accelerate the convergence speed of network training on image classification tasks, and can obtain higher accuracy on image classification tasks compared with the existing CBAM attention module based convolutional neural network image classification method.

[0115] Simulation 2, under the above simulation conditions, the existing convolutional neural network ResNet-34 image classification method, the existing CBAM attention module based convolutional neural network image classification method, and the image classification method based on the physical global filtering attention convolutional neural network of the present application are used respectively to perform image classification on the CIFAR-100 dataset, and the Top-1 verification accuracy of each is calculated, and the results are shown in Figure 6 .

[0116] It can be seen from Figure 6It can be seen that the image classification method based on the physical global filtering attention convolutional neural network of the present application has an initial image classification accuracy that is 21.39% higher than the existing convolutional neural network ResNet-34 image classification method and 19.56% higher than the existing convolutional neural network image classification method based on the CBAM attention module. It can also be seen that the image classification accuracy curve of the image classification method based on the physical global filtering attention convolutional neural network of the present application reaches the convergence state 20 rounds earlier than the image classification accuracy curve of the existing convolutional neural network ResNet-34 image classification method.

[0117] The simulation results show that in different categories of data sets, the image classification method based on the physical global filtering attention convolutional neural network of the present application still has good performance and can help the convolutional neural network to obtain higher accuracy in image classification tasks.

[0118] In simulation 3, under the above simulation conditions, the existing convolutional neural network ResNet-18, ResNet-34, ResNet-50 image classification method, the existing convolutional neural network ResNet-18, ResNet-34, ResNet-50 based on SE attention module convolutional neural network image classification method, the existing convolutional neural network ResNet-18, ResNet-34, ResNet-50 based on CBAM attention module convolutional neural network image classification method, the existing convolutional neural network ResNet-18, ResNet-34, ResNet-50 based on SimAM attention module convolutional neural network image classification method, and the image classification method based on the physical global filtering attention convolutional neural network of the present application using convolutional neural network ResNet-18, ResNet-34, ResNet-50 were used for image classification on the CIFAR-10 and CIFAR-100 data sets, and the Top-1 accuracy (%) of each was calculated, as shown in Table 1.

[0119] Table 1 Image classification results of different network classification methods on CIFAR-10 and CIFAR-100 data sets

[0120]

[0121]

[0122] From Table 1, it can be seen that on the CIFAR-10 and CIFAR-100 data sets, the Top-1 accuracy of the physical global filtering attention convolutional neural network image classification method based on the convolutional neural network ResNet-18, ResNet-34, ResNet-50 of the application is better than all existing convolutional neural network image classification methods and other attention module convolutional neural network image classification methods using convolutional neural network ResNet-18, ResNet-34, ResNet-50. The physical global filtering attention convolutional neural network image classification method based on the convolutional neural network ResNet-18, ResNet-34, ResNet-50 of the application improves the image classification accuracy by an average of 1.71% on the CIFAR-10 data set compared to existing convolutional neural network image classification methods, and by an average of 1.55% on the CIFAR-100 data set compared to existing convolutional neural network image classification methods.

[0123] Simulation 4, under the above simulation conditions, the existing convolutional neural network ResNet-18, ResNet-34, ResNet-50, ResNet-101 image classification method, the existing SE attention module based convolutional neural network image classification method using convolutional neural network ResNet-18, ResNet-34, ResNet-50, ResNet-101, the existing CBAM attention module based convolutional neural network image classification method using convolutional neural network ResNet-18, ResNet-34, ResNet-50, ResNet-101, the existing BAM attention module based convolutional neural network image classification method using convolutional neural network ResNet-50, ResNet-101, the existing AA attention module based convolutional neural network image classification method using convolutional neural network ResNet-50, ResNet-101, and the physical global filtering attention convolutional neural network image classification method based on the convolutional neural network ResNet-18, ResNet-34, ResNet-50, ResNet-101 of the application were used to classify images on the ImageNet-1K data set. The Top-1 and Top-5 accuracies (%) were calculated and the overall network parameters, attention module parameters and floating point operation counts were recorded, and the results are shown in Table 2.

[0124] Table 2 Image results of different network classification methods on ImageNet-1K data set

[0125]

[0126]

[0127] Wherein M represents million, G represents billion. As can be seen from Table 2, the image classification method based on the physical global filtering attention convolutional neural network of the present application shows significant advantages, especially in the parameter quantity and the computational complexity. For example, the parameter quantity of the attention module used in the image classification method based on the physical global filtering attention convolutional neural network of the present application using the convolutional neural network ResNet-18, ResNet-34, ResNet-50 and ResNet-101 is only 0.007M and 0.008M, while the parameter quantity of the attention module used in the image classification method based on the SE and CBAM attention module convolutional neural network constructed using the convolutional neural network ResNet-50 and ResNet-101 reaches the level of millions, which verifies that the image classification method based on the physical global filtering attention convolutional neural network of the present application significantly reduces the parameter quantity of the attention module. At the same time, the present application provides an image classification accuracy comparable to or even higher than other image classification methods while maintaining a low number of floating point operations, for example, the image classification Top-5 accuracy of the image classification method based on the physical global filtering attention convolutional neural network of the present application using the convolutional neural network ResNet-101 reaches 94.11%, ranking first.

[0128] All these results show that the image classification method based on the physical global filtering attention convolutional neural network designed by the present application can help the network to reduce the consumption of computing resources in the image classification task and improve the image classification accuracy.

[0129] It should be noted that the step labels in the specification and claims of the present application are only for the purpose of clearly describing the embodiments of the present application, facilitating understanding, and the sequence of the serial numbers is not limited.

Claims

1. An image classification method based on a physical global filtering attention convolutional neural network, characterized in that, Comprising the following steps: (1) Obtain an image classification dataset, divide it into a training set and a test set in a ratio of 8:2; (2) Construct a global filter attention module GFA: (2a) constructing a first encoder D1 consisting of a max-pooling layer and an average-pooling layer in the spatial dimension for soft-encoding the global channel information to generate a one-dimensional channel vector and constructing a second encoder D2 consisting of a max-pooling layer and an average-pooling layer in the channel dimension for soft-encoding the global spatial information to generate a two-dimensional spatial map and (2b) constructing a first global filtering layer G1 comprising a one-dimensional fast Fourier transform, a first globally learnable filter L1 and a one-dimensional inverse fast Fourier transform for filtering the one-dimensional channel vectors and to generate global channel feature vectors and constructing a second global filtering layer G2 comprising a two-dimensional fast Fourier transform, a second globally learnable filter L2 and a two-dimensional inverse fast Fourier transform for filtering the two-dimensional spatial maps and to generate global spatial feature maps and (2c) constructing a first feature fuser S1 including a first element-wise adder and a sigmoid activation function for fusing the global channel feature vectors and into a one-dimensional global channel weight M gc ; constructing a second feature fuser S2 including a second element-wise adder and a sigmoid activation function for fusing the global spatial feature maps and into a two-dimensional global spatial weight M gs ; (2d) setting an expander E for expanding the one-dimensional global channel weight M gc to a C x H x W size global channel weight M' gc , expanding the two-dimensional global spatial weight M gs to a C x H x W size global spatial weight M' gs , where C is the channel dimension, i.e., the number of channels, and H and W are the height and width of the spatial dimension, respectively. (2e) constructing a global attention fuser M comprising a third element-wise adder and a sigmoid activation function for aggregating the global channel weights M' of size C x H x W gc and the global spatial weights M' gs element-wise and aggregating the added result through a sigmoid activation function into a three-dimensional global filter weight M gf ; (2f) The first encoder D1, the first global filter layer G1, and the first feature fusioner S1 are cascaded to form the first filter attention branch; the second encoder D2, the second global filter layer G2, and the second feature fusioner S2 are cascaded to form the second filter attention branch; the expander E and the global attention fusioner M are cascaded to form the fusion branch; the expander E is connected to the first feature fusioner S1 in the first branch and the second feature fusioner S2 in the second branch respectively, forming the global filter attention module GFA; (3) Add the global filter attention module GFA to the residual block in the existing convolutional neural network Net1 architecture to obtain a physical global filter attention convolutional neural network Net2; (4) Input the training set into the physical global filter attention convolutional neural network Net2, and train it by back propagation method to obtain a trained physical global filter attention convolutional neural network Net3; (5) Input the test set into the trained physical global filter attention convolutional neural network Net3 to obtain the image classification results of the test set.

2. The method of claim 1, wherein, In step (2a), the first encoder D1 soft encodes the global channel information and the second encoder D2 soft encodes the global spatial information, which are implemented as follows: Given a feature F of CxHxW, where C is the channel dimension, i.e., the number of channels, H and W are the height and width of the spatial dimension, respectively; The maximum pooling operation is performed through a spatial dimension maximum pooling layer to obtain a one-dimensional channel maximum pooling vector of Cx1x1 The average pooling operation is performed through a spatial dimension average pooling layer to obtain a one-dimensional channel average pooling vector of Cx1x1 The maximum pooling operation is performed through a channel dimension maximum pooling layer to obtain a two-dimensional spatial maximum pooling graph of 1xHxW The average pooling operation is performed through a channel dimension average pooling layer to obtain a two-dimensional average pooling graph of 1xHxW That is, the channel dimension of the original feature F is reduced to 1, and the spatial dimension is kept unchanged.

3. The method of claim 1, wherein, The first global filtering layer G1 of step (2b) filters a one-dimensional channel vector and implementing step includes the following: (2b1) the one-dimensional channel vector outputted by the upper layer after pooling and one-dimensional FFT conversion to the frequency domain: where F1represents a one-dimensional channel vector and is a complex tensor, denoted as the spectrum of F1, denotes a one-dimensional FFT; (2b2) multiplying with a first globally learnable filter L1 obtaining a filtered spectrum (2b3) converting the filtered spectrum back to the spatial domain using a one-dimensional IFFT transforming back to the spatial domain to obtain one-dimensional features F'1: The one-dimensional feature F'1 is a one-dimensional global filter channel feature vector and denotes a one-dimensional IFFT.

4. The method of claim 1, wherein, The second global filtering layer G2 of step (2b) filters the two-dimensional spatial map and implementing the step comprises the following: (2b4) the two-dimensional spatial map outputted after the upper layer is pooled and performing two-dimensional FFT conversion to the frequency domain: where F2represents a two-dimensional spatial graph and is a complex tensor, denoted as the spectrum of F2, denotes a two-dimensional FFT; (2b5) multiplying with a second globally learnable filter L2 obtaining a filtered spectrum (2b6) converting the filtered spectrum back to the spatial domain using a two-dimensional IFFT transforming back to the original space to obtain a two-dimensional feature F'2: The two-dimensional feature F'2 is a two-dimensional global filtering spatial feature map and denotes a two-dimensional IFFT.

5. The method of claim 1, wherein: The first feature fusioner S1 fuses the global channel feature vectors and into a one-dimensional global channel weight M gc ; the second feature fusioner S2 fuses the global spatial feature maps and into a two-dimensional global spatial weight M gs , and the formulas are as follows respectively: where σ(·) denotes an activation function, is an element-wise addition, M gc is fused into a one-dimensional global channel weight, Mgsis fused into a two-dimensional global spatial weight by and and ​​ 6. The method of claim 1, wherein, The expander E in step (2d) expands the one-dimensional global channel weight M gc to a CxHxW size global channel weight M' gc and the expander E expands the two-dimensional global spatial weight M gs to a CxHxW size global spatial weight M' gs , the formulas are as follows, respectively: M' gc = expand(M gc ) M' gs = expand(M gs ) where expand(·) denotes a dimension expansion operation, M gc is a one-dimensional global channel weight of size C x 1 x 1, M gs is a two-dimensional global spatial weight of size 1 x H x W.

7. The method of claim 1, wherein, The global channel weight M' in step (2e) is aggregated by sigmoid activation function gc and global spatial weight M' gs The element-wise addition result is aggregated into a three-dimensional global filtering weight M gf , as follows: where M gf is a three-dimensional global filter weight, M' gc is a global channel weight, N' gs is a global spatial weight, σ(·) denotes an activation function, is an element-wise addition.

8. The method of claim 1, wherein, The physical global filter attention convolutional neural network Net2 obtained in step (3) includes the initial convolutional layer, the maximum pooling layer, the plurality of residual blocks, the global average pooling layer, the fully connected layer in the convolutional neural network Net1, and the global filter attention module GFA constructed in step (2). Each residual block includes a split convolutional layer, a normalization layer, an activation function, and an adder, and the global filter attention module is connected between the convolutional layer and the adder. The structural relationship is as follows: Initial convolutional layer—>maximum pooling layer—>a plurality of residual blocks—>global average pooling layer—>fully connected layer.

9. The method of claim 1, wherein, Step (4) trains the physical global filter attention convolutional neural network Net2 by back propagation method, and the implementation steps include the following: (4a) Define the existing cross-entropy loss function as the global loss function L of the physical global filter attention convolutional neural network Net2: where C is the total number of classes, y i is the true label, y i = 1 if class i is correct, otherwise y i = 0, p i is the probability that the model predicts class i. (4b) Input a batch of data in the training set into the network Net2, and calculate the global loss value of the training set according to the predicted class probability obtained by inputting the training set into the network; (4c) Update the network parameters by back propagation iteration of the training set global loss; (4d) Calculate the global loss of the validation set according to the predicted class probability obtained by inputting a batch of data in the validation set into the network, which is used to evaluate the performance of the current convolutional neural network; (4e) repeating (4b) to (4d) until the training set global loss converges, to obtain the trained physics-based global filter attention convolutional neural network Net3.

Images