Image classification method based on physical global filtering attention convolutional neural network

Abstract

The invention discloses an image classification method based on a physical global filtering attention convolutional neural network, which mainly solves the problem of insufficient recognition capability on complex modes and fine features in an image classification task in the prior art, and the implementation scheme is as follows: obtaining an image classification data set, and dividing the image classification data set into a training set and a test set; constructing a global filtering attention module comprising an encoder, a global filter layer, a feature fusion device, an expander and a global attention fusion device so as to fuse the one-dimensional channel weight and the two-dimensional space weight into a three-dimensional attention weight; adding a global filtering attention module in a residual block in the existing convolutional neural network architecture; training the image classification network by using the training set; and inputting the test set into the trained physical global filter-based attention convolutional neural network to obtain an image classification result of the test set. According to the method, the recognition capability of the network on complex modes and fine features in the image classification task is improved, the image classification accuracy is improved, and the method can be used for automatic driving and medical image analysis.

Description

Technical Field

[0001] The present invention belongs to the technical field of deep learning neural networks, and in particular relates to an image classification method based on convolutional neural networks, which can be used for autonomous driving and medical image analysis. Background Art

[0002] Convolutional neural networks (CNNs) have performed well in various visual tasks, such as image classification, object detection, and video understanding. A large number of studies have shown that the key to improving performance lies in designing a better CNN architecture. For decades, Fourier transform has been an important tool for digital signal and image processing. With the breakthrough of CNN in vision, various works have begun to incorporate Fourier transform into deep learning methods. Some of these studies use discrete Fourier transform to convert images to the frequency domain and use frequency information to improve the performance of certain tasks. Some studies also use the convolution theorem of Fourier transform to accelerate CNN.

[0003] At present, the design of image classification methods based on CNN architecture can be roughly divided into two categories: manual design MD and neural architecture search NAS. Regardless of the method used, the structure of CNN is usually composed of basic operators such as convolution, pooling and activation functions. For manual design methods, the most direct strategy is to increase the depth of CNN to utilize the ability of deep networks to approximate high-dimensional functions, thereby improving the final performance. The design concept of the classic convolutional neural network models AlexNet, ZFNet and VGGNet follows this principle. Another design concept is to build a convolution module with stronger feature representation capabilities. For example, the common convolutional neural network GoogLeNet uses initial blocks, DenseNet uses dense connections, and UNet uses full convolution. These network structures have been widely used in various tasks in different fields. However, although the CNN structure constructed by the convolution module has a high representation ability, it usually requires rich professional knowledge. For the neural architecture search method, the neural network structure is automatically constructed through search strategies such as evolutionary algorithms, reinforcement learning, gradient optimization and Bayesian optimization to avoid the shortcomings of manual design.

[0004] In addition to designing more complex CNN structures, another hot topic in image classification methods is the design of plug-and-play attention modules. These modules help the network pay more attention to important features by enhancing and suppressing feature maps.

[0005] Hu et al. proposed an attention mechanism module called Squeeze-and-Excitation in the article "Squeeze-and-excitation networks", which improves the representation ability of the network by modeling the interdependence between convolutional feature channels. This method enables the network to achieve feature recalibration and learn to use global information to selectively emphasize informative features and suppress less useful features to enhance the performance of image classification.

[0006] Park et al. proposed a module called BAM attention mechanism in the article "Bam: Bottleneck attention module". The model obtains an Attention Map through two separate paths, channel and spatial, to implement a 3D attention mechanism. This image classification method helps to effectively improve the representation ability of the network and performs well in large models.

[0007] Woo et al. proposed a CBAM attention module in the article "Cbam: Convolutional block attention module", which can be seamlessly integrated into any CNN architecture and can be trained end-to-end with the basic CNN. Through the intermediate feature map, the attention weights are inferred along the spatial and channel dimensions, and then multiplied with the original feature map to adaptively adjust the features. On different image classification datasets, after adding CBAM to different models using this method, the network's image classification performance has been consistently improved, demonstrating its wide applicability.

[0008] However, the above existing image classification methods have the following two problems:

[0009] First, since they use highly complex local operators such as full connection and convolution in the attention module, it not only limits their ability to obtain information from the global feature map, but also requires a large amount of matrix calculations, which affects the computational efficiency of the network and increases the time and cost of training.

[0010] Secondly, since they mainly focus on spatial information and ignore the effect of frequency information on the network, the network's ability to recognize complex patterns and subtle features in image classification tasks is insufficient, thus affecting the network's image classification capabilities. Summary of the invention

[0011] The purpose of the present invention is to address the deficiencies of the above-mentioned prior art and propose an image classification method based on a physical global filtering attention convolutional neural network to reduce the amount of matrix operations and enhance the network's feature extraction of image frequency information, thereby improving the computational efficiency of image classification, improving the image classification accuracy of the convolutional neural network, and enhancing the network's ability to recognize complex patterns and subtle features in image classification tasks.

[0012] The technical idea for achieving the purpose of the present invention is: through a global learnable filter, the long-term interaction and global information of channels and spaces are learned respectively to obtain one-dimensional global channel weights and two-dimensional global space weights; by aggregating the one-dimensional global channel weights and the two-dimensional global space weights to generate three-dimensional global filtering weights, physical-based global filtering attention is realized, thereby improving the image classification accuracy and computational efficiency of convolutional neural networks.

[0013] According to the above technical ideas, the technical solutions of the present invention include the following:

[0014] (1) Obtain an image classification dataset and divide it into a training set and a test set in a ratio of 8:2;

[0015] (2) Constructing the global filter attention module GFA:

[0016] (2a) Construct the first encoder D1 consisting of a maximum pooling layer and an average pooling layer in the spatial dimension to soft-encode the global channel information and generate a one-dimensional channel vector and Construct a second encoder D2 consisting of a maximum pooling layer and an average pooling layer in the channel dimension to soft-code the global spatial information and generate a two-dimensional spatial map and

[0017] (2b) constructing a first global filter layer G1 including a one-dimensional fast Fourier transform, a first global learnable filter L1 and a one-dimensional inverse fast Fourier transform for the one-dimensional channel vector and Filter and generate global channel feature vector and Construct a second global filter layer G2 including a two-dimensional fast Fourier transform, a second global learnable filter L2 and a two-dimensional inverse fast Fourier transform for two-dimensional spatial graph and Filter and generate a global spatial feature map and

[0018] (2c) Construct a first feature fuser S1 including a first element-by-element adder and a sigmoid activation function to convert the global channel feature vector and Fusion into a one-dimensional global channel weight M gc ; Construct a second feature fuser S2 including a second element-by-element adder and a sigmoid activation function to convert the global spatial feature map and Fusion into a two-dimensional global space weight M gs ;

[0019] (2d) Set the expander E to convert the one-dimensional global channel weight M gc The vector is expanded to a global channel weight M' of size C×H×W gc , the two-dimensional global space weight M gs Expanded to a global spatial weight M' of size C×H×W gs , where C is the channel dimension, i.e. the number of channels, H and W are the height and width of the spatial dimension respectively;

[0020] (2e) Construct a global attention fuser M including a third element-wise adder and a sigmoid activation function to expand the global channel weight M' to a size of C×H×W gc and the global spatial weight M' gs Add them element by element, and aggregate the results into a three-dimensional global filter weight M through the sigmoid activation function gf ;

[0021] (2f) The first encoder D1, the first global filter layer G1, and the first feature fusion device S1 are cascaded to form a first filter attention branch; the second encoder D2, the second global filter layer G2, and the second feature fusion device S2 are cascaded to form a second filter attention branch; the expander E and the global attention fusion device M are cascaded to form a fusion branch; the expander E is respectively connected to the first feature fusion device S1 in the first branch and the second feature fusion device S2 in the second branch to form a global filter attention module GFA;

[0022] (3) Adding the global filter attention module GFA to the convolutional neural block in the existing convolutional neural network Net1 architecture to obtain a physical global filter attention convolutional neural network Net2;

[0023] (4) Input the training set into the physics-based global filter attention convolutional neural network Net2, train it through the back propagation method, and obtain the trained physics-based global filter attention convolutional neural network Net3;

[0024] (5) The test set is input into the trained physics-based global filter attention convolutional neural network Net3 to obtain the image classification results of the test set.

[0025] Compared with the prior art, the present invention has the following advantages:

[0026] 1. Since the first global filter layer G1 and the second global filter layer G2 are provided in the present invention, the one-dimensional channel vector and the two-dimensional space graph output by the encoder can be converted into the frequency domain for learnable filtering, and the long-term interaction and global information of the channel and space in the frequency domain can be learned, so as to enhance the network's feature extraction of image frequency information and improve the network's recognition ability of complex patterns and subtle features in image classification tasks.

[0027] 2. Since the present invention uses one-dimensional fast Fourier transform and two-dimensional fast Fourier transform, the complexity is lower than that of the fully connected layer and convolutional layer used in the prior art, and a large number of matrix calculations are avoided, making the network more efficient in image classification tasks.

[0028] 3. The present invention sets an expander E and a global attention fusion device M, which can convert the one-dimensional global channel weight M gc vector and the 2D global space weight M gs They are all expanded to C×H×W size, then added element by element, and finally aggregated into the final three-dimensional global filter weight M through the sigmoid activation function. gf The three-dimensional global filtering attention aggregated by one-dimensional global channel attention and two-dimensional global spatial attention has strong global attention to channels and spaces, enhancing the network's ability to extract important information features of images. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a flow chart for realizing the present invention;

[0030] Figure 2 This is a structural diagram of the global filtering attention module constructed in the present invention;

[0031] Figure 3 It is a structural diagram of the global attention fusion device M in the global filtering attention module constructed in the present invention;

[0032] Figure 4 This is a structural diagram of the physical global filtering attention convolutional neural network ResNet-34 constructed in the present invention;

[0033] Figure 5 A comparison chart of the Top-1 accuracy curves calculated by using the existing convolutional neural network ResNet-34, the existing convolutional neural network ResNet-34 using the CBAM attention module, and the network of the present invention for image classification on the CIFAR-10 dataset;

[0034] Figure 6It is a comparison chart of the Top-1 accuracy curves obtained by performing image classification calculations on the CIFAR-100 dataset using the existing convolutional neural network ResNet-34, the existing convolutional neural network ResNet-34 using the CBAM attention module, and the network of the present invention. DETAILED DESCRIPTION

[0035] The implementation manner and effects of the present invention are further described in detail below with reference to the accompanying drawings.

[0036] Reference Figure 1 , the implementation steps of this example are as follows:

[0037] Step 1: Get the image classification dataset and divide it into training set and test set.

[0038] Obtain the 10-class CIFAR-10 dataset containing 60,000 color images, the 100-class CIFAR-100 dataset containing 60,000 color images, and the 1,000-class ImageNet-1K dataset containing more than 1 million images from public websites;

[0039] The obtained data set is divided into training set and test set in a ratio of 8:2.

[0040] Step 2: Construct the global filter attention module GFA.

[0041] Reference Figure 2 , the specific steps are as follows:

[0042] (2.1) Construct the first encoder D1 consisting of a maximum pooling layer and an average pooling layer in the spatial dimension to soft-code the global channel information. That is, for a given feature F of size C×H×W, two one-dimensional channel maximum pooling vectors of size C×1×1 are obtained by performing a global maximum pooling in a spatial dimension and an average pooling in a spatial dimension respectively. and the average pooling vector

[0043] (2.2) Construct a second encoder D2 consisting of a channel-dimensional maximum pooling layer and an average pooling layer to soft-code the global spatial information. That is, for a given feature F of size C×H×W, two two-dimensional spatial maximum pooling maps of size 1×H×W are obtained by performing a channel-dimensional maximum pooling and a channel-dimensional average pooling respectively. And average pooling map

[0044] (2.3) Construct a first global filter layer G1 including a one-dimensional fast Fourier transform, a first global learnable filter L1 and a one-dimensional inverse fast Fourier transform for the one-dimensional channel vector and To filter:

[0045] (2.3.1) The one-dimensional channel vector output by the upper layer after pooling and Perform a one-dimensional FFT conversion to the frequency domain:

[0046]

[0047] Where F1 represents the one-dimensional channel vector and is a complex tensor, denoted as the spectrum of F1, Represents a one-dimensional FFT. For a given signal sequence The one-dimensional discrete Fourier transform formula is as follows:

[0048]

[0049] Among them, x n Represents the value of the input signal at the nth sample point, N represents the total length of the signal, that is, the number of samples, and n is the time domain signal sample index from 0 to N-1; is the rotation factor, which represents the periodic frequency component, and j is an imaginary unit;

[0050] (2.3.2) is multiplied by the first global learnable filter L1 To control the spectrum, get the filtered spectrum

[0051]

[0052] in It is an element-by-element multiplication, L1 is the same as the input one-dimensional channel vector and a first global learnable filter having the same dimension;

[0053] (2.3.3) Use IFFT to filter the spectrum Transform back to the original space and get the one-dimensional feature F'1:

[0054]

[0055] The one-dimensional feature F'1 is the one-dimensional global filter channel feature vector and Represents one-dimensional IFFT, the one-dimensional discrete Fourier inverse transform formula is as follows:

[0056]

[0057] Where X[k] represents the value of the kth frequency component, and k is the frequency index of the frequency domain signal from 0 to N-1. Represented as the value of the signal at the nth sample point after inverse transformation.

[0058] (2.4) Construct a first feature fuser S1 including a first element-by-element adder and a sigmoid activation function to convert the one-dimensional global filter channel feature vector and Add them element by element, then input the sigmoid activation function to fuse them into a one-dimensional C×1×1 global channel weight M gc :

[0059]

[0060] Among them, σ(·) represents the activation function, ⊕ is the element-by-element addition, and M gc is the global channel eigenvector and Fusion into a one-dimensional global channel weight.

[0061] (2.5) Construct a second global filter layer G2 including a two-dimensional fast Fourier transform, a second global learnable filter L2 and a two-dimensional inverse fast Fourier transform for the two-dimensional spatial map and To filter:

[0062] (2.5.1) The two-dimensional spatial map output by the upper layer after pooling and Perform a two-dimensional FFT conversion to the frequency domain:

[0063]

[0064] Where F2 represents the two-dimensional space graph and is a complex tensor, denoted as the spectrum of F2, Represents two-dimensional FFT. For a given two-dimensional signal {x[z,m]:0≤z≤Z-1,0≤m≤M-1}, the two-dimensional discrete Fourier transform formula is as follows:

[0065]

[0066] Where x[z,m] represents the value of the two-dimensional signal at position [z,m], Z and M represent the number of rows and columns respectively, and u and v represent the horizontal frequency index and vertical frequency index of the two-dimensional frequency domain signal respectively; is the positive transformation rotation factor,

[0067] Used to convert the signal to the frequency domain, j is an imaginary unit;

[0068] (2.5.2) is multiplied by the second global learnable filter L2 To control the spectrum, get the filtered spectrum

[0069]

[0070] The dimension of the second global learnable filter L2 is the same as the two-dimensional spatial map output by the second encoder D2 and The dimensions are the same;

[0071] (2.5.3) Use two-dimensional IFFT to transform the filtered spectrum Transform back to the original space and get the two-dimensional feature F'2:

[0072]

[0073] The two-dimensional feature F'2 is the two-dimensional global filtering space feature map and Represents two-dimensional IFFT, the two-dimensional discrete Fourier inverse transform formula is as follows:

[0074]

[0075] Among them, X[u,v] represents the value of the two-dimensional signal at frequency [u,v], z and m represent the spatial index of the two-dimensional signal, Represents the value of the signal at position [z,m] after inverse transformation, is the normalization factor in the inverse transform; is the inverse transform rotation factor, which is used to convert the signal back to the original space.

[0076] (2.6) Construct a second feature fusion device S2 including a second element-by-element adder and a sigmoid activation function to transform the two-dimensional global filtering spatial feature map and Add them element by element, then input the activation function to fuse them into a two-dimensional 1×H×W global spatial weight M gs :

[0077]

[0078] Among them, σ(·) represents the activation function, ⊕ is the element-by-element addition, and M gs The global spatial feature map and The fused 2D global space weights.

[0079] (2.7) Set the expander E to convert the one-dimensional global channel weight M gcThe vector is expanded to a global channel weight M' of size C×H×W gc and the expander E converts the two-dimensional global spatial weight M gs Expanded to a global spatial weight M' of size C×H×W gs , the formulas are as follows:

[0080] M' gc = expand(M gc )

[0081] M' gs = expand(M gs )

[0082] Where expand(·) represents the dimension expansion operation, M gc is a one-dimensional global channel weight of size C×1×1, M gs is a two-dimensional global space weight of size 1×H×W.

[0083] (2.8) Construct a global attention fuser M including a third element-wise adder and a sigmoid activation function, such as Figure 3 As shown, it consists of a third element-by-element adder and a sigmoid activation function, which is used to expand the global channel weight M of size C×H×W gc and the global spatial weight M gs Add them element by element and aggregate them into the final three-dimensional global filter weight M through the sigmoid activation function gf :

[0084] M gf =σ(M' gc ⊕M' gs )

[0085] Among them, M gf is the three-dimensional global filter weight, M' gc is the global channel weight, M' gs is the global spatial weight, σ(·) represents the activation function, and ⊕ is the element-wise addition.

[0086] (2.9) Connect each part to form a global filter attention module GFA:

[0087] (2.9.1) Cascade the first encoder D1, the first global filter layer G1, and the first feature fuser S1 to form the first filter attention branch;

[0088] (2.9.2) Cascade the second encoder D2, the second global filter layer G2, and the second feature fuser S2 to form a second filter attention branch;

[0089] (2.9.3) Cascade the expander E and the global attention fusion unit M to form a fusion branch;

[0090] (2.9.4) The expander E is then connected to the first feature fuser S1 in the first branch and the second feature fuser S2 in the second branch to form a global filtering attention module GFA.

[0091] Step 3, add the global filter attention module GFA to the convolutional neural block in the existing convolutional neural network Net1 architecture to obtain the physical global filter attention convolutional neural network Net2.

[0092] The physical global filter attention convolutional neural network Net2 includes the initial convolution layer, the maximum pooling layer, several convolutional neural blocks, the global average pooling layer, the fully connected layer and the global filter attention module GFA constructed in step 2 in the convolutional neural network Net1. Each convolutional neural block includes a sub-convolutional layer, a normalization layer, an activation function, and an adder. The global filter attention module is connected between the convolutional layer and the adder. The structural relationship is as follows:

[0093] Initial convolution layer -> maximum pooling layer -> several convolutional neural blocks -> global average pooling layer -> fully connected layer.

[0094] The following example uses the convolutional neural network ResNet-34 network:

[0095] This example uses but is not limited to the ResNet-34 convolutional neural network, which is composed of a physical global filter attention convolutional neural network ResNet-34, such as Figure 4 As shown in the figure, the network includes an initial convolution layer, a maximum pooling, 32 convolution neural blocks, a global average pooling, and a fully connected layer. The convolution neural block consists of two sub-convolution layers, two normalization layers, two activation functions, an adder, and a global filter attention module GFA. Each convolution neural block consists of two parts, wherein the first part consists of a sub-convolution layer with a convolution kernel of 3×3, a normalization layer, and an activation function; the second part consists of a sub-convolution layer with a convolution kernel of 3×3 and a normalization layer. The output of the first part is connected to the input of the second part, and the output of the second part is connected to the adder. The global filter attention module GFA is connected between the input of the first part and the adder, and the output of the adder is connected to an activation function to form a physical global filter attention convolution neural network ResNet-34.

[0096] Step 4: Train the physics-based global filter attention convolutional neural network Net2 through back-propagation.

[0097] (4.1) The existing cross entropy loss function is defined as the global loss function L of the physics-based global filter attention convolutional neural network Net2:

[0098]

[0099] Where C is the total number of categories, y i is the true label. If category i is correct, then y i =1 otherwise y i =0, p i is the probability that the model predicts that it is category i, log(·) is the logarithm operation, and the base is the natural logarithm e;

[0100] (4.2) Input a batch of data in the training set into the network Net2, and calculate the global loss value of the training set based on the predicted category probability obtained by inputting the training set into the network;

[0101] (4.3) Back-propagate the global loss of the training set and iteratively update the network parameters;

[0102] (4.4) The global loss of the validation set is calculated based on the predicted category probability obtained by inputting a batch of data into the network, which is used to evaluate the performance of the current convolutional neural network;

[0103] (4.5) Repeat (4.2) to (4.4) until the global loss of the training set converges, and obtain the trained physics-based global filter attention convolutional neural network Net3.

[0104] Step 5: Input the test set into the trained physics-based global filter attention convolutional neural network Net3 to obtain the image classification results of the test set.

[0105] The effect of the present invention can be further illustrated by the following simulation results.

[0106] 1. Simulation conditions

[0107] 1. The simulation platform is a high-performance server computing platform with 56 CPUs and 502GB memory. Its operating system is Linux 4.4.0. Python3.8 and Pytorch1.1 are used to build and train the neural network model. Five different types of NVIDIA graphics cards and CUDA10.1 are used for acceleration. The NVIDIA graphics card models are 2 RTX2080Ti, 2 RTX2080Ti Rev and 1 TITAN V.

[0108] 2. The evaluation indicators used are Top-1 accuracy and Top-5 accuracy (%), and the Top-A accuracy calculation formula is:

[0109]

[0110] Where P is the total number of samples, y A is the true label of the a-th sample, is the first A categories predicted by the model for the a-th sample, 1(·) is an indicator function, which takes a value of 1 when the condition in the brackets is true, and 0 otherwise.

[0111] 2. Simulation content

[0112] Simulation 1: Under the above simulation conditions, the existing convolutional neural network ResNet-34 image classification method, the existing convolutional neural network image classification method based on the CBAM attention module, and the physical global filter attention convolutional neural network image classification method of the present invention are used to classify images on the CIFAR-10 dataset, and their respective Top-1 verification accuracy rates are calculated. The results are as follows: Figure 5 shown.

[0113] Depend on Figure 5 It can be seen that the initial image classification accuracy of the physical global filtering attention convolutional neural network image classification method of the present invention is 25.73% higher than that of the existing convolutional neural network ResNet-34 image classification method, and 11.34% higher than that of the existing CBAM attention module convolutional neural network image classification method. At the same time, it can also be seen that the accuracy curve of the physical global filtering attention convolutional neural network image classification method of the present invention reaches the convergence state 20 rounds earlier than the accuracy curve of the existing convolutional neural network ResNet-34 image classification method.

[0114] Simulation results show that the physical global filtering attention convolutional neural network image classification method of the present invention can accelerate the convergence speed of network training on image classification tasks, and can achieve higher accuracy in image classification tasks compared with the existing CBAM attention module convolutional neural network image classification method.

[0115] Simulation 2: Under the above simulation conditions, the existing convolutional neural network ResNet-34 image classification method, the existing convolutional neural network image classification method based on the CBAM attention module, and the physical global filter attention convolutional neural network image classification method of the present invention are used to classify images on the CIFAR-100 dataset, and their respective Top-1 verification accuracy rates are calculated. The results are as follows: Figure 6 shown.

[0116] Depend on Figure 6It can be seen that the initial image classification accuracy of the physical global filtering attention convolutional neural network image classification method of the present invention is 21.39% higher than that of the existing convolutional neural network ResNet-34 image classification method, and 19.56% higher than that of the existing CBAM attention module convolutional neural network image classification method. At the same time, it can also be seen that the image classification accuracy curve of the physical global filtering attention convolutional neural network image classification method of the present invention reaches the convergence state 20 rounds earlier than the image classification accuracy curve of the existing convolutional neural network ResNet-34 image classification method.

[0117] Simulation results show that in data sets with different numbers of categories, the physical global filtering attention convolutional neural network image classification method of the present invention still has good performance and can help convolutional neural networks achieve higher accuracy in image classification tasks.

[0118] Simulation 3. Under the above simulation conditions, the existing convolutional neural network ResNet-18, ResNet-34, and ResNet-50 image classification methods, the existing SE attention module convolutional neural network image classification methods using convolutional neural networks ResNet-18, ResNet-34, and ResNet-50, the existing CBAM attention module convolutional neural network image classification methods using convolutional neural networks ResNet-18, ResNet-34, and ResNet-50, the existing SimAM attention module convolutional neural network image classification methods using convolutional neural networks ResNet-18, ResNet-34, and ResNet-50, and the physical global filtering attention convolutional neural network image classification method based on convolutional neural networks ResNet-18, ResNet-34, and ResNet-50 of the present invention are respectively used for image classification on the CIFAR-10 dataset and the CIFAR-100 dataset, and the Top-1 accuracy (%) of each is calculated. The results are shown in Table 1.

[0119] Table 1 Image classification results on CIFAR-10 and CIFAR-100 datasets using different network classification methods

[0120]

[0121]

[0122] As can be seen from Table 1, on the CIFAR-10 and CIFAR-100 data sets, the Top-1 accuracy of the image classification method based on physical global filtering attention convolutional neural network using convolutional neural networks ResNet-18, ResNet-34, and ResNet-50 in the present invention is better than all existing convolutional neural network image classification methods and existing convolutional neural network image classification methods with other attention modules using convolutional neural networks ResNet-18, ResNet-34, and ResNet-50. The image classification accuracy of the image classification method based on physical global filtering attention convolutional neural network using convolutional neural networks ResNet-18, ResNet-34, and ResNet-50 on the CIFAR-10 data set is improved by an average of 1.71% over the image classification accuracy obtained by the existing convolutional neural network image classification method, and the image classification accuracy of the image classification method on the CIFAR-100 data set is improved by an average of 1.55% over the image classification accuracy obtained by the existing convolutional neural network image classification method.

[0123] Simulation 4. Under the above simulation conditions, the existing convolutional neural network ResNet-18, ResNet-34, ResNet-50, ResNet-101 image classification method, the existing convolutional neural network image classification method based on SE attention module using convolutional neural network ResNet-18, ResNet-34, ResNet-50, ResNet-101, the existing convolutional neural network image classification method based on CBAM attention module using convolutional neural network ResNet-18, ResNet-34, ResNet-50, ResNet-101, the existing convolutional neural network image classification method based on CBAM attention module using convolutional neural network ResNet-50, Re The convolutional neural network image classification method based on the BAM attention module of sNet-101, the existing convolutional neural network image classification method based on the AA attention module using the convolutional neural network ResNet-50 and ResNet-101, and the physical global filtering attention convolutional neural network image classification method based on the convolutional neural network ResNet-18, ResNet-34, ResNet-50, and ResNet-101 of the present invention are used to perform image classification on the ImageNet-1K data set, calculate the respective Top-1 and Top-5 accuracy (%), and record the overall parameters of each network, attention module parameters and the number of floating-point operations. The results are shown in Table 2.

[0124] Table 2 Image results of different network classification methods on the ImageNet-1K dataset

[0125]

[0126]

[0127] Where M represents one million and G represents one billion. It can be seen from Table 2 that the physical global filtering attention convolutional neural network image classification method of the present invention shows significant advantages, especially in terms of parameter quantity and computational complexity. For example, the parameter quantity of the attention module used in the physical global filtering attention convolutional neural network image classification method based on the convolutional neural network ResNet-18, ResNet-34, ResNet-50, and ResNet-101 of the present invention is only 0.007M and 0.008M, while the parameter quantity of the attention module based on the SE and CBAM attention module convolutional neural network image classification method using the convolutional neural network ResNet-50 and ResNet-101 reaches millions of levels, which verifies that the physical global filtering attention convolutional neural network image classification method based on the present invention significantly reduces the parameter quantity of the attention module. At the same time, the present invention provides an image classification accuracy comparable to or even higher than other image classification methods while maintaining a low number of floating-point operations. For example, the image classification Top-5 accuracy of the physical global filtering attention convolutional neural network image classification method based on the convolutional neural network ResNet-101 of the present invention reaches 94.11%, ranking first.

[0128] All these results indicate that the physical global filtering attention convolutional neural network image classification method designed in the present invention can help the network reduce computing resource consumption in image classification tasks and improve the image classification accuracy.

[0129] It should be noted that the step numbers in the specification and claims of the present invention are only for a clear description of the implementation scheme of the present invention to facilitate understanding, and the order of the step numbers is not limited.

Claims

1. An image classification method based on physical global filtering attention convolutional neural network, characterized in that: The steps include: (1) Obtain an image classification dataset and divide it into a training set and a test set in a ratio of 8:2; (2) Constructing the global filter attention module GFA: (2a) Construct the first encoder D1 consisting of a maximum pooling layer and an average pooling layer in the spatial dimension to soft-encode the global channel information and generate a one-dimensional channel vector and Construct a second encoder D2 consisting of a maximum pooling layer and an average pooling layer in the channel dimension to soft-encode the global spatial information and generate a two-dimensional spatial map and (2b) constructing a first global filter layer G1 including a one-dimensional fast Fourier transform, a first global learnable filter L1 and a one-dimensional inverse fast Fourier transform for the one-dimensional channel vector and Filter and generate global channel feature vector and Construct a second global filter layer G2 including a two-dimensional fast Fourier transform, a second global learnable filter L2 and a two-dimensional inverse fast Fourier transform for two-dimensional spatial graph and Filter and generate a global spatial feature map and (2c) Construct a first feature fuser S1 including a first element-by-element adder and a sigmoid activation function to convert the global channel feature vector and Fusion into a one-dimensional global channel weight M gc ; Construct a second feature fuser S2 including a second element-by-element adder and a sigmoid activation function to convert the global spatial feature map and Fusion into a two-dimensional global space weight M gs ; (2d) Set the expander E to convert the one-dimensional global channel weight M gc The vector is expanded to a global channel weight M' of size C×H×W gc , the two-dimensional global space weight M gs Expanded to a global spatial weight M' of size C×H×W gs , where C is the channel dimension, i.e. the number of channels, H and W are the height and width of the spatial dimension respectively; (2e) Construct a global attention fuser M including a third element-wise adder and a sigmoid activation function to expand the global channel weight M' to a size of C×H×W gc and the global spatial weight M' gs Add them element by element, and aggregate the results into a three-dimensional global filter weight M through the sigmoid activation function gf ; (2f) The first encoder D1, the first global filter layer G1, and the first feature fusion device S1 are cascaded to form a first filter attention branch; the second encoder D2, the second global filter layer G2, and the second feature fusion device S2 are cascaded to form a second filter attention branch; the expander E and the global attention fusion device M are cascaded to form a fusion branch; the expander E is respectively connected to the first feature fusion device S1 in the first branch and the second feature fusion device S2 in the second branch to form a global filter attention module GFA; (3) Adding the global filter attention module GFA to the residual block in the existing convolutional neural network Net1 architecture to obtain a physical global filter attention convolutional neural network Net2; (4) Input the training set into the physics-based global filter attention convolutional neural network Net2, train it through the back propagation method, and obtain the trained physics-based global filter attention convolutional neural network Net3; (5) The test set is input into the trained physics-based global filter attention convolutional neural network Net3 to obtain the image classification results of the test set.

2. The method according to claim 1, characterized in that: In step (2a), the first encoder D1 soft encodes the global channel information and the second encoder D2 soft encodes the global spatial information, which are respectively implemented as follows: Given a C×H×W feature F, where C is the channel dimension, i.e. the number of channels, and H and W are the height and width of the spatial dimension respectively; The maximum pooling operation is performed through the spatial dimension maximum pooling layer to obtain a one-dimensional channel maximum pooling vector of C×1×1 The average pooling operation is performed through the spatial dimension average pooling layer to obtain a one-dimensional channel average pooling vector of C×1×1 The maximum pooling operation is performed through the channel dimension maximum pooling layer to obtain a 1×H×W two-dimensional space maximum pooling map The average pooling operation is performed through the channel dimension average pooling layer to obtain a 1×H×W two-dimensional average pooling map That is, the channel dimension of the original feature F is reduced to 1, while the spatial dimension remains unchanged.

3. The method according to claim 1, characterized in that Step (2b) The first global filter layer G1 performs a one-dimensional channel vector and To filter, the implementation steps include the following: (2b1) The one-dimensional channel vector output by the upper layer after pooling and Perform a one-dimensional FFT conversion to the frequency domain: Where F1 represents the one-dimensional channel vector and is a complex tensor, denoted as the spectrum of F1, Represents one-dimensional FFT; (2b2) ​​is multiplied by the first global learnable filter L1 Get the filtered spectrum (2b3) Use one-dimensional IFFT to transform the filtered spectrum Transform back to the original space and get the one-dimensional feature F'1: The one-dimensional feature F'1 is the one-dimensional global filter channel feature vector and Represents a one-dimensional IFFT.

4. The method according to claim 1, characterized in that Step (2b) The second global filter layer G2 performs a two-dimensional spatial image and To filter, the implementation steps include the following: (2b4) The two-dimensional spatial map output by the upper layer after pooling and Perform a two-dimensional FFT conversion to the frequency domain: Where F2 represents the two-dimensional space graph and is a complex tensor, denoted as the spectrum of F2, Represents a two-dimensional FFT; (2b5) is multiplied by the second global learnable filter L2 Get the filtered spectrum (2b6) Use two-dimensional IFFT to transform the filtered spectrum Transform back to the original space and get the two-dimensional feature F'2: The two-dimensional feature F'2 is the two-dimensional global filtering space feature map and Represents a two-dimensional IFFT.

5. The method according to claim 1, characterized in that: Step (2c) The first feature fusion unit S1 transforms the global channel feature vector and Fusion into a one-dimensional global channel weight M gc ; The second feature fusion device S2 converts the global spatial feature map and Fusion into a two-dimensional global space weight M gs , the formulas are as follows: Among them, σ(·) represents the activation function, is element-wise addition, M gc is the global channel eigenvector and Fusion into a one-dimensional global channel weight. Mgs is composed of the global spatial feature map and The fused 2D global space weights.

6. The method according to claim 1, characterized in that In step (2d), the expander E converts the one-dimensional global channel weight M gc The vector is expanded to a global channel weight M' of size C×H×W gc and the expander E converts the two-dimensional global spatial weight M gs Expanded to a global spatial weight M' of size C×H×W gs , the formulas are as follows: M' gc =expand(M gc ) M' gs =expand(M gs ) Where expand(·) represents the dimension expansion operation, M gc is a one-dimensional global channel weight of size C×1×1, M gs is a two-dimensional global space weight of size 1×H×W.

7. The method according to claim 1, characterized in that In step (2e), the global channel weight M' is activated by the sigmoid activation function. gc and the global spatial weight M' gs The element-by-element addition results are aggregated into a three-dimensional global filter weight M gf , the formula is as follows: Among them, M gf is the three-dimensional global filter weight, M' gc is the global channel weight, N' gs is the global spatial weight, σ(·) represents the activation function, is element-wise addition.

8. The method according to claim 1, characterized in that The physical global filter attention convolutional neural network Net2 obtained in step (3) includes the initial convolutional layer, the maximum pooling layer, several residual blocks, the global average pooling layer, the fully connected layer and the global filter attention module GFA constructed in step (2) in the convolutional neural network Net1. Each residual block includes a sub-convolutional layer, a normalization layer, an activation function, and an adder. The global filter attention module is connected between the convolutional layer and the adder. The structural relationship is as follows: Initial convolutional layer -> maximum pooling layer -> several residual blocks -> global average pooling layer -> fully connected layer.

9. The method according to claim 1, characterized in that: Step (4) trains the physics-based global filter attention convolutional neural network Net2 by back propagation method, and the implementation steps include the following: (4a) Define the existing cross entropy loss function as the global loss function L of the physics-based global filter attention convolutional neural network Net2: Where C is the total number of categories, y i is the true label. If category i is correct, then y i =1 otherwise y i =0, p i is the probability that the model predicts category i; (4b) Input a batch of data in the training set into the network Net2, and calculate the global loss value of the training set based on the predicted category probability obtained by inputting the training set into the network; (4c) Back-propagate the global loss of the training set and iteratively update the network parameters; (4d) The global loss of the validation set is calculated based on the predicted category probability obtained by inputting a batch of data into the network, which is used to evaluate the performance of the current convolutional neural network; (4e) Repeat (4b) to (4d) until the global loss of the training set converges, and obtain the trained physics-based global filter attention convolutional neural network Net3.

Images