CNN (Convolutional Neural Network) and Transform combined classification method based on reinforcement learning

Abstract

The invention provides a convolutional neural network (CNN) and Transform combined classification method based on reinforcement learning, which is characterized in that a convolutional neural network (CNN), a reinforced feature sampling module (RFS), a Transform and a Head output module which are connected in sequence are used, a Hard Attention mechanism is adopted, an input feature map is sampled in a parallel mode, and a classification result is obtained by combining the convolutional neural network (CNN), the reinforced feature sampling module (RFS), the Transform and the Head output module. According to the method, reasonable feature sampling is carried out from a low-resolution feature map by utilizing a hard decision-making mechanism, so that a large amount of redundant noise data is eliminated, the data volume processed by the Transform model is reduced, and the overall calculation efficiency of the hybrid model is improved; and the method has good interpretability and model performance.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and computer vision, and is a method for fusing a convolutional neural network with a Transformer model, which is applied to processing image classification problems in the field of computer vision. Background Art

[0002] In the field of computer vision, convolutional neural networks (CNN) and Transformer are two widely used deep learning models. CNN has powerful image feature extraction and hierarchical representation learning capabilities, and performs well in computer vision tasks such as image classification, object detection, and semantic segmentation. Transformer was originally applied in the field of natural language processing and was later introduced into the field of computer vision. It can capture long-distance dependencies through the self-attention mechanism and has excellent global modeling capabilities.

[0003] Convolutional neural networks (CNNs) have the advantages of strong local perception, strong robustness, and strong scalability, but they also have some limitations, such as (1) lack of perception of global information, resulting in poor processing of long sequences; (2) CNN uses convolution operations to extract features, which will cause the position information of the input data to be lost; (3) when the convolution kernel and image size are large, the amount of calculation is large and requires higher computing resources; (4) due to the parameter sharing mechanism, it performs poorly for some tasks that require consideration of details, such as image super-resolution tasks.

[0004] Although the Transformer model has a natural advantage in capturing long-distance dependencies, it has high requirements for data sets, lacks the inductive bias characteristics of CNN, and has high computational complexity. Especially when processing high-resolution data such as images, the growth rate of computational complexity is the quadratic power of the image size.

[0005] In recent years, more and more research has been devoted to combining the two models to make full use of the advantages of their respective architectures, while maintaining computational efficiency and improving the performance of the models in a variety of computer vision tasks. The most direct idea is to introduce the multi-scale and residual structures commonly found in CNN into the visual transformer. Wang et al. referred to and introduced the pyramid structure in the CNN architecture to improve the effect of Vision Transformer (ViT) when processing high-resolution images, and proposed the Pyramid Vision Transformer model, namely PVT. PVT adopts a progressive shrinkage strategy, using a patch embedding layer and an encoder with a spatial reduction mechanism to achieve flexible adjustment of the feature map scale, which enables PVT to generate multi-scale feature maps like the traditional CNN model while reducing computational costs.

[0006] HRFormer introduces Transformer based on the multi-resolution feature fusion architecture of HRNet. This allows the model to have the advantages of a multi-resolution feature fusion architecture while also being able to better model long-distance dependencies and capture more global semantic information.

[0007] Carion et al. proposed the Detection Transformer (DETR) by connecting the CNN feature extraction method to the Transformer, and used the Transformer for image object detection for the first time. This architecture uses the CNN network to extract the low-resolution two-dimensional feature map of the image, and then reshapes the feature map into a series of feature sequences and encodes their positions. The results are input into the Transformer for learning to obtain classification labels and prediction boxes. This method effectively reduces the input size of the Transformer, making it easier for the Transformer to quickly learn the global features of the input image, and improves the learning speed and overall performance of the model.

[0008] Dai et al. adopted the same splicing strategy and proposed CoAtNet. They believed that convolution blocks are better at acquiring local priors, so they should be designed before the Transformer module. They introduced the characteristics of CNN and Transformer into the model by using the reverse residual bottleneck MBConv module and relative self-attention module based on deep convolution, and successfully integrated the advantages of CNN's translation equivariance and Transformer's input-adaptive weighting and global receptive field into a single architecture, obtaining a hybrid model with generalization ability, model capacity and model efficiency.

[0009] The ConT block designed by Yan et al. adopts the serial splicing strategy of Transformer and CNN, and builds ConTNet on this basis. They regard the standard Transformer encoder as an independent component identical to the convolution block, and then connect the two encoders in series with a convolution layer with a convolution kernel size of 3×3, and name it the ConT block. After entering the network, the image will first pass through a convolution layer with a convolution kernel size of 7×7 and a maximum pooling layer, then pass through a framework composed of multiple ConT blocks stacked together, and finally pass through a global average pooling layer and a fully connected layer to achieve related computer vision tasks. This not only utilizes the powerful representation ability of Transformer, but also has the bias induction characteristics of CNN. The model has the advantages of easy optimization and robustness, and does not rely on powerful data enhancement and training techniques, and has good transfer learning capabilities.

[0010] In summary, the combination of convolutional neural networks and Transformer models has been widely studied in computer vision research, and a variety of fusion methods have been used to improve the overall effect of the model. The purpose is to make full use of the respective characteristics of convolutional neural networks and Transformer models and achieve complementary advantages.

[0011] At present, the industry has made a lot of progress in the combination of the two models, and proposed a variety of model architectures, which have achieved certain results. However, there are still obvious shortcomings when processing image data: 1. When processing high-resolution or large-size images, the current solution uses a convolutional neural network to perform downsampling to obtain a low-resolution feature map of the image. However, the Transformer model still has a very high computational complexity when processing the feature map, which limits the use of the model.

[0012] 2. There is a lot of data in the image that is redundant for the model. The calculation method of the Tranformer model will introduce a lot of unnecessary noise, which will reduce the calculation efficiency of the model and also affect the accuracy of the model; 3. The current combination method is not interpretable enough. For example, in image classification tasks, sometimes we need to pay attention to what the model has learned, or learn which area of ​​the image is important. The current combination mechanism is essentially based on the soft attention model, which is not as interpretable as the hard attention model. Summary of the invention

[0013] Different from the existing combination methods, the present invention considers the respective characteristics of convolutional neural network (CNN) and Transformer, starts from attention and reinforcement learning, and designs a reinforced feature sampling module (Reinforced Feature Sampling), hereinafter referred to as RFS. This module adopts a hard attention mechanism to sample the input feature map in parallel, and then uses these sampled features to send them to the subsequent Transformer model for further processing. This method uses a hard decision mechanism to perform reasonable feature sampling from low-resolution feature maps, which not only eliminates a large amount of redundant noise data, reduces the amount of data processed by the Transformer model, and improves the overall computational efficiency of the hybrid model, but also has good interpretability and model performance.

[0014] In order to achieve the above-mentioned object of the invention, the technical solution of the present invention is implemented in the following manner: A classification method combining convolutional neural network and Transformer based on reinforcement learning, which uses a convolutional neural network (CNN), a reinforced feature sampling module (RFS), a Transformer and a Head output module connected in sequence, and the method steps are as follows: (1) First, the input image is passed through the CNN backbone network to extract the feature map; (2) The feature map is input into the RFS module for sampling processing, which is divided into the prediction stage and the training stage: A) Training phase: In this phase, the actor module and the critic module in RFS work simultaneously. The calculation method of the actor module and the critic module can be expressed as: (1) (2) Among them, x is the input feature map, It is the output of the actor module, with a channel number of 1 and a size consistent with the actor input feature map size. Its value range is 0~1 and can be regarded as a sampling probability. v represents the expected value (reward) of the current feature map, which is a scalar.

[0015] The actor module uses a hard decision mechanism to Parallel independent sampling, here For the sampling probability of each grid, we have: (3) (4) For the current The joint probability after performing parallel sampling, is the sampling probability of each grid, label is the sampling value, which can be 0 or 1.

[0016] Since the RFS module uses hard decision sampling, the gradient is not differentiable. Here, the Proximal Policy Optimization (PPO) algorithm in the field of reinforcement learning is used for training. Because reinforcement learning training requires a reward function, here, according to the characteristics of the classification task and the number limit of the sampled feature sequences, the reward function is defined as follows: (5) Among them, p is the prediction result of the classification module corresponding to the label, Indicates the number of grids whose values ​​are 0 when the actor module performs sampling. is the sum of all grid numbers, i.e. the size of the feature map output by the actor. The actor module uses Bernoulli sampling when training: (6) Output values ​​for actor modules, The value is 0 or 1, 1 means the feature corresponding to the grid is selected, otherwise, it is not selected. The distribution of Bernoulli distribution.

[0017] B) Prediction phase: In the prediction phase of the RFS module, only the actor module works. The actor module uses the following method to parallel sample: (7) , As defined above.

[0018] (3) The feature sequence sampled in step 2 is sent to the Transformer module for encoding; (4) The Head output module classifies the encoded feature sequence. This is also divided into a training phase and a prediction phase. In the training phase, the output value corresponding to the label is taken and substituted into the reward function, i.e., training in Formula 5. In the prediction phase, the final classification result is directly output.

[0019] The present invention uses an enhanced feature sampling module to effectively combine CNN and Transformer. Compared with the prior art, the present invention has the following advantages: 1. The enhanced feature sampling module (RFS) performs sampling in parallel, which is more efficient and flexible than the previous method; 2. The model can automatically select certain feature data of the image for classification, which greatly reduces redundant data and improves the performance and computational efficiency of the model; 3. Through the enhanced feature sampling module (RFS) module, only a part of the feature data is selected for the feature map output by the convolutional network, which greatly reduces the amount of data that the Transformer needs to process, improves the computational efficiency of the Transformer, and can be further extended to process large-size images; 4. By selecting features through hard decision-making, you can intuitively see which areas of the image are involved in the classification task, and understand what the model has learned. It has better interpretability than soft attention.

[0020] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 The overall flow chart of the method of the present invention is as follows; Figure 2 This is the structural diagram of the enhanced feature sampling module; Figure 3 This is the network structure diagram of the action module actor and the evaluation module critic. DETAILED DESCRIPTION The present invention is a classification method based on a convolutional neural network and a Transformer combined with reinforcement learning. It uses a convolutional neural network (CNN), a reinforced feature sampling module (RFS), a Transformer, and a Head output module connected in sequence. The method steps are as follows: 1. Convolutional neural network CNN extracts feature maps The present invention designs the backbone network based on the ResNet50 network. The standard ResNet50 includes 4 blocks. Here, only the output of the first two blocks, that is, the second block, is used as the extracted feature map. Here, the input image size is 3*224*224, where 3 is the number of image channels, and the following two numbers represent the height and width of the image, respectively. Therefore, the feature map size output by the backbone network is batchsize*512*28*28, where batchsize represents the number of samples in a batch.

[0023] 2. Enhanced feature sampling module (RFS) samples feature maps The RFS module consists of two parts: the action module (actor) and the evaluation module (critic), such as Figure 2 As shown in the figure, the actor module uses a hard decision mechanism to sample the input feature map in parallel. A sampling value of 1 indicates that the corresponding grid is selected, and 0 indicates that it is not selected. The critic module is used to calculate the value of the current input feature map, which is a scalar. Note that the dotted box in RFS, i.e., the critic module, is only used during training, and only the actor module is used during normal reasoning.

[0024] The structure of actor module and critic module is as follows Figure 3 As shown in the figure, the actor module consists of three convolutional layers and a Sigmoid activation function. The first two convolution kernels are both 3 in size, and the number of input and output channels is 512. The third convolution kernel is 1 in size and the number of output channels is 1. The critic module has the first three convolution kernels of 3 in size, and the number of input and output channels is 512. The MaxPool pooling module has a step size of 2, a pooling kernel size of 2, a fourth convolution kernel size of 1, and an output channel of 1. The last AvgPool is a global pooling operation. The calculation method of the actor module and the critic module can be expressed as: (8) (9) Where x is the input feature map, is the output of the actor module, with 1 channel and the same size as the actor input feature map, because the last module is the Sigmoid activation function. The value range of is 0~1, which can be regarded as sampling probability. v represents the expected value (reward) of the current feature map, which is a scalar. The working mode of the RFS module is divided into two different modes according to the prediction stage and the training stage: A) Training phase: In this phase, the actor module and the critic module in RFS work simultaneously. The calculation methods of the actor module and the critic module are as shown in formulas 8 and 9. The actor module uses a hard decision mechanism to Parallel independent sampling, here For the sampling probability of each grid, we have: (10) (11) For the current The joint probability after performing parallel sampling, is the sampling probability of each grid, label is the sampling value, which can be 0 or 1.

[0025] Since the RFS module uses hard decision sampling, the gradient is not differentiable. Here, the Proximal Policy Optimization (PPO) algorithm in the field of reinforcement learning is used for training. Because reinforcement learning training requires a reward function, here, according to the characteristics of the classification task and the number limit of the sampled feature sequences, the reward function is defined as follows: (12) Where p is the prediction result of the classification module corresponding to the label, Indicates the number of grids whose values ​​are 0 when the actor module performs sampling. is the sum of all grid numbers, i.e. the size of the feature map output by the actor. The actor module uses Bernoulli sampling when training: (13) Output values ​​for actor modules, The value is 0 or 1, 1 means the feature corresponding to the grid is selected, otherwise, it is not selected. The distribution of Bernoulli distribution.

[0026] B) Prediction phase: In the prediction phase of the RFS module, only the actor module works. The actor module uses the following method to parallel sample: (14) , As defined above.

[0027] 3. Send the feature sequence sampled in step 2 to the Transformer module for encoding The classic Transformer model consists of an encoder and a decoder, and only the encoder is used here. In this design, the encoder is composed of three encoding layers with the same structure. The encoding layer is mainly composed of a multi-head attention layer and a feedforward layer. The number of neurons in the multi-head attention layer is set to 512, the number of attention heads is set to 4, and the dimension of the feedforward layer is set to 1024.

[0028] 4. The Head output module classifies the encoded feature sequence This layer mainly processes the feature sequence encoded by the Transformer model, fuses the feature sequence and outputs the classification result. The Head module consists of a multi-dimensional attention module and a linear classifier. The calculation method of the Head module is as follows: (15) Among them, seq is the input feature sequence, the dimension is n*d, n is the number of sequences, d is the feature dimension, It means that the softmax operation is performed on the matrix x along dimension 1, that is, the multi-dimensional attention operation. It means to perform a sum operation on the matrix x along dimension 1. are the linear classifier parameters.

[0029] The processing of the Head module is also divided into the training stage and the prediction stage. When it is the training stage, the output value corresponding to the label is taken and brought into the reward function, that is, the training in Formula 12. When it is the prediction stage, the final classification result is directly output.

[0030] The embodiments of the present invention are only for illustrating the technical solutions of the present application. Similar substitutions made by those skilled in the art on the basis of the present application, such as replacing the convolutional neural networks in the action module and the evaluation module with other deep learning models or machine learning solutions, or replacing the backbone network with other network structures, shall all fall within the scope of protection of the present application.

Claims

1. A classification method combining CNN and Transformer based on reinforcement learning, characterized in that: A convolutional neural network (CNN), an enhanced feature sampling module (RFS), a Transformer, and a Head output module are connected in sequence. The enhanced feature sampling module includes two submodules, an action module and an evaluation module, which are applied to combine CNN and Transformer, and include the following steps: Step 1: Extract feature maps from the input image through the CNN backbone network; Step 2: Input the feature map into the RFS module for sampling processing, which is divided into the prediction stage and the training stage; Step 3: Send the feature sequence sampled in step 2 to the Transformer module for encoding. Step 4: The Head output module classifies the encoded feature sequence, which is divided into training stage and prediction stage.

2. The CNN and Transformer combined classification method based on reinforcement learning as claimed in claim 1, characterized in that: Step 2: The RFS module consists of two parts: an action module (actor) and a critic module (critic).

3. The CNN and Transformer combined classification method based on reinforcement learning as described in claim 1 and claim 3, characterized in that: The working mode of the RFS module is divided into two different modes: prediction stage and training stage; in the training stage, the actor module and the critic module in the RFS work simultaneously, the RFS module adopts hard decision sampling, and adopts the proximal policy optimization algorithm (PPO) algorithm in the field of reinforcement learning for training, and Bernoulli sampling is adopted during actor module training; in the prediction stage, only the actor module of the RFS module works.

4. The CNN and Transformer combined classification method based on reinforcement learning as claimed in claim 1, characterized in that: In step 3, only the encoder operation is used. The encoder is composed of three encoding layers with the same structure. The encoding layer is mainly composed of a multi-head attention layer and a feedforward layer. The number of neurons in the multi-head attention layer is set to 512, the number of attention heads is set to 4, and the dimension of the feedforward layer is set to 1024.

5. The CNN and Transformer combined classification method based on reinforcement learning as claimed in claim 1, characterized in that: Step 4 mainly processes the feature sequence encoded by the Transformer model, fuses the feature sequence and outputs the classification result. The Head module includes a multi-dimensional attention module and a linear classifier.

6. The CNN and Transformer combined classification method based on reinforcement learning as described in claim 1 and claim 6, characterized in that: The processing of the Head module is also divided into the training phase and the prediction phase. In the training phase, the output value corresponding to the label is taken and brought into the reward function. In the prediction phase, the final classification result is directly output.

Images