A neural network regularization method based on attention mechanism

Abstract

The present invention discloses a neural network regularization method based on an attention mechanism. The present invention comprises the following steps: (1) batch image input and preprocessing; (2) feature extraction using a CNN convolutional neural network; (3) key feature screening using an attention mechanism; (4) exchanging key features of different target images; (5) key features and non-key features of different images together constitute the semantic information of the reconstructed image; (6) designing an adaptive confidence estimation method to determine the category to which the reconstructed image belongs; (7) using the reconstructed image to train the neural network to obtain a more accurate training model. The present invention improves the robustness and overall performance of the model by sample expansion and fully mining the potential features of the image. The advantage of the present invention is that the attention mechanism is used to guide the adaptive reconstruction of image features, and the application range is wide, the model has high accuracy, and strong robustness.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, in particular to an attention mechanism in deep learning and a neural network regularization method, and a method that combines feature reconstruction and adaptive confidence estimation. Background Art

[0002] In recent years, with the development of deep neural networks, deep learning tasks in the fields of object recognition, semantic segmentation, and image and text description have become increasingly complex and diverse. Faced with complex tasks, wider widths and deeper models tend to have stronger expressive power. However, larger models mean more parameters, which requires more computer resources: memory usage, parameters, operation counts, inference time, and power consumption.

[0003] In particular, when the neural network reaches a certain depth, due to overfitting and other reasons, the accuracy no longer improves with the depth. The larger the model, the easier it is to fit. The reason is that there are few data samples, insufficient data completeness, or it is difficult to support the optimization of huge parameters, which leads to high training accuracy and low test accuracy. Therefore, network regularization technology is a good way to improve the robustness and overall performance of the model. Summary of the invention

[0004] The purpose of the present invention is to address the overfitting problem that is easily caused by insufficient data completeness during complex model training. The present invention designs a neural network regularization method based on the attention mechanism, aiming to perform sample expansion and fully explore the potential features of the image to improve the robustness and overall performance of the model. First, the image semantic information is obtained through the CNN convolutional neural network, and then the key features of the image are screened and extracted using the attention mechanism. Then an adaptive confidence estimation method is introduced to guide the reconstruction of the key features of different images, and finally the reconstructed feature map is used to train the neural network to obtain better training results. The advantage of the present invention is that the attention mechanism is used to guide the adaptive reconstruction of image features, which has a wide range of applications, high model accuracy, and strong robustness.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0006] Step 1: Preprocessing of batch images: including random cropping, random flipping, and regularization operations;

[0007] Step 2: Use the weight-shared encoder network for feature extraction: Input the preprocessed training image into the Resnet model, extract the high-dimensional feature map from any intermediate layer of the Resnet model, and the network before the intermediate layer is the weight-shared encoder network. The extracted high-dimensional feature map will be used for subsequent feature reconstruction, and the feature map after feature reconstruction will be re-input into the network after the intermediate layer for training;

[0008] Step 3: Use the attention mechanism to screen key features: high-dimensional feature maps are obtained through step 2, and these high-dimensional feature maps are input into the attention extraction network SENet, and the attention mechanism is used to screen and extract key features in the feature maps;

[0009] Step 4: Exchange the key features of different target images: randomly select the key features on one feature map to replace the key features on another feature map, and obtain a new reconstructed feature map, which contains the information of the targets of the two images;

[0010] Step 5: The key features of the current image and the supplementary features of another image together constitute the semantic information of the reconstructed image: the reconstructed feature map contains the key features on one feature map and the supplementary features on another feature map;

[0011] Step 6: Design an adaptive confidence estimation method to determine the category of the reconstructed image: the new reconstructed feature map contains the target information of the two images. The adaptive confidence estimation method is used to evaluate the probability of the target category to which the reconstructed feature map belongs.

[0012] Step 7: Use the reconstructed image to train the neural network to obtain a more accurate training model: The reconstructed feature map and its labels are obtained through the above steps. Therefore, the reconstructed feature map can be used for network training to fully tap the learning potential of the neural network and enhance the robustness of the model.

[0013] After testing, the advantages of the present invention compared with the previous network regularization method are:

[0014] 1. Use the attention mechanism to adaptively filter and reorganize features, and integrate the attention mechanism into the neural network regularization method;

[0015] 2. A feature map reconstruction method based on feature level, and the introduction of attention as a guide for feature reconstruction. Without changing the original image information, the feature information can be effectively deployed and combined to conduct purposeful training and maximize network performance;

[0016] 3. The method proposed in the invention can be applied to various network models for various tasks such as target detection and segmentation. It has a wide range of applications, is easy to insert, has high model accuracy and strong robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 Flowchart of execution of the method of the present invention

[0018] Figure 2 Schematic diagram of the invention applied to deep learning model construction DETAILED DESCRIPTION

[0019] The specific embodiments of the present invention will be further described below in conjunction with the accompanying drawings.

[0020] Different from the previous image-level region discarding or hybrid regularization methods, the present invention proposes a regularization method based on feature screening and adaptive reconstruction of the attention mechanism. In particular, the invention improves a powerful attention mechanism, performs feature screening at the feature layer, and then extracts key features that are helpful for the task for reconstruction, and proposes an adaptive confidence method to evaluate the target attributes of the reconstructed image. The method proposed by the invention acts on the feature level of the neural network, which can fully tap the learning potential of the neural network and greatly improve the robustness and overall capabilities of the convolutional neural network. Moreover, it can be used in combination with existing data expansion methods to further improve the performance of the model at different levels. It has the characteristics of wide application range, high model accuracy, and strong robustness.

[0021] This paper mainly proposes a neural network regularization method based on attention mechanism to improve the accuracy and robustness of the algorithm model. Figure 1 It is the specific implementation step of the present invention:

[0022] Step 1: Preprocessing of batch images: including random cropping, random flipping, and regularization operations;

[0023] Step 2. Use the weight-shared encoder network for feature extraction: Input the preprocessed training image into the Resnet model, extract the high-dimensional feature map from any intermediate layer of the Resnet model, and the network before the intermediate layer is the weight-shared encoder network. The extracted high-dimensional feature map will be used for subsequent feature reconstruction, and the feature map after feature reconstruction will be re-input into the network after the intermediate layer for training; such as the feature maps output by the four blocks of the Resnet network.

[0024] Step 3: Use the attention mechanism to screen key features: high-dimensional feature maps are obtained through step 2, and these high-dimensional feature maps are input into the attention extraction network SENet, and the attention mechanism is used to screen and extract key features in the feature maps;

[0025] Step 4: Exchange the key features of different target images: randomly select the key features on one feature map to replace the key features on another feature map, and obtain a new reconstructed feature map, which contains the information of the targets of the two images;

[0026] Step 5: The key features of the current image and the supplementary features of another image together constitute the semantic information of the reconstructed image: the reconstructed feature map contains the key features on one feature map and the supplementary features on another feature map;

[0027] Step 6: Design an adaptive confidence estimation method to determine the category of the reconstructed image: the new reconstructed feature map contains the target information of the two images. The adaptive confidence estimation method is used to evaluate the probability of the target category to which the reconstructed feature map belongs.

[0028] Step 7: Use the reconstructed image to train the neural network to obtain a more accurate training model: The reconstructed feature map and its labels are obtained through the above steps. Therefore, the reconstructed feature map can be used for network training to fully tap the learning potential of the neural network and enhance the robustness of the model.

[0029] Furthermore, the batch of images input in step 1 can be from any dataset.

[0030] Furthermore, the specific steps of step 2 to obtain the feature map are as follows: after the image is input into the Resnet model, a middle layer is randomly selected for regularization operation; the network before the middle layer is a weight-shared encoder network; the output of the middle layer is the high-dimensional feature map of the batch image to be extracted, which is recorded as Where c, h, and w represent the number of channels, height, and width of the feature map, respectively.

[0031] Furthermore, the steps of using the attention mechanism to screen key features in step 3 are as follows:

[0032] 4-1 Input the high-dimensional feature map obtained in step 2 into the channel attention network SENet for attention scoring. The higher the attention score, the greater the contribution of the feature map to the classification performance.

[0033] 4-2 Sort the feature maps of c channels according to the attention scores;

[0034] Furthermore, the key features of the target image are extracted in step 4, specifically, the partial channel feature map with the highest attention score is selected, and the partial channel is c / 2 channels.

[0035] Furthermore, the key features of the current image in step 5 and the supplementary features of the other image together constitute the semantic information of the reconstructed image. The key features come from step (4), and the supplementary features come from the feature map corresponding to the other image except the key channel. Assuming that the image contains 10 channels in total, the channels corresponding to the key feature map of image A are [2, 3, 4, 6, 8]. Then the features of the reconstructed image come from the [2, 3, 4, 6, 8] channels of the feature map of A and the [0, 1, 5, 7, 9] channels of the feature map of B.

[0036] Furthermore, the steps of the adaptive confidence estimation method in step 6 are specifically:

[0037] 6-1 According to step 5, a reconstructed feature map containing target information of two images is obtained;

[0038] 6-2 Re-input the reconstructed feature map into the intermediate layer extracted in step 2 of the Resnet model, train the entire Resnet network, and obtain the prediction probabilities of different categories in the final network output layer

[0039] 6-3 Select the probability P of the two categories to which the original image corresponding to the reconstructed feature map belongs from the predicted probability A and P B As a result of the adaptive confidence, such as P A =0.4,P B =0.2, then the adaptive confidence is:

[0040] [λ A ,λ B ]=soft max([P A ,P B ])=[0.67,0.33]

[0041] 6-4 The original image category corresponding to the reconstructed feature map and the adaptive confidence together constitute the training label of the reconstructed image. The objective function of the training can be expressed as:

[0042]

[0043] where λ A ,λ B are the target categories y A and B The confidence level is calculated by using the cross entropy loss function L CE , is the predicted output;

[0044] In step 7, a regularization method based on the attention mechanism is applied to reconstruct the image feature map, and the reconstructed feature map is used to train the neural network to obtain a more accurate training model; these reconstructed feature maps are adaptively generated by the neural network according to the attention guidance, so the reconstructed feature maps can be used to guide the network to train in a larger sample space, so as to fully tap the learning potential of the neural network, enhance the robustness of the model, and obtain a better training model.

Claims

1. A neural network regularization method based on attention mechanism, characterized in that The following steps are involved: Step 1: Preprocessing of batch images: including random cropping, random flipping, and regularization operations; Step 2: Use the weight-shared encoder network for feature extraction: Input the preprocessed training image into the Resnet model, extract the high-dimensional feature map from any intermediate layer of the Resnet model, and the network before the intermediate layer is the weight-shared encoder network. The extracted high-dimensional feature map will be used for subsequent feature reconstruction, and the feature map after feature reconstruction will be re-input into the network after the intermediate layer for training; Step 3: Use the attention mechanism to screen key features: Get the high-dimensional feature maps through step 2, input these high-dimensional feature maps into the attention extraction network SENet, and use the attention mechanism to screen and extract the key features in the feature maps; Step 4: Exchange the key features of different target images: randomly select the key features on one feature map to replace the key features on another feature map, and obtain a new reconstructed feature map, which contains the information of the targets of the two images; Step 5: The key features of the current image and the supplementary features of another image together constitute the semantic information of the reconstructed image: the reconstructed feature map contains the key features on one feature map and the supplementary features on another feature map; Step 6: Design an adaptive confidence estimation method to determine the category of the reconstructed image: the new reconstructed feature map contains the target information of the two images. The adaptive confidence estimation method is used to evaluate the probability of the target category to which the reconstructed feature map belongs. Step 7: Use the reconstructed image to train the neural network to obtain a more accurate training model: The reconstructed feature map and its labels are obtained through the above steps. Therefore, the reconstructed feature map can be used for network training to fully tap the learning potential of the neural network and enhance the robustness of the model.

2. A neural network regularization method based on attention mechanism according to claim 1, characterized in that The batch of images input in step 1 can be from any dataset.

3. A neural network regularization method based on attention mechanism according to claim 1, characterized in that The specific steps of step 2 to obtain the feature map are as follows: after the image is input into the Resnet model, a middle layer is randomly selected for regularization operation; the network before the middle layer is a weight-shared encoder network; the output of the middle layer is the high-dimensional feature map of the batch image to be extracted, denoted as Where c, h, and w represent the number of channels, height, and width of the feature map, respectively.

4. A neural network regularization method based on attention mechanism according to claim 3, characterized in that The specific steps of using the attention mechanism to screen key features in step 3 are: 4-1 Input the high-dimensional feature map obtained in step 2 into the channel attention network SENet for attention scoring. The higher the attention score, the greater the contribution of the feature map to the classification performance. 4-2 Sort the feature maps of c channels according to the attention scores.

5. A neural network regularization method based on attention mechanism according to claim 4, characterized in that The key features in the feature map extracted in step 3 are specifically selected from the partial channel feature map with the highest attention score, and the partial channel is selected from c / 2 channels.

6. A neural network regularization method based on attention mechanism according to claim 5, characterized in that In step 5, the key features of the current image and the supplementary features of another image together constitute the semantic information of the reconstructed image. The key features come from step 4, and the supplementary features come from the feature map corresponding to the other image except the key channel.

7. A neural network regularization method based on attention mechanism according to claim 6, characterized in that The steps of the adaptive confidence estimation method in step 6 are specifically as follows: 6-1 obtaining a reconstructed feature map containing target information of two images according to step 5; 6-2 Re-input the reconstructed feature map into the intermediate layer extracted in step 2 of the Resnet model, train the entire Resnet network, and obtain the prediction probabilities of different categories in the final network output layer 6-3 Select the probability P of the two categories to which the original image corresponding to the reconstructed feature map belongs from the predicted probability A and P B As a result of the adaptive confidence, the adaptive confidence is: [l A ,l B ]=softmax([P A ,P B ]) 6-4 The original image category corresponding to the reconstructed feature map and the adaptive confidence together constitute the training label of the reconstructed image. The objective function of the training can be expressed as: Among them, λ A ,λ B are the target categories y A and B The confidence level is calculated by using the cross entropy loss function L CE , is the predicted output.

Images