A Method for Training Residual Neural Networks for Image Classification
By introducing a classifier inside the ResNet network and training it jointly, the problem of improving the accuracy of the ResNet network without increasing its depth is solved, and an accuracy improvement is achieved on the CIFAR100 dataset.
Patent Information
- Application Number
- CN202310037919.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-01-10
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2043-01-10
AI Technical Summary
Existing ResNet networks offer limited improvement in image classification accuracy without increasing network depth, making them particularly difficult to apply on resource-constrained mobile or embedded devices.
Several classifiers are introduced into the ResNet network and trained jointly using an image dataset to supervise these classifiers. A total loss function is constructed for the joint training of multiple classifiers, and the weights of the ResNet network are updated.
Without increasing network depth, it significantly improves the overall accuracy of the ResNet network, especially on the CIFAR100 dataset, where the accuracy is improved by 1.4% (ResNet18) and 2.4% (ResNet50).
Smart Images

Figure CN116206151B_ABST
Abstract
Description
Technical Field
[0001] This invention proposes a method for training an image classification residual neural network, which belongs to the field of image classification and neural network technology. Background Technology
[0002] With the development and progress of artificial intelligence, deep neural network algorithms have been widely applied and have achieved remarkable results in various fields. They have been widely used in information processing in fields such as artificial intelligence, automatic control, robotics, and statistics. Taking image classification as an example, classic image classification networks such as VGG network, ResNet network, and ResNext network are trained on specific datasets for specific image classification tasks. Under ordinary training methods, the classification accuracy depends on the structure of the image classification network itself.
[0003] To achieve better classification performance, increasing network depth can significantly improve network performance. While deeper networks outperform shallower ones, excessive depth can lead to neural network degradation and vanishing gradients. In 2015, Kaiming He et al. proposed the ResNet network, which, by stacking residual structures, increased network depth while mitigating the vanishing gradient problem, further improving the performance of neural networks in image classification tasks. Currently, even larger residual neural networks (ResNet1000) have been proposed, but the performance improvement is extremely limited, usually at the cost of massive computation and storage. Implementing large-scale neural networks is exceptionally difficult for mobile or embedded devices with high real-time requirements and limited resources. Therefore, improving the accuracy of ResNet networks without increasing network depth is crucial. Summary of the Invention
[0004] This invention proposes a method for training a residual neural network for image classification, which aims to improve the overall accuracy of the ResNet network without increasing the network depth.
[0005] The technical solution of this invention: a method for training an image classification residual neural network, the method comprising:
[0006] 1. Several classifiers are introduced inside the ResNet network; the ResNet network includes a first convolutional layer, a max pooling layer, a first module, a second module, a third module, a fourth module, a first adaptive average pooling layer, a fourth fully connected layer, and a fourth probability transformation function;
[0007] 2. Jointly train several classifiers using an image dataset;
[0008] 3. Supervise the ResNet network and several classifiers using real labels and supervise the ResNet network using the prediction results of the ResNet network to obtain the total loss of joint training;
[0009] 4. Update the ResNet network weights under the influence of the total loss.
[0010] Furthermore, the introduction of several classifiers within the ResNet network specifically includes: adding a first shallow module, a first fully connected layer, and a first probability transformation function sequentially as a first classifier based on the first module; adding a second shallow module, a second fully connected layer, and a second probability transformation function sequentially as a second classifier based on the second module; and adding a third shallow module, a third fully connected layer, and a third probability transformation function sequentially as a third classifier based on the third module.
[0011] Furthermore, the joint training of several classifiers using the image dataset specifically includes:
[0012] 2-1. Divide the training set of the image dataset into two parts: Part A and Part B.
[0013] 2-2. The images in the training set of Part B first pass through the first convolutional layer, the max pooling layer, and the first module to obtain the output feature map of the first module;
[0014] 2-3. Use the output feature map of the first module as the input feature map of the second module and the input feature map of the first shallow module;
[0015] 2-4. The input feature map of the first shallow module passes through the first shallow module, the first fully connected layer, and the first probability transformation function in sequence to obtain the output result of the first classifier. The output result of the first classifier is the first classification result.
[0016] 2-5. The input feature map of the second module is processed by the second module to obtain the output feature map of the second module;
[0017] 2-6. Use the output feature map of the second module as the input feature map of the third module and the second shallow module;
[0018] 2-7. The input feature map of the second shallow module passes through the second shallow module, the second fully connected layer, and the second probability transformation function in sequence to obtain the output result of the second classifier. The output result of the second classifier is the second classification result.
[0019] 2-8. The input feature map of the third module is processed by the third module to obtain the output feature map of the third module;
[0020] 2-9. Use the output feature map of the third module as the input feature map of the fourth module and the input feature map of the third shallow module;
[0021] 2-10. The input feature map of the third shallow module passes through the third shallow module, the third fully connected layer, and the third probability transformation function in sequence to obtain the output result of the third classifier. The output result of the third classifier is the third classification result.
[0022] 2-11. The input feature map of the fourth module is processed by the fourth module to obtain the output feature map of the fourth module;
[0023] 2-12. The output feature map of the fourth module then passes through the first adaptive average pooling layer, the fourth fully connected layer, and the fourth probability transformation function in sequence to obtain the output result of the ResNet network. The output result of the ResNet network is the fourth classification result.
[0024] Furthermore, the method of jointly training several classifiers using an image dataset specifically includes:
[0025] 2-13. The images in Part A of the training set are sequentially passed through the first convolutional layer, the max pooling layer, the first module, the second module, the third module, the fourth module, the first adaptive average pooling layer, the fourth fully connected layer, and the fourth probability transformation function to obtain the prediction results of the ResNet network; the prediction results of the ResNet network are the classification results predicted by the ResNet network.
[0026] Furthermore, the image dataset includes a training set and a test set.
[0027] Furthermore, each of the first, second, third, and fourth modules includes several convolutional layers (conv); the input feature map of each module is processed by the convolutional layer inside the corresponding module to obtain the output feature map of the corresponding module after entering the corresponding module.
[0028] Furthermore, each of the first, second, and third shallow modules includes several convolutional layers and an adaptive average pooling layer; each of the first, second, and third shallow modules processes the image through its internal convolutional layers and adaptive average pooling layer.
[0029] Furthermore, the first probability transformation function, the second probability transformation function, and the third probability transformation function are all Softmax functions; the fourth probability transformation function introduces an additional parameter temp into the Softmax function, as shown in equation (1):
[0030]
[0031] In equation (1), z j This refers to the output result of the j-th category. If the total number of categories is M, then the value of j ranges from [1, M]. n For the nth class output of the image after passing through the ResNet network, pro n The output probability of the nth class is given to the ResNet network. temp is an additional parameter that must be greater than 0. The class refers to the M classes of the image dataset.
[0032] Furthermore, the step of supervising the ResNet network and several classifiers using real labels and supervising the ResNet network using the prediction results of the ResNet network to obtain the total loss for joint training specifically includes:
[0033] 3-1. Compare the first classification result with the true label to obtain the first error loss value, compare the second classification result with the true label to obtain the second error loss value, compare the third classification result with the true label to obtain the third error loss value, and compare the fourth classification result with the true label to obtain the fourth error loss value.
[0034] 3-2. Compare the fourth classification result with the prediction result of the ResNet network to obtain the fifth error loss value;
[0035] 3-3. Obtain the total loss of joint training using formula (2); the specific formula (2) is as follows:
[0036]
[0037] In formula (2), Loss is the total loss, loss1 is the first error loss value, loss2 is the second error loss value, loss3 is the third error loss value, loss4 is the fourth error loss value, and lossN is the fifth error loss value; q tN TN represents the output of the fourth probability transition function when training the ResNet network with images in the B training set, where temp = 2 in the fourth probability transition function; N in formula (2) represents the ResNet network corresponding to different series; TN represents the output of the fourth probability transition function when training the ResNet network with images in the A training set, where temp = 2 in the fourth probability transition function; Cr represents the cross-entropy loss function; KL represents the KL divergence; q i When i = 1, 2, 3; when i = 1, q i q represents the output of the first probability transition function in the first classifier when training the ResNet network with images in the B training set, where q = 2. iq represents the output of the second probability transition function in the second classifier when training the ResNet network with images in the B part of the training set, where q is the output of the second classifier when i = 3. i represents the output of the third probability transformation function in the third classifier when training the ResNet network with images in the B part training set; q represents the output of the fourth probability transformation function when temp=1 when training the ResNet network with images in the B part training set; y represents the true label of the image in the B part training set.
[0038] Furthermore, updating the ResNet network weights under the total loss specifically includes:
[0039] 4-1. Train the ResNet network in several batches using an image dataset; each batch training consists of M rounds of training. The specific number of rounds in each batch training is M = the number of images in the training set / the number of images processed in each round of training. Each round will train Batchsize images, where Batchsize represents the number of images processed in each round of training.
[0040] 4-2. During each training round in each batch, the ResNet network weights are updated once using the total loss. After each batch of training is completed, the test set in the image dataset is used for testing to obtain the corresponding test accuracy for each batch. This process continues until several batches of training are completed, and the highest test accuracy among all batches is taken as the final accuracy of the ResNet network.
[0041] The beneficial effects of this invention are:
[0042] 1) This invention introduces an additional module structure within the ResNet network framework to construct multiple classifiers for joint training, which can help improve the overall accuracy of the ResNet network compared to the original training method;
[0043] 2) Simulation results show that the overall accuracy is improved on the CIFAR100 dataset; the accuracy of the ResNet18 network increases from 77.6% to 79.0%, an improvement of 1.4%; and the accuracy of the ResNet50 network increases from 77.1% to 79.5%, an improvement of 2.4%.
[0044] 3) This invention is universal and can be widely applied to tasks such as image classification, image denoising, image segmentation and image super-resolution based on ResNet networks. Attached Figure Description
[0045] Appendix Figure 1 It is a framework based on the ResNet network.
[0046] Appendix Figure 2This is a framework based on the ResNet18 network.
[0047] Appendix Figure 3 This is a shallow module based on the ResNet18 network.
[0048] Appendix Figure 4 This is a framework based on the ResNet50 network.
[0049] Appendix Figure 5 This is a shallow module based on the ResNet50 network. Detailed Implementation
[0050] A method for training an image classification residual neural network, comprising the following steps:
[0051] 1. Several classifiers are introduced inside the ResNet network; the ResNet network includes a first convolutional layer (conv), a max pooling layer (maxpool), a first module (ResNBlock1), a second module (ResNBlock2), a third module (ResNBlock3), a fourth module (ResNBlock4), a first adaptive average pooling layer (AdaptiveAvgPool), a fourth fully connected layer (FC4-100), and a fourth probability transformation function (Softmax4 function);
[0052] 2. Jointly train several classifiers using an image dataset;
[0053] 3. Supervise the ResNet network and several classifiers using real labels and supervise the ResNet network using the prediction results of the ResNet network to obtain the total loss of joint training;
[0054] 4. Update the ResNet network weights under the influence of the total loss.
[0055] The ResNet network is preferably a ResNet18 network or a ResNet50 network; when the ResNet network is ResNet18, the first module is Res18Block1, the second module is Res18Block2, the third module is Res18Block3, and the fourth module is Res18Block4; when the ResNet network is ResNet50, the first module is Res50Block1, the second module is Res50Block2, the third module is Res50Block3, and the fourth module is Res50Block4.
[0056] The method involves introducing several classifiers within the ResNet network, preferably three classifiers.
[0057] The introduction of several classifiers within the ResNet network specifically includes: adding a first shallow module, a first fully connected layer (FC1-100), and a first probability transformation function (Softmax1 function) sequentially to the first module as the first classifier; adding a second shallow module, a second fully connected layer (FC2-100), and a second probability transformation function (Softmax2 function) sequentially to the second module as the second classifier; and adding a third shallow module, a third fully connected layer (FC3-100), and a third probability transformation function (Softmax3 function) sequentially to the third module as the third classifier.
[0058] The method of jointly training several classifiers using an image dataset specifically includes the following steps:
[0059] 2-1. Divide the training set of the image dataset into two parts: Part A and Part B.
[0060] 2-2. The images in the training set of Part B first pass through the first convolutional layer, the max pooling layer, and the first module to obtain the output feature map of the first module;
[0061] 2-3. Use the output feature map of the first module as the input feature map of the second module and the input feature map of the first shallow module;
[0062] 2-4. The input feature map of the first shallow module passes through the first shallow module, the first fully connected layer, and the first probability transformation function in sequence to obtain the output result of the first classifier. The output result of the first classifier is the first classification result.
[0063] 2-5. The input feature map of the second module is processed by the second module to obtain the output feature map of the second module;
[0064] 2-6. Use the output feature map of the second module as the input feature map of the third module and the second shallow module;
[0065] 2-7. The input feature map of the second shallow module passes through the second shallow module, the second fully connected layer, and the second probability transformation function in sequence to obtain the output result of the second classifier. The output result of the second classifier is the second classification result.
[0066] 2-8. The input feature map of the third module is processed by the third module to obtain the output feature map of the third module;
[0067] 2-9. Use the output feature map of the third module as the input feature map of the fourth module and the input feature map of the third shallow module;
[0068] 2-10. The input feature map of the third shallow module passes through the third shallow module, the third fully connected layer, and the third probability transformation function in sequence to obtain the output result of the third classifier. The output result of the third classifier is the third classification result.
[0069] 2-11. The input feature map of the fourth module is processed by the fourth module to obtain the output feature map of the fourth module;
[0070] 2-12. The output feature map of the fourth module then passes through the first adaptive average pooling layer, the fourth fully connected layer, and the fourth probability transformation function in sequence to obtain the output result of the ResNet network. The output result of the ResNet network is the fourth classification result.
[0071] 2-13. The images in Part A of the training set are sequentially passed through the first convolutional layer, the max pooling layer, the first module, the second module, the third module, the fourth module, the first adaptive average pooling layer, the fourth fully connected layer, and the fourth probability transformation function to obtain the prediction results of the ResNet network; the prediction results of the ResNet network are the classification results predicted by the ResNet network.
[0072] The image dataset includes a training set and a test set; the image dataset is preferably the CIFAR100 dataset.
[0073] Each of the first, second, third, and fourth modules includes several convolutional layers (conv); the input feature map of each module is processed by the convolutional layer (conv) inside the corresponding module to obtain the output feature map of the corresponding module.
[0074] The working process of each convolutional layer in this invention includes: taking an H*W*N feature map (height, H; width, W; depth, N, where N is equal to the number of channels of the output feature map of the previous convolutional layer) and passing it through a convolutional kernel of size 3 and number m (m is equal to the number of channels of the output feature map) to form an H`*W`*M feature map.
[0075] Each of the first, second, and third shallow modules includes several convolutional layers (conv) and an adaptive average pooling layer (AdaptiveAvgPool). Each shallow module processes the image through its internal convolutional layers and adaptive average pooling layer. Each shallow module is implemented by stacking several convolutional layers (conv), and the specific stacking method depends on the specific ResNet network (ResNet18, ResNet50) and requires different stacking methods (see attached diagram). Figure 3 Appendix Figure 5 The first adaptive average pooling layer and the adaptive average pooling layer in each shallow module can both reduce the height and width of the feature map input to the corresponding adaptive average pooling layer to 1.
[0076] The first probability transformation function, the second probability transformation function, and the third probability transformation function are all preferably ordinary Softmax functions; the fourth probability transformation function introduces an additional parameter temp into the ordinary Softmax function, as shown in equation (1):
[0077]
[0078] In equation (1), z j This refers to the output result of the j-th category. If the total number of categories is M, then the value of j ranges from [1, M]. n For the nth class output of the image after passing through the ResNet network, pro n The output probability of the nth class is given by the ResNet network. temp is an additional parameter that must be greater than 0. The larger the value of temp, the more even the output probabilities of each class. When temp is set to 1, equation (1) degenerates into the ordinary Softmax function. The class refers to the M classes of the image dataset. For example, when the image dataset is the CIFAR100 dataset, the CIFAR100 dataset has 100 classes. For example, an image of a cat belongs to the cat class. The M classes here refer to these 100 classes of the CIFAR100 dataset. After an image is output by Softmax, the probability of each class will be generated for the corresponding image. For example, an image of a cat will generate the probability of 100 classes after Softmax output, but the probability of the cat class may be the largest. The sum of the probabilities of these 100 classes is 1.
[0079] The process of supervising the ResNet network and several classifiers using real labels and supervising the ResNet network using the prediction results of the ResNet network to obtain the total loss of joint training specifically includes the following steps:
[0080] 3-1. The first classification result is compared with the true label to obtain the first error loss value (loss1), the second classification result is compared with the true label to obtain the second error loss value (loss2), the third classification result is compared with the true label to obtain the third error loss value (loss3), and the fourth classification result is compared with the true label to obtain the fourth error loss value (loss4). The true label is the correct category inherent in the image itself, that is, the category corresponding to each image in the image dataset. For example, for an image of a cat, the correct category is "cat", and the true label is the correct "cat" category attached to the image of a cat.
[0081] 3-2. Compare the fourth classification result with the prediction result of the ResNet network to obtain the fifth error loss value (lossN);
[0082] 3-3. Obtain the total loss of joint training using formula (2); the specific formula (2) is as follows:
[0083]
[0084] In formula (2), Loss is the total loss, loss1 is the first error loss value, loss2 is the second error loss value, loss3 is the third error loss value, loss4 is the fourth error loss value, and lossN is the fifth error loss value; q tN TN represents the output of the fourth probability transformation function (Softmax4 function) when training the ResNet network with images in the B training set, which is the fourth classification result. Here, it is preferred that temp = 2 in the fourth probability transformation function; N in formula (2) represents the ResNet network corresponding to different series. When targeting the ResNet18 network, N is 18, and when targeting the ResNet50 network, N is 50; TN represents the output of the fourth probability transformation function (Softmax4 function) when training the ResNet network with images in the A training set, which is the prediction result of the ResNet network. Here, it is preferred that temp = 2 in the fourth probability transformation function; Cr represents the cross-entropy loss function; KL represents the KL divergence; q i (i=1,2,3), q when i=1 i This represents the output of the first probability transformation function (Softmax1 function) in the first classifier when training the ResNet network with images in the B training set, which is the first classification result. When i=2, q i This represents the output of the second probability transition function (Softmax2 function) in the second classifier when training the ResNet network on images in the B training set, which is the second classification result. When i=3, q iq represents the output of the third probability transition function (Softmax3 function) in the third classifier when training the ResNet network with images in the B part training set, which is the third classification result; q represents the output of the fourth probability transition function (Softmax4 function) when temp is preferably 1 when training the ResNet network with images in the B part training set, which is the fourth classification result when temp = 1; y represents the value of the true label of the image in the B part training set.
[0085] The process of updating the ResNet network weights under the influence of the total loss includes the following steps:
[0086] 4-1. Train the ResNet network in several batches using an image dataset, preferably using an image dataset. Each batch training consists of M rounds of training. The specific number of rounds in each batch training is M = the number of images in the training set / the number of images processed in each round of training. Each round will train Batchsize images, where Batchsize represents the number of images processed in each round of training.
[0087] 4-2. During each training round in each batch, the ResNet network weights are updated once using the total loss. After each batch of training is completed, the test set in the image dataset is used for testing to obtain the corresponding test accuracy for each batch. This process continues until several batches of training are completed, and the highest test accuracy among all batches is taken as the final accuracy of the ResNet network. The preferred number of batches is 200 batches.
[0088] ResNet is a type of neural network, which contains nodes and parameters to simulate human neurons. A neural network involves two processes: forward propagation of the signal and backward propagation of the error. Error output is calculated from input to output, while weight adjustment is done from output to input. Forward propagation yields the output value, which is then compared to the expected output value (i.e., the predicted result is compared to the true label) to calculate the cross-entropy loss, thus obtaining the error value. The partial derivative of each node is calculated to obtain the error gradient for each node, resulting in the error loss value. Backward propagation of the error is then performed by applying this error loss value to the error gradient of each node. In practical applications, it's often impossible to load all the data into memory at once (due to insufficient computing power), so training is done in rounds. In each round, the loss gradient is calculated to point towards the local optimum. The SGD optimizer is used to converge the gradient towards the global optimum. The loss is the error loss value; aiming for both local and global optima means continuously reducing the loss.
[0089] This invention introduces additional modular structures to construct several classifiers for joint training, thereby improving the accuracy of the ResNet network and solving the problem that the accuracy of the ResNet network (residual neural network) is limited by the network structure under existing training methods. This invention can be widely applied to image classification, image denoising, image segmentation and image super-resolution based on ResNet networks.
[0090] The following describes in detail, with reference to specific embodiments, the implementation method of the present invention for training image classification residual neural networks based on two classification networks, ResNet18 and ResNet50. Without increasing the network depth, multiple classifiers are constructed and trained together by introducing additional module structures into the ResNet network, thereby improving the accuracy of the ResNet network.
[0091] Example 1
[0092] A joint training framework based on the ResNet18 network, such as... Figure 2 As shown; Figure 2 In this diagram, Conv3×3 represents a convolutional layer with a kernel size of 3, and 64, 128, 256, and 512 represent the number of channels in the output feature map of the corresponding convolutional layer; maxpool represents a max pooling layer; AdaptiveAvgPool represents an adaptive average pooling layer; FC1-100, FC2-100, FC3-100, and FC4-100 all represent fully connected layers with 100 nodes; Softmax1, Softmax2, Softmax3, and Softmax4 all represent the Softmax function, and label represents the true label of the image; the ResNet18 network consists of a convolutional layer, a max pooling layer, a Res18Block1 module, a Res18Block2 module, and a Res18Block4 module. The Res18Block1 module consists of a 8Block3 module, a Res18Block4 module, an AdaptiveAvgPool layer, an FC4-100 fully connected layer, and a Softmax4 function. Classifier 1 is constructed by sequentially adding a first shallow module, an FC1-100 fully connected layer, and a Softmax1 function to the Res18Block1 module. Classifier 2 is constructed by sequentially adding a second shallow module, an FC2-100 fully connected layer, and a Softmax2 function to the Res18Block2 module. Classifier 3 is constructed by sequentially adding a third shallow module, an FC3-100 fully connected layer, and a Softmax3 function to the Res18Block3 module. The structures of the first, second, and third shallow modules are as follows: Figure 3 As shown; Figure 3In the diagram, Conv3×3 represents a convolutional layer with a kernel size of 3, and 64, 128, 256, and 512 represent the number of channels in the output feature map of the corresponding convolutional layer; AdaptiveAvgPool represents an adaptive average pooling layer; the first shallow module consists of 6 convolutional layers and one AdaptiveAvgPool adaptive average pooling layer; the second shallow module consists of 4 convolutional layers and one AdaptiveAvgPool adaptive average pooling layer; and the third shallow module consists of 2 convolutional layers and one AdaptiveAvgPool adaptive average pooling layer.
[0093] In this embodiment, the joint training process based on the ResNet18 network involves the following steps: The image first passes through a convolutional layer and a max-pooling layer, then through the Res18Block1 module to obtain its output feature map. This output feature map serves as the input feature map for the Res18Block2 module and the first shallow module. The input feature map of the first shallow module then passes through the first shallow module, an FC1-100 fully connected layer, and a Softmax1 function to obtain the output of classifier 1. Similarly, the input feature map of the Res18Block2 module passes through the Res18Block2 module to obtain its output feature map. This output feature map serves as the input feature map for the Res18Block3 module and the second shallow module. The input feature map of the second shallow module then passes through the second shallow module, an FC2-100 fully connected layer, and a Softmax2 function to obtain the output of classifier 2. The input feature map of the ck3 module is passed through the Res18Block3 module to obtain the output feature map of the Res18Block3 module; the output feature map of the Res18Block3 module is used as the input feature map of the Res18Block4 module and the third shallow module; the input feature map of the third shallow module is passed through the third shallow module, the FC3-100 fully connected layer, and the Softmax3 function to obtain the output result of classifier 3; the input feature map of the Res18Block4 module is passed through the Res18Block4 module to obtain the output feature map of the Res18Block4 module; the output feature map of the Res18Block4 module is then passed through the AdaptiveAvgPool adaptive average pooling layer, the FC4-100 fully connected layer, and the Softmax4 function to obtain the output result of the ResNet18 network; Softmaxi (i=1,2,3) is the ordinary Softmax function, and Softmax4 introduces an additional parameter temp in the ordinary Softmax function, as shown in Equation (1); in Equation (1), z jThis refers to the output result of the j-th category. If the total number of categories is M, then the value of j ranges from [1, M]. n The output of the nth class after passing through the fully connected layer FC4-100 of the ResNet18 network, pro n For the output probability of the nth class of the ResNet18 network, when temp is set to 1, equation (1) degenerates into the ordinary Softmax function;
[0094]
[0095] The ResNet18 network joint training described in this embodiment uses the CIFAR100 dataset for 200 epochs. The neural network is optimized using the SGD optimizer. The initial learning rate is 0.1. When training reaches 66, 133 and 190 epochs, the learning rate is divided by 10, the weight decay is weight_decay = 5e-4, and the change amount of each update is momentum = 0.9. The batch size is 128. All experiments are conducted on GPU devices in the PyTorch 1.9.1 environment. Before training, the training set of the CIFAR100 dataset is processed as follows: (1) The images are randomly cropped. The size after cropping is 32, the padding value is set to 4, and the fill value is set to 0.9. (1) Set to 128; (2) Randomly flip the image horizontally; (3) Convert the image format to tensor format; Normalize each value of the image to [0,1]; (4) Standardize each value of the image, with the mean of the standardization being (0.4914, 0.4822, 0.4465) and the variance being (0.2023, 0.1994, 0.2010); The test set of the CIFAR100 dataset is processed as follows: (1) Convert the image format to tensor format; Normalize each value of the image to [0,1]; (2) Standardize each value of the image, with the mean of the standardization being (0.4914, 0.4822, 0.4465) and the standard deviation being (0.2023, 0.1994, 0.2010).
[0096] The training set of the CIFAR100 dataset is divided into two parts, A and B. Only part B is selected to train the entire network framework, generating prediction results (1, 2, 3, 4), and updating the network framework weights. Part A is used to generate the prediction result T18 of ResNet18. Unlike the ordinary training method where only the ResNet18 network is supervised by the real label, in the joint training framework, the ResNet18 network and the three classifiers are all supervised by the real label, and the ResNet18 network is additionally supervised by the prediction result T18. Based on this, the total loss function Loss during the joint training of ResNet18 consists of five parts (loss1, loss2, loss3, loss4, loss18), as shown in equation (3). In equation (3), Cr represents the cross-entropy loss function, and KL represents the KL divergence. i (i = 1, 2, 3) represents the output of Softmaxi (i = 1, 2, 3) in classifier i (i = 1, 2, 3) when training the network framework with data from part B. q, q t18 represents the output of Softmax4 when temp=1 and temp=2 when training the network framework with data from part B, respectively; y represents the value of the true label in part B; T18 represents the output of Softmax4 when temp=2 after inputting data from part A into the network framework;
[0097]
[0098] Experimental testing showed that, on the CIFAR100 dataset, the accuracy of the ResNet18 network in this embodiment improved from 77.6% to 79.0%, an improvement of 1.4%.
[0099] Example 2
[0100] The joint training framework based on the ResNet50 network described in this embodiment is as follows: Figure 4As shown, Conv3×3 represents a convolutional layer with a kernel size of 3, and 64, 128, 256, 512, 1024, and 2048 represent the number of channels in the output feature map of the corresponding convolutional layer; maxpool represents a max pooling layer; AdaptiveAvgPool represents an adaptive average pooling layer; FC5-100, FC6-100, FC7-100, and FC8-100 all represent fully connected layers with 100 nodes; Softmax5, Softmax6, Softmax7, and Softmax8 all represent the Softmax function, and label represents the true label of the image; the ResNet50 network consists of one convolutional layer, one max pooling layer, a Res50Block1 module, and a Res50Block2 module. The Res50Block1 module consists of a Res50Block3 module, a Res50Block4 module, an AdaptiveAvgPool layer, an FC8-100 fully connected layer, and a Softmax8 function. Classifier 5 is constructed by adding a first shallow module, an FC5-100 fully connected layer, and a Softmax5 function sequentially to the Res50Block1 module. Classifier 6 is constructed by adding a second shallow module, an FC6-100 fully connected layer, and a Softmax6 function sequentially to the Res50Block2 module. Classifier 7 is constructed by adding a third shallow module, an FC7-100 fully connected layer, and a Softmax7 function sequentially to the Res50Block3 module. The structures of the first, second, and third shallow modules are as follows: Figure 5 As shown; Figure 5 In the diagram, Conv3×3 represents a convolutional layer with a kernel size of 3, and 512 and 2048 represent the number of channels in the output feature map of the corresponding convolutional layer; AdaptiveAvgPool represents an adaptive average pooling layer; the first shallow module consists of 3 convolutional layers and one AdaptiveAvgPool adaptive average pooling layer; the second shallow module consists of 3 convolutional layers and one AdaptiveAvgPool adaptive average pooling layer; and the third shallow module consists of 3 convolutional layers and one AdaptiveAvgPool adaptive average pooling layer.
[0101] In this embodiment, the joint training process based on the ResNet50 network involves the following steps: The image first passes through a convolutional layer and a max-pooling layer, then through the Res50Block1 module to obtain its output feature map. This output feature map serves as the input feature map for the Res50Block2 module and the first shallow module. The input feature map of the first shallow module then passes through the first shallow module, an FC5-100 fully connected layer, and a Softmax5 function to obtain the output of classifier 5. The input feature map of the Res50Block2 module then passes through the Res50Block2 module to obtain its output feature map. This output feature map serves as the input feature map for the Res50Block3 module and the second shallow module. The input feature map of the second shallow module then passes through the second shallow module, an FC6-100 fully connected layer, and a Softmax6 function to obtain the output of classifier 6. The input feature map of the ck3 module is passed through the Res50Block3 module to obtain the output feature map of the Res50Block3 module; the output feature map of the Res50Block3 module is used as the input feature map of the Res50Block4 module and the third shallow module; the input feature map of the third shallow module is passed through the third shallow module, the FC7-100 fully connected layer, and the Softmax7 function to obtain the output result of classifier 7; the input feature map of the Res50Block4 module is passed through the Res50Block4 module to obtain the output feature map of the Res50Block4 module; the output feature map of the Res50Block4 module is then passed through the AdaptiveAvgPool adaptive average pooling layer, the FC8-100 fully connected layer, and the Softmax8 function to obtain the output result of the ResNet50 network; Softmaxi (i=5,6,7) is the ordinary Softmax function, and Softmax8 introduces an additional parameter temp in the ordinary Softmax function, as shown in equation (4); in equation (4), z j This refers to the output result of the j-th category. If the total number of categories is M, then the value of j ranges from [1, M]. m After passing through the fully connected layer FC8-100 of the ResNet50 network, the first m The output of the category, pro m Given the output probability of the m-th class in the ResNet50 network, when temp is set to 1, equation (4) degenerates into the ordinary Softmax function:
[0102]
[0103] The ResNet50 network joint training described in this embodiment uses the CIFAR100 dataset for 200 epochs. The neural network is optimized using the SGD optimizer. The initial learning rate is 0.1. When training reaches 66, 133 and 190 epochs, the learning rate is divided by 10, the weight decay is weight_decay = 5e-4, and the change amount of each update is momentum = 0.9. The batch size is 128. All experiments are performed on GPU devices in the PyTorch 1.9.1 environment. Before training, the training set of the CIFAR100 dataset was processed as follows: (1) The images were randomly cropped. The size after cropping is 32, the padding value is set to 4, and the fill value is set to 0.9. (1) Set to 128; (2) Randomly flip the image horizontally; (3) Convert the image format to tensor format; Normalize each value of the image to [0,1]; (4) Standardize each value of the image, with the mean of the standardization being (0.4914, 0.4822, 0.4465) and the variance being (0.2023, 0.1994, 0.2010); The test set of the CIFAR100 dataset is processed as follows: (1) Convert the image format to tensor format; Normalize each value of the image to [0,1]; (2) Standardize each value of the image, with the mean of the standardization being (0.4914, 0.4822, 0.4465) and the standard deviation being (0.2023, 0.1994, 0.2010).
[0104] The training set of the CIFAR100 dataset is divided into two parts, A and B. Only part B is selected to train the entire network framework, generating prediction results (5, 6, 7, 8), and updating the network framework weights. Part A is used to generate the prediction result T50 of ResNet50. Unlike the ordinary training method where only the ResNet50 network is supervised by the real label, in the joint training framework, the ResNet50 network and the three classifiers are supervised by the real label, and the ResNet50 network is additionally supervised by the prediction result T50. Based on this, the total loss function Loss during the joint training of ResNet50 consists of five parts (loss5, loss6, loss7, loss8, loss50), as shown in Equation (5). In Equation (5), Cr represents the cross-entropy loss function, KL represents the KL divergence, and q i (i = 5, 6, 7) represents the output of Softmaxi (i = 5, 6, 7) in classifier i (i = 5, 6, 7) when training the network framework on data from part B. q, q t50represents the output of Softmax8 with temp=1 and temp=2 when training the network framework with data from part B, respectively; y represents the value of the true label in part B; T50 represents the output of Softmax8 with temp=2 after inputting data from part A into the network framework;
[0105]
[0106] Experimental testing showed that, on the CIFAR100 dataset, the accuracy of the ResNet50 network in this embodiment improved from 77.1% to 79.5%, an improvement of 2.4%.
[0107] In summary, this invention constructs a residual neural network training method by introducing additional module structures to build multiple classifiers for joint training. Experimental data show that this method can help improve network accuracy.
Claims
1. A method for training an image classification residual neural network, characterized by: include: 1) Several classifiers are introduced inside the ResNet network; the ResNet network includes a first convolutional layer, a max pooling layer, a first module, a second module, a third module, a fourth module, a first adaptive average pooling layer, a fourth fully connected layer, and a fourth probability transformation function; 2) Jointly train several classifiers using an image dataset; 3) Supervise the ResNet network and several classifiers using real labels and supervise the ResNet network using the prediction results of the ResNet network to obtain the total loss of joint training; 4) Update the ResNet network weights under the influence of the total loss; The method of jointly training several classifiers using an image dataset specifically includes: 2-1) Divide the training set of the image dataset into two parts: Part A and Part B. 2-2) The images in the training set of part B first pass through the first convolutional layer, the max pooling layer, and the first module to obtain the output feature map of the first module; 2-3) Use the output feature map of the first module as the input feature map of the second module and the input feature map of the first shallow module; 2-4) The input feature map of the first shallow module passes through the first shallow module, the first fully connected layer, and the first probability transformation function in sequence to obtain the output result of the first classifier, which is the first classification result; 2-5) The input feature map of the second module is processed by the second module to obtain the output feature map of the second module; 2-6) Use the output feature map of the second module as the input feature map of the third module and the second shallow module; 2-7) The input feature map of the second shallow module passes through the second shallow module, the second fully connected layer, and the second probability transformation function in sequence to obtain the output result of the second classifier. The output result of the second classifier is the second classification result. 2-8) The input feature map of the third module is processed by the third module to obtain the output feature map of the third module; 2-9) Use the output feature map of the third module as the input feature map of the fourth module and the input feature map of the third shallow module; 2-10) The input feature map of the third shallow module passes through the third shallow module, the third fully connected layer, and the third probability transformation function in sequence to obtain the output result of the third classifier. The output result of the third classifier is the third classification result. 2-11) The input feature map of the fourth module is processed by the fourth module to obtain the output feature map of the fourth module; 2-12) The output feature map of the fourth module then passes through the first adaptive average pooling layer, the fourth fully connected layer, and the fourth probability transformation function in sequence to obtain the output result of the ResNet network. The output result of the ResNet network is the fourth classification result. The process of supervising the ResNet network and several classifiers using real labels and supervising the ResNet network using the prediction results of the ResNet network to obtain the total loss of joint training specifically includes: 3-1) The first classification result is compared with the true label to obtain the first error loss value, the second classification result is compared with the true label to obtain the second error loss value, the third classification result is compared with the true label to obtain the third error loss value, and the fourth classification result is compared with the true label to obtain the fourth error loss value. 3-2) The fourth classification result is compared with the prediction result of the ResNet network to obtain the fifth error loss value; 3-3) The total loss of joint training is obtained using formula (2); the specific formula (2) is as follows: (2); In formula (2), Loss is the total loss, loss1 is the first error loss value, loss2 is the second error loss value, loss3 is the third error loss value, loss4 is the fourth error loss value, and lossN is the fifth error loss value. TN represents the output of the fourth probability transformation function when training the ResNet network with images in the B training set, where temp=2 in the fourth probability transformation function; N in formula (2) represents the ResNet network corresponding to different series; TN represents the output of the fourth probability transformation function when training the ResNet network with images in the A training set, where temp=2 in the fourth probability transformation function; KL represents the cross-entropy loss function; KL represents the KL divergence. When i=1,2,3; when i=1 This represents the output of the first probability transition function in the first classifier when training the ResNet network with images in the B training set, where i=2. This represents the output of the second probability transition function in the second classifier when training the ResNet network with images in the B training set, where i=3. This represents the output of the third probability transformation function in the third classifier when training the ResNet network using images from the B part of the training set. This represents the output of the fourth probability transition function when temp = 1 when training the ResNet network using images from the B part of the training set; This represents the true label of the images in part B of the training set.
2. The image classification residual neural network training implementation method according to claim 1, characterized in that: The introduction of several classifiers within the ResNet network specifically includes: adding a first shallow module, a first fully connected layer, and a first probability transformation function sequentially to the first module as the first classifier; adding a second shallow module, a second fully connected layer, and a second probability transformation function sequentially to the second module as the second classifier; and adding a third shallow module, a third fully connected layer, and a third probability transformation function sequentially to the third module as the third classifier.
3. The image classification residual neural network training implementation method according to claim 1, characterized in that: The method of jointly training several classifiers using an image dataset specifically includes: 2-13) The images in Part A of the training set are sequentially passed through the first convolutional layer, the max pooling layer, the first module, the second module, the third module, the fourth module, the first adaptive average pooling layer, the fourth fully connected layer, and the fourth probability transformation function to obtain the prediction results of the ResNet network; the prediction results of the ResNet network are the classification results predicted by the ResNet network.
4. The image classification residual neural network training implementation method according to claim 1, characterized in that: The image dataset includes a training set and a test set.
5. The image classification residual neural network training implementation method according to claim 1, characterized in that: Each of the first, second, third, and fourth modules includes several convolutional layers. The input feature map of each module is processed by the convolutional layer inside the corresponding module to obtain the output feature map of the corresponding module.
6. The image classification residual neural network training implementation method according to claim 2, characterized in that: Each of the first, second, and third shallow modules includes several convolutional layers and an adaptive average pooling layer; each of the first, second, and third shallow modules processes the image through its internal convolutional layers and adaptive average pooling layer.
7. The image classification residual neural network training implementation method according to claim 2, characterized in that: The first probability transformation function, the second probability transformation function, and the third probability transformation function are all Softmax functions; the fourth probability transformation function introduces an additional parameter temp into the Softmax function, as shown in equation (1): (1); In equation (1), This refers to the output result of the j-th category. If the total number of categories is M, then the value of j ranges from [1, M]. The image after passing through the ResNet network is the first... The output of the category, Output the first ResNet network The output probability of the category, temp is an additional parameter that must be greater than 0; the category refers to the M categories of the image dataset.
8. The method for training an image classification residual neural network according to claim 1, characterized in that: The process of updating the ResNet network weights under the influence of the total loss specifically includes: 4-1) Train the ResNet network in several batches using an image dataset; each batch training consists of M rounds of training. The specific number of rounds in each batch training is M = the number of images in the training set / the number of images processed in each round of training. Each round will train Batchsize images, where Batchsize represents the number of images processed in each round of training. 4-2) During each training round in each batch, the ResNet network weights are updated once using the total loss. After each batch of training is completed, the test set in the image dataset is used to test and obtain the corresponding test accuracy for each batch. This process continues until several batches of training are completed, and the highest test accuracy among all batches is taken as the final accuracy of the ResNet network.
Citation Information
Patent Citations
Self-distillation training method for image classification model training
CN115512156A
