Image classification method based on deep learning series optimizer

Abstract

The invention discloses an image classification method based on a deep learning series optimizer, and the method comprises the steps: constructing the deep learning series optimizer based on an Adam optimizer and an SGD optimizer for the respective advantages and disadvantages of the Adam optimizer and the SGD optimizer, taking the Adam optimizer as a main optimizer, taking the SGD optimizer as an auxiliary optimizer, and carrying out the deep learning series optimizer based on the SGD optimizer. And optimizing the training process of the image classification network model by adopting a deep learning series optimizer to obtain a better network parameter value so as to realize more accurate image classification.

Description

Technical Field

[0001] The present invention belongs to the technical field of pattern recognition, machine learning and image classification, and specifically relates to an image classification method based on a deep learning cascade optimizer. Background Art

[0002] Deep learning technology has achieved remarkable results in the field of machine vision, such as image classification. The complexity of neural network models and the size of datasets are key factors in their powerful performance. Although more complex models and larger datasets can improve performance, this usually requires longer training time and higher computing resources. Therefore, there is an increasing demand in industry and academia for distributed deep learning training on large clusters, and data parallelism has become the mainstream method for distributed training.

[0003] In the study of distributed image classification, the Adam optimizer is widely used due to its efficiency. However, image classification training based on Adam faces a challenge: using small batches of data will lead to too long training time, while using large batches of data may lead to a significant decrease in model accuracy. This is mainly because: first, the increase in the amount of data means that the number of model parameter updates is reduced, resulting in a slower convergence speed, making it difficult for the optimizer to achieve the ideal convergence effect within a limited number of updates; second, large batches of data may cause the optimizer to fall into a sharp local minimum, thereby reducing the generalization ability of the model. Therefore, it is of great research value to design a deep learning optimizer that can maintain image classification accuracy while being efficient and stable. Summary of the invention

[0004] In view of this, the present invention provides an image classification method based on deep learning cascade optimizer, which is composed of an Adam optimizer and an SGD optimizer in series, with the Adam optimizer as the main optimizer and the SGD optimizer as the sub-optimizer, to achieve accurate image classification.

[0005] The present invention provides an image classification method based on a deep learning cascade optimizer, which specifically includes the following steps:

[0006] Step 1: collect image data to establish an image data set, and create labels for the image data in the image data set to form a training sample set;

[0007] Step 2: Select a neural network model to establish an image classification network model. The input of the image classification network model is the image to be classified, and the output is the type of the image to be classified.

[0008] Step 3, construct a deep learning tandem optimizer formed by the SGD optimizer and the Adam optimizer in series, the input is the gradient of the network parameter value of the image classification network model after the current k-1th iteration, and the output is the network parameter value of the image classification network model required to be set for the kth iteration; in the deep learning tandem optimizer, the network parameter value obtained by processing the input by the Adam optimizer is used as the first parameter value, and the network parameter value obtained by processing the sum of the input and the first parameter value by the SGD optimizer is used as the second parameter value, and the second parameter value is used as the output;

[0009] Step 4: Input the sample images in the training sample set into the image classification network model to obtain the labels of the sample images, and then use the deep learning cascade optimizer to optimize the network parameter values ​​of the image classification network model to complete one iteration. When the number of iterations k reaches the set threshold, the training of the image classification network model is completed;

[0010] Step 5: Input the image to be classified into the trained image classification network model. The image classification network model outputs the category of the image to be classified to complete the image classification.

[0011] Furthermore, the neural network model is CapsNet, Unet, WideResNet-28, ResNet-34, ResNet-18 or DenseNet-12.

[0012] Furthermore, in step 3, the network parameter value obtained by processing the input by the Adam optimizer is used as the calculation method of the first parameter value as follows:

[0013]

[0014] Among them, g k-1 is the gradient of the network parameter value of the image classification network model after the k-1th iteration; m k 、m k-1 are the first-order momentum of the Adam optimizer at the kth and k-1th iterations, respectively; v k 、v k-1 are the second-order momentum of the Adam optimizer at the kth and k-1th iterations, respectively; β 2 , β 3 are the accumulation coefficients of the first-order momentum and second-order momentum of the Adam optimizer, β 2 and β 3 The value range of is (0,1); is the Hadamard inner product; ε>0 is the smoothness coefficient; η Adam is the initial learning rate of the Adam optimizer; is the generalized learning rate of the kth iteration of the Adam optimizer; The first parameter value of the image classification network model output by the Adam optimizer at the k-1th iteration; is the first parameter value to be adopted in the kth iteration.

[0015] Furthermore, in step 3, the network parameter value obtained by processing the sum of the input and the first parameter value by the SGD optimizer as the second parameter value is calculated as follows:

[0016]

[0017] in, is the second parameter value of the image classification network model obtained by the SGD optimizer after the k-1th iteration; η SGD is the initial learning rate of the SGD optimizer; is the second parameter value to be adopted in the kth iteration.

[0018] Beneficial effects:

[0019] In view of the advantages and disadvantages of the Adam optimizer and the SGD optimizer, a deep learning tandem optimizer based on the Adam optimizer and the SGD optimizer is constructed. The Adam optimizer is used as the main optimizer and the SGD optimizer is used as the sub-optimizer. The deep learning tandem optimizer is used to optimize the training process of the image classification network model to obtain better network parameter values, so as to achieve more accurate image classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram comparing the accuracy of image classification results obtained by using an image classification method based on a deep learning cascade optimizer provided by the present invention and using existing image classification results. DETAILED DESCRIPTION

[0021] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0022] By analyzing the advantages and disadvantages of the Adam optimizer and the SGD optimizer, the SGD optimizer is used to correct the network parameter values ​​of the image classification network model obtained by the Adam optimizer, which can further improve the generalization performance of the image classification network model. Therefore, building a deep learning tandem optimizer based on the Adam optimizer and the SGD optimizer will produce the best training effect. To this end, the present invention considers designing a deep learning tandem optimizer to realize the combination of the Adam optimizer and the SGD optimizer, and through theoretical analysis and a large number of experimental verifications, it is finally determined that the Adam optimizer is used as the main optimizer and the SGD optimizer is used as the control basis of the secondary optimizer.

[0023] The present invention provides an image classification method based on a deep learning cascade optimizer, the basic idea of ​​which is: establishing an image data set, preprocessing the image data set to mark different types of images to form a training sample set; selecting a neural network model for image classification, and constructing a new deep learning cascade optimizer CAS suitable for the neural network model; using the training sample set and the deep learning cascade optimizer CAS to complete the training of the image classification neural network model; finally, using the trained image classification neural network model to complete the classification of the image to be classified.

[0024] The present invention provides an image classification method based on a deep learning cascade optimizer, which specifically includes the following steps:

[0025] Step 1: Collect image data to establish an image data set, and create labels for the image data in the image data set to form a training sample set.

[0026] Step 2: Select a neural network model to build an image classification network model. The input of the image classification network model is the image to be classified, and the output is the type of the image to be classified. For example, the following existing neural network models can be selected: CapsNet, Unet, WideResNet-28, ResNet-34, ResNet-18, and DenseNet-12, etc.

[0027] Step 3: Construct a deep learning cascade optimizer CAS. The input of the deep learning cascade optimizer CAS is the gradient of the network parameter value of the image classification network model after the current k-1th iteration, and the output is the network parameter value of the image classification network model that needs to be set for the kth iteration.

[0028] The deep learning cascade optimizer CAS constructed by the present invention is formed by the SGD optimizer and the Adam optimizer in series, that is, the network parameter value obtained by processing the gradient of the current network parameter value by the Adam optimizer is used as the first parameter value, and then the network parameter value obtained by processing the gradient of the current network parameter value and the sum of the first parameter value by the SGD optimizer is used as the second parameter value, and finally the second parameter value is used as the network parameter value adopted by the next iteration of the image classification network model.

[0029] Specifically, the Adam optimizer is used to process the gradient of the network parameter value after the current k-1th iteration to obtain the first parameter value required for the kth iteration of the image classification network model. The calculation process is shown in formula (1):

[0030]

[0031] Among them, g k-1 is the gradient of the network parameter value of the image classification network model after the k-1th iteration; m k、m k-1 are the first-order momentum of the Adam optimizer at the kth and k-1th iterations, respectively; v k 、v k-1 are the second-order momentum of the Adam optimizer at the kth and k-1th iterations, respectively; β 2 , β 3 are the accumulation coefficients of the first-order momentum and second-order momentum of the Adam optimizer, β 2 and β 3 The value range of is (0,1); is the Hadamard inner product; ε>0 is the smoothness coefficient; η Adam is the initial learning rate of the Adam optimizer; is the generalized learning rate of the kth iteration of the Adam optimizer; The first parameter value of the image classification network model output by the Adam optimizer at the k-1th iteration.

[0032] Use the SGD optimizer to obtain the second parameter value required for the kth iteration of the image classification network model The calculation process is shown in formula (2):

[0033]

[0034] in, is the second parameter value of the image classification network model obtained by the SGD optimizer after the k-1th iteration; η SGD is the initial learning rate of the SGD optimizer; It is the first parameter value of the image classification network model required for the kth iteration of the image classification network model obtained by the Adam optimizer.

[0035] Step 4: Input the sample images in the training sample set into the image classification network model to obtain the labels of the sample images, and then use the deep learning cascade optimizer CAS to optimize the network parameter values ​​of the image classification network model to complete an iteration. When the number of iterations k reaches the set threshold, the training of the image classification network model is completed.

[0036] Step 5: Input the image to be classified into the trained image classification network model. The image classification network model outputs the category of the image to be classified to complete the image classification.

[0037] Example:

[0038] This embodiment adopts an image classification method based on a deep learning cascade optimizer provided by the present invention, selects a neural network model WideResNet-28 as an image classification network model to complete the image classification process, and specifically includes the following steps:

[0039] S1. Establish training sample set The data comes from the public dataset STL10, which contains N = 113,000 images, where the i-th image x i The image size is 96×96, which contains the color image of the classified object; i Indicates the category label corresponding to the i-th image, y i is a 10-dimensional vector with a value of 0 or 1, y i The position k with a value of 1 indicates that the image belongs to the kth category, and the value of 0 indicates that it does not belong to the corresponding category.

[0040] S2. Input the sample images in the training sample set into the image classification network model to obtain the labels of the sample images, and then use the deep learning cascade optimizer CAS to optimize the network parameter values ​​of the image classification network model to complete the training of the image classification network model.

[0041] In this embodiment, a random depth mechanism is set, a cross entropy loss function is selected, the value of the number of batches is set to 256, the threshold of the number of training iterations is 150, and a fixed strategy learning rate decay method is adopted, that is, when k = 50 and k = 100, the learning rate of the deep learning optimizer is reduced by 10 times. Among them, the number of batches refers to the number of pictures input into the network model at each iteration, for example, the number of input pictures is 256.

[0042] In order to verify the efficiency of the method of the present invention, another 9 publicly available deep learning optimizers, including SGD, AdaBelief, AdaBound, Adam, AdamW, SGD-Momentum (SGDM) and RAdam, were selected for comparative testing.

[0043] The specific parameter settings are as follows:

[0044] The initial learning rate of AdaBelief, AdaBound, Adam, AdamW and Radam deep learning optimizers is set to 0.001 (λ = 10 -3 ), the accumulation coefficients of the first-order momentum and the second-order momentum are set to β 2 = 0.9 and β 3 = 0.999, the initial learning rate of SGD and SGDM deep learning optimizers is set to 0.1 (λ = 10 -1 ), the momentum coefficient of the SGDM optimizer is set to β 1 =0.9; In the CAS deep learning optimizer, the main optimizer Adam and the sub-optimizer SGD are set to the same initial learning rate set to 0.001 (λ = 10 -3 ).

[0045] The image classification results of the WideResNet-28 neural network model trained by the optimizer established by the present invention are compared with the image classification results of the WideResNet-28 neural network model trained by the existing optimizer. Figure 1 As shown in the figure, the horizontal axis represents the number of iterations and the vertical axis represents the test accuracy. Figure 1 It can be seen that after the CAS optimizer provided by the present invention has undergone two learning rate decays, the generalization performance of the CAS optimizer is better than that of other optimizers.

[0046] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An image classification method based on deep learning cascade optimizer, characterized in that: The specific steps include: Step 1: collect image data to establish an image data set, and create labels for the image data in the image data set to form a training sample set; Step 2: Select a neural network model to establish an image classification network model. The input of the image classification network model is the image to be classified, and the output is the type of the image to be classified. Step 3, construct a deep learning tandem optimizer formed by the SGD optimizer and the Adam optimizer in series, the input is the gradient of the network parameter value of the image classification network model after the current k-1th iteration, and the output is the network parameter value of the image classification network model required to be set for the kth iteration; in the deep learning tandem optimizer, the network parameter value obtained by processing the input by the Adam optimizer is used as the first parameter value, and the network parameter value obtained by processing the sum of the input and the first parameter value by the SGD optimizer is used as the second parameter value, and the second parameter value is used as the output; Step 4: Input the sample images in the training sample set into the image classification network model to obtain the labels of the sample images, and then use the deep learning cascade optimizer to optimize the network parameter values ​​of the image classification network model to complete one iteration. When the number of iterations k reaches the set threshold, the training of the image classification network model is completed; Step 5: Input the image to be classified into the trained image classification network model. The image classification network model outputs the category of the image to be classified to complete the image classification.

2. The image classification method according to claim 1, characterized in that: The neural network model is CapsNet, Unet, WideResNet-28, ResNet-34, ResNet-18 or DenseNet-12.

3. The image classification method according to claim 1, characterized in that: The calculation method of using the network parameter value obtained by processing the input by the Adam optimizer as the first parameter value in step 3 is: Among them, g k-1 is the gradient of the network parameter value of the image classification network model after the k-1th iteration; m k 、m k-1 are the first-order momentum of the Adam optimizer at the kth and k-1th iterations, respectively; v k 、v k-1 are the second-order momentum of the Adam optimizer at the kth and k-1th iterations, respectively; β2 and β3 are the cumulative coefficients of the first-order momentum and second-order momentum of the Adam optimizer, respectively, and the value range of β2 and β3 is (0,1); is the Hadamard inner product; ε>0 is the smoothness coefficient; η Adam is the initial learning rate of the Adam optimizer; is the generalized learning rate of the kth iteration of the Adam optimizer; The first parameter value of the image classification network model output by the Adam optimizer at the k-1th iteration; is the first parameter value to be adopted in the kth iteration.

4. The image classification method according to claim 3, characterized in that: The calculation method of using the network parameter value obtained by processing the sum of the input and the first parameter value by the SGD optimizer as the second parameter value in step 3 is: in, is the second parameter value of the image classification network model obtained by the SGD optimizer after the k-1th iteration; η SGD is the initial learning rate of the SGD optimizer; is the second parameter value to be adopted in the kth iteration.

Images