An image classification method based on deep learning cascade optimizer

Abstract

The present invention discloses an image classification method based on a deep learning cascade optimizer. In view of the respective advantages and disadvantages of the Adam optimizer and the SGD optimizer, a deep learning cascade optimizer based on the Adam optimizer and the SGD optimizer is constructed. The Adam optimizer is used as the main optimizer and the SGD optimizer is used as the secondary optimizer. The deep learning cascade optimizer is used to optimize the training process of the image classification network model to obtain better network parameter values, thereby achieving more accurate image classification.

Description

Technical Field

[0001] The present invention belongs to the technical fields of pattern recognition, machine learning and image classification, and specifically relates to an image classification method based on a deep learning cascade optimizer. Background Art

[0002] Deep learning technology has achieved remarkable results in machine vision fields, such as image classification. The complexity of neural network models and the size of datasets are key factors in their robust performance. While more complex models and larger datasets can improve performance, they typically require longer training times and higher computing resources. Consequently, there is a growing demand in both industry and academia for distributed deep learning training on large clusters, with data parallelism becoming the mainstream approach for distributed training.

[0003] In distributed image classification research, the Adam optimizer is widely adopted for its efficiency. However, image classification training based on Adam faces a challenge: using small batches of data results in excessively long training times, while using large batches can significantly reduce model accuracy. This is primarily due to the following reasons: First, an increase in data volume means fewer model parameter updates, which slows convergence and makes it difficult for the optimizer to achieve ideal convergence within a limited number of updates; second, large batches of data can cause the optimizer to fall into sharp local minima, reducing the model's generalization ability. Therefore, designing a deep learning optimizer that maintains image classification accuracy while being efficient and stable is of great research value. Summary of the Invention

[0004] In view of this, the present invention provides an image classification method based on a deep learning cascade optimizer, which is composed of an Adam optimizer and an SGD optimizer in series, with the Adam optimizer as the main optimizer and the SGD optimizer as the sub-optimizer, to achieve accurate image classification.

[0005] The present invention provides an image classification method based on a deep learning cascade optimizer, which specifically includes the following steps:

[0006] Step 1: Collect image data to establish an image dataset, and create labels for the image data in the image dataset to form a training sample set;

[0007] Step 2: Select a neural network model to establish an image classification network model. The input of the image classification network model is the image to be classified, and the output is the type of the image to be classified.

[0008] Step 3: Construct a deep learning tandem optimizer formed by the SGD optimizer and the Adam optimizer in series, with the input being the gradient of the network parameter value of the image classification network model after the current k-1th iteration, and the output being the network parameter value of the image classification network model required to be set for the kth iteration; in the deep learning tandem optimizer, the network parameter value obtained by processing the input by the Adam optimizer is used as the first parameter value, and the network parameter value obtained by processing the sum of the input and the first parameter value by the SGD optimizer is used as the second parameter value, and the second parameter value is used as the output;

[0009] Step 4: Input the sample images in the training sample set into the image classification network model to obtain the labels of the sample images, and then use the deep learning cascade optimizer to optimize the network parameter values ​​of the image classification network model to complete one iteration. When the number of iterations k reaches the set threshold, the training of the image classification network model is completed;

[0010] Step 5: Input the image to be classified into the trained image classification network model. The image classification network model outputs the category of the image to be classified, completing the image classification.

[0011] Furthermore, the neural network model is CapsNet, Unet, WideResNet-28, ResNet-34, ResNet-18 or DenseNet-12.

[0012] Furthermore, in step 3, the network parameter value obtained by processing the input through the Adam optimizer is calculated as the first parameter value in the following manner:

[0013]

[0014] Among them, g k-1 is the gradient of the network parameter value of the image classification network model after the k-1th iteration; m k 、m k-1 are the first-order momentum of the Adam optimizer at the kth and k-1th iterations, respectively; v k 、v k-1 are the second-order momentum of the Adam optimizer at the kth and k-1th iterations, respectively; β2 and β3 are the cumulative coefficients of the first-order momentum and second-order momentum of the Adam optimizer, respectively. The value range of β2 and β3 is (0,1); is the Hadamard inner product; ε>0 is the smoothness coefficient; η Adam is the initial learning rate of the Adam optimizer; is the generalized learning rate of the kth iteration of the Adam optimizer; The first parameter value of the image classification network model output by the Adam optimizer at the k-1th iteration; is the first parameter value to be adopted in the kth iteration.

[0015] Furthermore, in step 3, the network parameter value obtained by processing the sum of the input and the first parameter value by the SGD optimizer as the second parameter value is calculated as follows:

[0016]

[0017] in, is the second parameter value of the image classification network model obtained by the SGD optimizer after the k-1th iteration; η SGD is the initial learning rate of the SGD optimizer; is the second parameter value to be adopted in the kth iteration.

[0018] Beneficial effects:

[0019] In view of the respective advantages and disadvantages of the Adam optimizer and the SGD optimizer, this paper constructs a deep learning cascade optimizer based on the Adam optimizer and the SGD optimizer. The Adam optimizer is used as the main optimizer and the SGD optimizer is used as the secondary optimizer. The deep learning cascade optimizer is used to optimize the training process of the image classification network model to obtain better network parameter values, thereby achieving more accurate image classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a schematic diagram comparing the accuracy of image classification results obtained by using an image classification method based on deep learning cascade optimizer provided by the present invention and the accuracy of existing image classification results. DETAILED DESCRIPTION

[0021] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0022] By analyzing the advantages and disadvantages of the Adam optimizer and the SGD optimizer, and using the SGD optimizer to correct the network parameter values ​​of the image classification network model obtained by the Adam optimizer, the generalization performance of the image classification network model can be further improved. Therefore, constructing a deep learning tandem optimizer based on the Adam optimizer and the SGD optimizer will produce the best training effect. To this end, the present invention considers designing a deep learning tandem optimizer to achieve the combination of the Adam optimizer and the SGD optimizer, and through theoretical analysis and a large number of experimental verifications, it is finally determined that the Adam optimizer is used as the main optimizer and the SGD optimizer is used as the control basis of the sub-optimizer.

[0023] The present invention provides an image classification method based on a deep learning cascade optimizer. The basic idea is: establish an image data set, preprocess the image data set to mark different types of images to form a training sample set; select a neural network model for image classification, and construct a new deep learning cascade optimizer CAS suitable for the neural network model; use the training sample set and the deep learning cascade optimizer CAS to complete the training of the image classification neural network model; finally, use the trained image classification neural network model to complete the classification of the image to be classified.

[0024] The present invention provides an image classification method based on a deep learning cascade optimizer, which specifically includes the following steps:

[0025] Step 1: Collect image data to establish an image dataset, and create labels for the image data in the image dataset to form a training sample set.

[0026] Step 2: Select a neural network model to build an image classification network model. The input of the image classification network model is the image to be classified, and the output is the type of image to be classified. For example, the following existing neural network models can be selected: CapsNet, Unet, WideResNet-28, ResNet-34, ResNet-18, and DenseNet-12.

[0027] Step 3: Construct a deep learning cascade optimizer CAS. The input of the deep learning cascade optimizer CAS is the gradient of the network parameter value of the image classification network model after the current k-1th iteration, and the output is the network parameter value of the image classification network model required to be set for the kth iteration.

[0028] The deep learning cascade optimizer CAS constructed by the present invention is formed by the SGD optimizer and the Adam optimizer in series, that is, the network parameter value obtained by processing the gradient of the current network parameter value by the Adam optimizer is used as the first parameter value, and then the network parameter value obtained by processing the gradient of the current network parameter value and the sum of the first parameter value by the SGD optimizer is used as the second parameter value, and finally the second parameter value is used as the network parameter value adopted by the next iteration of the image classification network model.

[0029] Specifically, the Adam optimizer is used to process the gradient of the network parameter value after the current k-1 iteration to obtain the first parameter value required for the k-th iteration of the image classification network model. The calculation process is shown in formula (1):

[0030]

[0031] Among them, g k-1 is the gradient of the network parameter value of the image classification network model after the k-1th iteration; m k、m k-1 are the first-order momentum of the Adam optimizer at the kth and k-1th iterations, respectively; v k 、v k-1 are the second-order momentum of the Adam optimizer at the kth and k-1th iterations, respectively; β2 and β3 are the cumulative coefficients of the first-order momentum and second-order momentum of the Adam optimizer, respectively. The value range of β2 and β3 is (0,1); is the Hadamard inner product; ε>0 is the smoothness coefficient; η Adam is the initial learning rate of the Adam optimizer; is the generalized learning rate of the kth iteration of the Adam optimizer; The first parameter value of the image classification network model output by the Adam optimizer at the k-1th iteration.

[0032] Use SGD optimizer to get the second parameter value required for the kth iteration of the image classification network model The calculation process is shown in formula (2):

[0033]

[0034] in, is the second parameter value of the image classification network model obtained by the SGD optimizer after the k-1th iteration; η SGD is the initial learning rate of the SGD optimizer; The first parameter value of the image classification network model required for the k-th iteration of the image classification network model obtained by the Adam optimizer.

[0035] Step 4: Input the sample images in the training sample set into the image classification network model to obtain the labels of the sample images, and then use the deep learning cascade optimizer CAS to optimize the network parameter values ​​of the image classification network model to complete one iteration. When the number of iterations k reaches the set threshold, the training of the image classification network model is completed.

[0036] Step 5: Input the image to be classified into the trained image classification network model. The image classification network model outputs the category of the image to be classified, completing the image classification.

[0037] Example:

[0038] This embodiment adopts an image classification method based on a deep learning cascade optimizer provided by the present invention, selects the neural network model WideResNet-28 as the image classification network model to complete the image classification process, and specifically includes the following steps:

[0039] S1. Establish a training sample set The data comes from the public dataset STL10, which contains N=113,000 images, where the i-th image x i The image size is 96×96, which is a color image containing classified objects; i Indicates the category label corresponding to the i-th picture, y i is a 10-dimensional vector with a value of 0 or 1, y i The value of position k in the _ is 1, which means that the image belongs to the kth category, and the value of 0 means that it does not belong to the corresponding category.

[0040] S2. Input the sample images in the training sample set into the image classification network model to obtain the labels of the sample images, and then use the deep learning cascade optimizer CAS to optimize the network parameter values ​​of the image classification network model to complete the training of the image classification network model.

[0041] In this example, a random depth mechanism was set, a cross-entropy loss function was selected, a batch size of 256 was set, a training iteration threshold of 150, and a fixed learning rate decay method was used, where the learning rate of the deep learning optimizer was reduced by a factor of 10 when k = 50 and k = 100. The batch size refers to the number of images fed into the network model at each iteration, for example, 256 images.

[0042] In order to verify the efficiency of the method of the present invention, another 9 publicly available deep learning optimizers SGD, AdaBelief, AdaBound, Adam, AdamW, SGD-Momentum (SGDM) and RAdam were selected for comparative testing.

[0043] The specific parameter settings are as follows:

[0044] The initial learning rate of AdaBelief, AdaBound, Adam, AdamW and Radam deep learning optimizers is set to 0.001 (λ = 10 -3 ), the accumulation coefficients of the first-order momentum and the second-order momentum are set to β2 = 0.9 and β3 = 0.999, and the initial learning rate of the SGD and SGDM deep learning optimizers is set to 0.1 (λ = 10 -1 ), the momentum coefficient of the SGDM optimizer is set to β1 = 0.9; in the CAS deep learning optimizer, the main optimizer Adam and the sub-optimizer SGD are set to the same initial learning rate set to 0.001 (λ = 10 -3 ).

[0045] The image classification results of the WideResNet-28 neural network model trained by the optimizer established by the present invention are compared with the image classification results of the WideResNet-28 neural network model trained by the existing optimizer. Figure 1As shown in , the horizontal axis represents the number of iterations and the vertical axis represents the test accuracy. Figure 1 It can be seen that after the CAS optimizer provided by the present invention has undergone two learning rate decays, the generalization performance of the CAS optimizer is better than that of other optimizers.

[0046] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An image classification method based on deep learning cascade optimizer, characterized in that, The specific steps include: Step 1: Collect image data to establish an image dataset, and create labels for the image data in the image dataset to form a training sample set; Step 2: Select a neural network model to establish an image classification network model. The input of the image classification network model is the image to be classified, and the output is the type of the image to be classified. Step 3: Construct a deep learning tandem optimizer formed by the SGD optimizer and the Adam optimizer in series, with the input being the gradient of the network parameter value of the image classification network model after the current k-1th iteration, and the output being the network parameter value of the image classification network model required to be set for the kth iteration; in the deep learning tandem optimizer, the network parameter value obtained by processing the input by the Adam optimizer is used as the first parameter value, and the network parameter value obtained by processing the sum of the gradient of the current network parameter value and the first parameter value by the SGD optimizer is used as the second parameter value, and the second parameter value is used as the output; Step 4: Input the sample images in the training sample set into the image classification network model to obtain the labels of the sample images, and then use the deep learning cascade optimizer to optimize the network parameter values ​​of the image classification network model to complete one iteration. When the number of iterations k reaches the set threshold, the training of the image classification network model is completed; Step 5: Input the image to be classified into the trained image classification network model, and the image classification network model outputs the category of the image to be classified to complete the image classification; In step 3, the network parameter value obtained by processing the sum of the gradient of the current network parameter value and the first parameter value by the SGD optimizer as the second parameter value is calculated as follows: in, is the second parameter value of the image classification network model obtained by the SGD optimizer after the k-1th iteration; η SGD is the initial learning rate of the SGD optimizer; is the second parameter value to be adopted in the kth iteration.

2. The image classification method based on deep learning cascade optimizer according to claim 1, characterized in that: The neural network model is CapsNet, Unet, WideResNet-28, ResNet-34, ResNet-18 or DenseNet-12.

3. The image classification method based on deep learning cascade optimizer according to claim 1, characterized in that: The calculation method of using the network parameter value obtained by processing the input by the Adam optimizer as the first parameter value in step 3 is: Among them, g k-1 is the gradient of the network parameter value of the image classification network model after the k-1th iteration; m k 、m k-1 are the first-order momentum of the Adam optimizer at the kth and k-1th iterations, respectively; v k 、v k-1 are the second-order momentum of the Adam optimizer at the kth and k-1th iterations, respectively; β2 and β3 are the cumulative coefficients of the first-order momentum and second-order momentum of the Adam optimizer, respectively. The value range of β2 and β3 is (0,1); is the Hadamard inner product; ε>0 is the smoothness coefficient; η Adam is the initial learning rate of the Adam optimizer; is the generalized learning rate of the kth iteration of the Adam optimizer; The first parameter value of the image classification network model output by the Adam optimizer at the k-1th iteration; is the first parameter value to be adopted in the kth iteration.

Images