Deep learning PID optimizer-based image classification method

Abstract

Disclosed in the present invention is a deep learning PID optimizer-based image classification method, which uses a generative algorithm to adaptively generate respective coefficients KP, KI and KD of different sub-network layers in a model, uses a PID optimizer to update an inner loop learnable parameter of the model at an inner loop, and uses an Adam optimizer to update an outer loop learnable parameter of the model at an outer loop. The deep learning PID optimizer-based image classification method in the present invention uses a PID optimizer-based image classification method to solve the problems of low efficiency and low precision of existing image classification tasks, and uses the generative algorithm to adaptively generate the coefficients KP, KI and KD of each sub-network layer of the model, solving the problem of manual parameter adjustment in a PID control algorithm, allowing for a greater adaptive capacity and higher recognition precision, and being able to quickly adapt to new tasks.

Description

An image classification method based on deep learning PID optimizer Technical Field

[0001] The present invention belongs to the technical field of computer vision and pattern recognition, and relates to an image classification method based on a deep learning PID optimizer. Background Art

[0002] Deep learning-based image processing technology has achieved significant breakthroughs in machine vision fields (such as image classification), overcoming the challenges of low accuracy and efficiency associated with traditional feature engineering techniques. The powerful performance of deep learning is driven by large datasets, complex neural network structures, and the enormous computational effort and training time required. To reduce model training time and accelerate learning efficiency, the use of fast and robust optimizers is a core technology in deep learning models. These optimizers can rapidly update model weights and learn effective empirical knowledge, while also enabling the model to converge to a near global optimum.

[0003] The existing optimizer algorithms in deep learning have low learning efficiency when the distribution of training tasks and test tasks is inconsistent. They require dozens of rounds of iterative learning to converge, and the recognition accuracy is greatly reduced. Secondly, there is a lack of theoretical knowledge research, weak adaptability, and only improvements in areas involved in specific tasks.

[0004] PID control theory has a well-established application in the field of control science, enabling rapid adjustments to minimize deviations. Deep neural networks are widely used as feature extractors, mapping image data into an embedding space to construct specific feature vectors, which are then used in downstream tasks. A parameter update criterion based on a PID optimizer can quickly reduce deviations in algorithmic predictions. The proportional term P is considered the error between the model's predicted score for a category and the actual label, and is used to adjust the model's importance to each category. The integral term I is used to address long-term error accumulation, helping the model better learn the characteristics of scarce categories. The differential term D balances P and I, preventing the model from overfitting or over-adjusting certain samples. By adjusting the model's weights based on future predictions and historical accumulation, the algorithm is able to quickly adapt to new tasks and maintain strong recognition performance for entirely new tasks, significantly improving image classification accuracy.

[0005] Therefore, an image classification method based on deep learning PID optimizer is needed to solve the above problems. Summary of the Invention

[0006] In order to solve the above-mentioned problem of lack of specific, complete and mature practical experimental equipment, the present invention provides an image classification method based on deep learning PID optimizer.

[0007] An image classification method based on a deep learning PID optimizer of the present invention comprises the following steps:

[0008] Step S1: Divide the dataset into training dataset, validation dataset and test dataset;

[0009] Step S2: Construct an image classification model based on deep learning, wherein the image classification model includes an N_layer layer subnetwork, and initialize the weight and maximum number of iterations of the N_layer layer subnetwork;

[0010] Step S3: using the training data set to train the image classification model and calculate the loss between the model prediction value and the true label;

[0011] Step S4: Generate K for each layer of sub-network based on the generation algorithm P ,K I and K D coefficient;

[0012] Step S5: Based on the K generated in step S4 P ,K I and K D The coefficients use the PID optimizer to update the inner-loop learnable parameters of the image classification model, and the Adam optimizer to update the outer-loop learnable parameters of the image classification model;

[0013] Step S6: testing the generalization performance of the image classification model on the validation dataset, and using the PID optimizer to adjust the weight parameters of the N_layer subnetwork as the initial weight values ​​of the image classification model;

[0014] Step S7: Repeat steps S3-S6 until the number of iterations reaches the maximum number of iterations set in step S2, and obtain the optimized image classification model.

[0015] Furthermore, step one also includes image preprocessing: normalizing the mean and variance of each image in the dataset, and then converting the image data into a tensor format.

[0016] Furthermore, the image classification model in step S2 is composed of a 4-layer convolutional neural network CNN or a ResNet12 structure.

[0017] Furthermore, the number of convolution kernels of the convolutional neural network CNN is 64, and the convolution kernel size is 3.

[0018] Furthermore, in step S3, the image classification model is trained using the training data set: a fixed number of batch data are read in each round of training, and each batch data includes N_way×(K support +K query) sample data, where N_way is the number of categories, K support is the number of support sets for each category, K query is the number of query sets for each category.

[0019] Furthermore, the loss value in step S3 is calculated by the cross entropy loss function.

[0020] Furthermore, the generated algorithm in step S4 consists of two Linear layers, whose input dimension is N_layer×3 and output dimension is N_layer×3, where N_layer is the number of layers of the image classification model.

[0021] Furthermore, in step S4, the K of each sub-network is generated using the following formula: P ,K I and K D coefficient:

[0022] Where f is the generation algorithm, N_layer is the number of layers of the image classification model, and are the mean, variance, and gradient values ​​of each layer parameter of the image classification model.

[0023] Furthermore, in step S5, the PID optimizer is used to update the inner loop learnable parameters of the image classification model using the following formula:

[0024] Where K P ,K I ,K D is the adaptive coefficient generated by the generation algorithm in step S5, is the gradient value of the parameter weight of the i-th layer sub-network of the image classification model at the t-th iteration, α is the weight attenuation factor, L t and L t-1 are the loss values ​​of the image classification model at the tth and t-1th iterations, respectively. and are the parameter values ​​of the i-th layer sub-network of the image classification model at the t-th and t-1-th iterations, α t-m and α t-m+1 are the weight attenuation factors at time tm and t-m+1 respectively.

[0025] Beneficial effects: The image classification method based on deep learning PID optimizer of the present invention solves the problem of slow efficiency and low precision of existing image classification tasks by using the image classification method based on PID optimizer, and adopts the generative algorithm to adaptively generate the K of each layer sub-network of the model. P ,KI and K D The coefficient solves the problem of manually adjusting parameters in the PID control algorithm, has stronger generalization and robustness, and can achieve the effect of quickly adapting to new tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] FIG1 is a schematic flow chart of an image classification method based on a deep learning PID optimizer according to the present invention;

[0027] Figure 2 is a schematic diagram of the parameter update criteria of the inner loop PID optimizer. DETAILED DESCRIPTION

[0028] The present invention will now be described in further detail with reference to the accompanying drawings, which are simplified schematic diagrams that illustrate the basic structure of the present invention in a schematic manner.

[0029] As shown in Figure 1-2, the image classification method based on deep learning PID optimizer of the present invention includes the following steps:

[0030] Step S1: Divide the entire dataset into training dataset, validation dataset and test dataset according to a certain ratio, and there are no repeated categories among the three. Taking the mini-ImageNet dataset as an example, the dataset contains 100 categories, each category contains 600 images, and 70 categories are selected as training datasets, 20 categories as validation datasets, and 10 categories as test datasets; normalize the mean and variance of each image, and then convert the image data into tensor format. Read a fixed number of batch data in each round of training. Each batch data contains N_way×(K support +K query ) sample data, where N_way is the number of categories, usually set to 5 or 20, K support is the number of support sets for each category, usually set to 5 or 1, K query is the number of query sets for each category, usually set to 15;

[0031] Step S2: Construct a small sample image classification learning model and initialize the weights: The constructed model consists of a 4-layer CNN or ResNet12 structure, where the number of convolution kernels in each layer of the CNN model is set to 64 and the convolution kernel size is set to 3; in particular, each sub-network of the constructed model can use its own weights to forward calculate the feature vector of the image, or use specified weights to calculate the feature vector of the image; for the convolution layer of the model, use Gaussian or Xavier initialization to initialize the weights, and for the bias layer of the model, use zero initialization to initialize the weights;

[0032] Step S3: Pack the images of the training dataset into batch data as described in S1, set the number of iterations, input the packed batch data into the model built in S2 for forward calculation, and map the image features into a feature vector of fixed length;

[0033] Step S4: Train the model based on the training dataset and calculate the loss between the model-predicted labels and the true label values ​​corresponding to the images. The loss function can be calculated using cross entropy loss.

[0034] Step S5: Based on the generation algorithm, adaptively generate K for each layer of the sub-network of the model P ,K I ,K D The generation algorithm consists of two layers of learnable Linear layers, with an input dimension of N_layer×3 and an output dimension of N_layer×3, where N_layer is the number of layers in the model constructed in step 2. Specifically, the input data is the mean and variance of the weights of each layer of the model and the gradient value corresponding to the weights of each layer of the model, that is:

[0035] Where f is the generation algorithm,

[0036] N_layer is the number of layers of the model constructed in step 2, and the output is the K of each layer of the sub-network of the model P ,K I ,K D coefficient.

[0037] Step S6: Based on the loss value calculated in S4, the gradient value of each sub-network is obtained through gradient back propagation operation, and the PID optimizer is used to update the inner loop learnable parameters of the model, and the Adam optimizer is used to update the outer loop learnable parameters of the model. The calculation formula of the PID optimizer parameter update model is:

[0038] Where K P ,K I ,K D is the adaptive coefficient generated by the generation algorithm in step S5, is the gradient value of the parameter weight of the i-th layer sub-network of the model constructed in step S2 at the t-th iteration, and α is the weight decay factor;

[0039] Step S7: In the above S6, after each iteration of the training dataset, the generalization performance of the model is tested on the validation dataset, and the PID optimizer is used to fine-tune the model weight parameters as the initial weight values ​​of the model;

[0040] Step S8: Use the model weights learned in the above S7 as the initial weight values ​​of the test task, and after a few steps of iterative calculation, test the recognition accuracy of the model on the test dataset.

[0041] The present invention adopts PID optimizer to update the learnable parameters of the inner loop of the model, which solves the problem of low learning efficiency when the training and test tasks are distributed differently in the image classification learning algorithm; the K of each layer of the sub-network of the model is adaptively generated by the generation algorithm. P ,K I ,K D The new method solves the problem of manual parameter adjustment in traditional PID controllers and makes the model more adaptable. The introduction of PID control theory algorithms avoids the weak interpretability of traditional image classification learning methods. Compared with previous methods, it has stronger adaptability and higher recognition accuracy, and can quickly adapt to new tasks.

[0042] With the above-described preferred embodiments of the present invention as a guide, and with reference to the above description, relevant personnel are fully capable of making various changes and modifications without departing from the technical scope of this invention. The technical scope of this invention is not limited to the contents of the specification and must be determined according to the scope of the claims.

Claims

1. An image classification method based on deep learning PID optimizer, characterized in that, The following steps are involved: Step S1: Divide the dataset into training dataset, validation dataset and test dataset; Step S2: Construct an image classification model based on deep learning, wherein the image classification model includes an N_layer layer subnetwork, and initialize the weight and maximum number of iterations of the N_layer layer subnetwork; Step S3: using the training data set to train the image classification model and calculate the loss between the model prediction value and the true label; Step S4: Generate K for each layer of sub-network based on the generation algorithm P ,K I and K D coefficient; Step S5: Based on the K generated in step S4 P ,K I and K D The coefficients use the PID optimizer to update the inner-loop learnable parameters of the image classification model, and the Adam optimizer to update the outer-loop learnable parameters of the image classification model; Step S6: testing the generalization performance of the image classification model on the validation dataset, and using the PID optimizer to adjust the weight parameters of the N_layer subnetwork as the initial weight values of the image classification model; Step S7: Repeat steps S3-S6 until the number of iterations reaches the maximum number of iterations set in step S2, and obtain the optimized image classification model.

2. The image classification method based on deep learning PID optimizer according to claim 1, characterized in that Step 1 also includes image preprocessing: normalizing the mean and variance of each image in the dataset, and then converting the image data into tensor format.

3. The image classification method based on deep learning PID optimizer according to claim 1, characterized in that: The image classification model in step S2 is composed of a 4-layer convolutional neural network CNN or a ResNet12 structure.

4. The image classification method based on deep learning PID optimizer according to claim 3, characterized in that: The number of convolution kernels of the convolutional neural network CNN is 64, and the convolution kernel size is 3.

5. The image classification method based on deep learning PID optimizer according to claim 1, characterized in that: In step S3, the image classification model is trained using the training data set: a fixed number of batch data are read in each round of training, and each batch data includes N_way×(K support +K query ) sample data, where N_way is the number of categories, K support is the number of support sets for each category, K query is the number of query sets for each category.

6. The image classification method based on deep learning PID optimizer according to claim 1, characterized in that: The loss value in step S3 is calculated using the cross entropy loss function.

7. The image classification method based on deep learning PID optimizer according to claim 1, characterized in that: The generated algorithm in step S4 consists of two Linear layers, whose input dimension is N_layer×3 and output dimension is N_layer×3, where N_layer is the number of layers of the image classification model.

8. The image classification method based on deep learning PID optimizer according to claim 1, characterized in that: In step S4, the following formula is used to generate K for each layer of sub-network: P ,K I and K D coefficient: Where f is the generation algorithm, N_layer is the number of layers of the image classification model, and are the mean, variance, and gradient values of each layer parameter of the image classification model.

9. The image classification method based on deep learning PID optimizer according to claim 1, characterized in that: In step S5, the PID optimizer is used to update the inner loop learnable parameters of the image classification model using the following formula: Where K P ,K I ,K D is the adaptive coefficient generated by the generation algorithm in step S5, is the gradient value of the parameter weight of the i-th layer sub-network of the image classification model at the t-th iteration, α is the weight attenuation factor, L t and L t-1 are the loss values of the image classification model at the tth and t-1th iterations, respectively. and are the parameter values of the i-th layer sub-network of the image classification model at the t-th and t-1-th iterations, α t-m and α t-m+1 are the weight attenuation factors at time tm and t-m+1 respectively.

Images