A method for training an image classification model based on meta learning and application thereof

Abstract

The application discloses a kind of training method and application of image classification model based on meta learning, image classification technical field, at the end of the first round in-loop iteration in the pre-training phase of meta learning, pruning operation is carried out to subtask model, a lot of unnecessary calculation is saved under the condition of guaranteeing model accuracy, the calculation amount of pre-training phase and fine-tuning phase is greatly reduced, and the threshold of meta learning fine-tuning phase on mobile terminal device is reduced;At the same time, the accuracy loss caused by pruning is made up by the fine-tuning phase after pre-training, the size of model is reduced without changing the model accuracy, so that fast and efficient training is realized on mobile terminal device.

Description

Technical Field

[0001] This invention belongs to the field of image classification technology, and more specifically, relates to a training method and application of an image classification model based on meta-learning. Background Technology

[0002] Image classification has wide applications in various aspects of daily life, such as image review and object recognition. Using deep learning to solve image classification problems is a current research hotspot. However, deep learning often uses large amounts of data from a specific scenario to train the model. When the scenario or dataset changes, the model needs to be retrained, resulting in low training efficiency. To address this issue, researchers proposed the concept of meta-learning. This involves using various learning tasks as training data to train a model, enabling it to learn new tasks with only a small number of samples. The emergence of meta-learning provides a solution for training and effectively operating image classification models across different application scenarios and datasets.

[0003] However, with the popularization of edge intelligent devices, in order to make full use of the computing resources of edge devices, training tasks are often migrated from the cloud to the edge, making mobile terminal devices an indispensable carrier for deploying and training models. However, due to the limited storage and computing power of mobile terminal devices, and the fact that the pre-training stage of meta-learning often needs to be trained on multiple tasks, the computational load is large. Furthermore, the application characteristics of meta-learning require the fine-tuning stage of meta-learning to be fast and efficient in practical applications. Therefore, existing image classification models based on meta-learning cannot achieve fast and efficient training on mobile terminal devices. Summary of the Invention

[0004] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention provides a training method and application of an image classification model based on meta-learning, so as to solve the technical problem that the existing image classification models based on meta-learning cannot achieve fast and efficient training on mobile terminal devices.

[0005] To address the aforementioned issues, firstly, this invention provides a training method for an image classification model based on meta-learning, comprising the following steps:

[0006] S1. Divide the collected image samples with classification labels into a meta-training set and a meta-test set with different image categories;

[0007] S2. Randomly sample N1 subtasks from several pre-prepared subtasks; divide the meta-training set to obtain subtask sets corresponding to the N1 subtasks; each subtask set includes a support set and a query set; one subtask corresponds to one subtask model; N1 is an integer greater than 1.

[0008] S3. Train the corresponding sub-task model using the support sets of each sub-task set under the meta-training set; determine whether the current inner loop iteration count is 1. If yes, perform pruning operation on the sub-task model and go to step S4; otherwise, go directly to step S4.

[0009] S4. Repeat step S3 to iterate the inner loop until the current number of inner loop iterations reaches the first preset number of iterations.

[0010] S5. Update the parameters in the image classification model to be trained based on the average classification loss value of each sub-task model; wherein, the classification loss value of each sub-task model is obtained when the query set of each sub-task set under the meta-training set is input into the corresponding sub-task model for performance testing;

[0011] S6. Repeat steps S2-S5 to perform outer loop iterations until the current number of outer loop iterations reaches the second preset number of iterations;

[0012] S7. Sample N2 subtasks from multiple pre-prepared subtasks, divide the meta-test set to obtain the subtask set corresponding to the N2 subtasks, and use the support set in the subtask set corresponding to the N2 subtasks to fine-tune the image classification model to obtain the trained image classification model; where N2 is an integer greater than or equal to 1.

[0013] Both the sub-task model and the image classification model are based on neural networks.

[0014] More preferably, in step S3, channel pruning and weight pruning are performed on the subtask model in sequence.

[0015] More preferably, the sub-task model is a convolutional neural network; the method for pruning the sub-task model includes:

[0016] S31. By applying L1 norm penalty to the scaling factor of the BN layer in the subtask model, the scaling factor of the BN layer in the subtask model is sparsified during training. The convolutional layer channels corresponding to the scaling factors whose absolute value is less than the preset channel pruning threshold are removed to obtain the subtask model after channel convolution.

[0017] S32. For all neuron connections between adjacent layers in the subtask model after channel convolution, remove neuron connections whose L1 norm values ​​of neuron connection weights are less than the preset weight pruning threshold, thus completing the pruning operation on the subtask model.

[0018] More preferably, the training method of the above-mentioned image classification model based on meta-learning further includes: step S8 executed between step S5 and step S6, and step S9 executed between step S6 and step S7;

[0019] Step S8 includes: performing a quantization operation on the image classification model;

[0020] Step S9 includes: performing an inverse quantization operation on the image classification model.

[0021] More preferably, after each round of inner loop iteration, the performance of the corresponding sub-task model is tested using the query set in the sub-task set under the meta-training set to obtain the classification loss value of the sub-task model under the corresponding inner loop iteration round; at this time, the above step S5 includes: assigning different weight values ​​to all iteration rounds of the inner loop respectively, calculating the weighted average of the classification loss values ​​of the sub-task model under each inner loop iteration round, and updating the parameters in the image classification model to be trained based on the obtained weighted average value.

[0022] More preferably, in the subtask set, the number of image samples in the support set is less than the number of image samples in the query set.

[0023] More preferably, in each subtask set, the ratio of the number of image samples in the support set to the number of image samples in the query set is 1:15.

[0024] More preferably, the data distribution is the same for each subtask set.

[0025] Secondly, the present invention provides an image classification method, comprising: inputting an image to be classified into an image classification model trained using the training method of the meta-learning-based image classification model provided in the first aspect of the present invention, and obtaining an image classification result.

[0026] Thirdly, the present invention provides an image classification system, comprising: a memory and a processor, wherein the memory stores a computer program, and the processor executes the image classification method provided in the second aspect of the present invention when executing the computer program.

[0027] Fourthly, the present invention also provides a computer-readable storage medium comprising a stored computer program, wherein, when the computer program is executed by a processor, it controls the device where the storage medium is located to execute the training method of the image classification model based on meta-learning provided in the first aspect of the present invention and / or the image classification method provided in the second aspect of the present invention.

[0028] In summary, the above-described technical solutions conceived in this invention can achieve the following beneficial effects:

[0029] 1. This invention provides a training method for an image classification model based on meta-learning. At the end of the first iteration of the inner loop in the pre-training phase of meta-learning, a pruning operation is performed on the sub-task model to remove channels and weights with poor expressive power. This allows the calculation of the pruned channels to be skipped in subsequent iterations, achieving a simplification of model computation. While maintaining model accuracy, many unnecessary calculations are saved, greatly reducing the computational load in the pre-training and fine-tuning phases, and lowering the threshold for meta-learning fine-tuning on mobile devices. At the same time, this invention compensates for the accuracy loss caused by pruning through the fine-tuning phase after pre-training, reducing the model size without changing the model accuracy, thereby achieving fast and efficient training on mobile devices.

[0030] 2. The training method for the image classification model based on meta-learning provided by this invention uses a channel pruning compression method to perform structured pruning on the sub-task model, removing channels with weak expressive power to achieve coarse-grained pruning of the sub-task model, and further removing weights with poor expressive power in the fully connected layer of the pruned sub-training model to achieve fine-grained pruning of the fully connected layer, thereby significantly reducing the amount of computation and achieving the effect of acceleration.

[0031] 3. The training method for the image classification model based on meta-learning provided by this invention quantizes the image classification model after the pre-training stage, reducing the precision from 32-bit to 8-bit, thus reducing the size of the image classification model to about one-quarter of its previous size. In the fine-tuning stage, the model is retrained by inverse quantization to compensate for the precision loss caused by quantization. By cooperating with the fine-tuning stage, the size of the model can be further reduced without changing the model precision.

[0032] 4. The training method for the image classification model based on meta-learning provided in this invention calculates the loss value of each inner loop update round when updating the parameters of the image classification model to be trained during the pre-training stage, assigns different weights to each, and finally performs a weighted average and performs an update once. This not only allows the loss of the previous inner loop updates to participate in the optimization, but also reduces the computation time required, reduces the number of backpropagation, and further increases the stability and generalization performance of the model. Attached Figure Description

[0033] Figure 1 The flowchart shows the training method of the image classification model based on meta-learning provided in Embodiment 1 of the present invention.

[0034] Figure 2 This is a schematic diagram of the training data distribution provided in Embodiment 1 of the present invention;

[0035] Figure 3 This is a schematic diagram illustrating the pre-training of a single task provided in Embodiment 1 of the present invention;

[0036] Figure 4 A flowchart of the fine-tuning stage provided in Embodiment 1 of the present invention;

[0037] Figure 5 This is a schematic diagram of the channel pruning process provided in Embodiment 1 of the present invention;

[0038] Figure 6 This is a schematic diagram of weighted pruning provided in Embodiment 1 of the present invention;

[0039] Figure 7 This is a flowchart of the model quantization calculation method in the meta-learning pre-training stage provided in Embodiment 1 of the present invention;

[0040] Figure 8 A flowchart of the pre-training stage provided in Embodiment 1 of the present invention;

[0041] Figure 9 This is a schematic diagram of pruning optimization provided in Embodiment 1 of the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0043] Example 1

[0044] A training method for an image classification model based on meta-learning, such as Figure 1 As shown, it includes the following steps:

[0045] S1. Divide the collected image samples with classification labels into a meta-training set and a meta-test set with different image categories;

[0046] S2. Randomly sample N1 subtasks from several pre-prepared subtasks; divide the meta-training set to obtain subtask sets corresponding to the N1 subtasks; each subtask set includes a support set and a query set; one subtask corresponds to one subtask model; N1 is an integer greater than 1.

[0047] S3. Train the corresponding sub-task model using the support sets of each sub-task set under the meta-training set; determine whether the current inner loop iteration count is 1. If yes, perform pruning operation on the sub-task model and go to step S4; otherwise, go directly to step S4.

[0048] S4. Repeat step S3 to perform inner loop iteration until the current number of inner loop iterations reaches the first preset number of iterations (5 in this embodiment);

[0049] S5. Update the parameters in the image classification model to be trained based on the average classification loss value of each sub-task model; wherein, the classification loss value of each sub-task model is obtained when the query set of each sub-task set under the meta-training set is input into the corresponding sub-task model for performance testing;

[0050] S6. Repeat steps S2-S5 to perform outer loop iterations until the current number of outer loop iterations reaches the second preset number of iterations (5 in this embodiment);

[0051] S7. Sample N2 subtasks from multiple pre-prepared subtasks, divide the meta-test set to obtain the subtask set corresponding to the N2 subtasks, and use the support set in the subtask set corresponding to the N2 subtasks to fine-tune the image classification model to obtain the trained image classification model; where N2 is an integer greater than or equal to 1.

[0052] Both the sub-task model and the image classification model are based on neural networks.

[0053] It should be noted that the training method for the image classification model based on meta-learning provided in this invention mainly consists of two stages: the first stage is the pre-training stage of meta-learning, which trains the network's initialization parameters on a given subset of tasks, specifically corresponding to steps S2-S6 above; the second stage involves fine-tuning the initialization parameters according to the new task, specifically corresponding to step S7 above. The overall idea is to train parameters sensitive to different tasks through iterative training, enabling the model to be sensitive to new tasks and quickly adapt to them. This allows the model to fine-tune its parameters on new tasks using only a small number of samples and one or more gradient iterations.

[0054] The data distribution is the same for each subtask set. Within each subtask set, the number of image samples in the support set is less than the number of image samples in the query set. In this embodiment, the ratio of image samples in the support set to those in the query set is 1:15 for each subtask set. Specifically, throughout the entire process, all training and testing data are processed on a subtask-by-subtask basis; the model training process revolves around each subtask. This is done while training the image classification model M. fine-tune During the training process, the categories included P1 to P5, with 5 labeled image samples per category for training and 15 labeled image samples per category for testing. In addition to the labeled image samples from P1 to P5, the training data also included images from ten other categories, C1 to C5. 10 Each class has 50 labeled samples to help train the meta-learning model M. meta N-way K-shot is used to describe a task containing N data sample classifications, with K training samples in each classification. Common data sample configurations include 5-way 1-shot, 5-way 5-shot, 20-way 1-shot, and 20-way 5-shot. This embodiment uses the 5-way 5-shot data configuration.

[0055] At this time, C1~C 10 That is, the meta-training set, C1 to C2. 10 The sample contains a total of 500 samples, denoted as D. meta-train It is used to train the sub-task model M meta The dataset is defined as follows. In contrast, P1 to P5 constitute the meta-test set, containing a total of 100 samples, denoted as D. meta-test It is used to train and test image classification model M fine-tune The dataset.

[0056] Based on the 5-way 5-shot setup, in training sub-task model M meta The stages, from C1 to C 10Five classes are randomly selected from the dataset, and 20 labeled samples are randomly selected from each class to form a TaskT. Five labeled samples are then selected from these 20 samples to form the SupportSet of TaskT, and the remaining 15 samples are called the QuerySet of TaskT. All samples in this TaskT can be compared to training data samples in typical machine learning.

[0057] Randomly selecting several tasks according to this pattern forms a batch needed for training. The training data distribution is as follows: Figure 2 As shown in the figure, the entire training set is a collection of Tasks, called Tasks. Each row represents a Task, and the training set (SupportSet) for each Task consists of one sample from each of the five animal classes. The QuerySet for each Task also consists of one sample from each of the five animal classes. The upper part of the figure represents the training set for the experiment, and the lower part represents the test set. The data distribution is the same across all Tasks.

[0058] In the first stage, the pre-training stage, the sub-task models for each sub-task are trained using the support sets of the sub-tasks under the meta-training set. This trains the sub-task model parameters for each sub-task, primarily to obtain the initial parameters for each task and perform fitting. During training, the parameters θ' of the i-th sub-task model are... i The update formula is: in, Let α be the cross-entropy loss function of the i-th sub-task model; α is the inner loop learning rate; the analytical diagram of pre-training a single task is shown below. Figure 3 As shown; specifically, in this process, the image classification model to be trained is first copied to obtain multiple sub-task models, and then each sub-task model is trained separately, performing backpropagation and parameter updates. Next, the performance of the sub-task models is tested using query sets from different sub-tasks, the loss is calculated and averaged, and then the parameters θ in the image classification model to be trained are updated using gradient descent, as shown in the formula. As shown; where β is the outer loop learning rate; T i Let p(T) be the i-th subtask; p(T) is the subtask distribution in the meta-training set; in this process, the parameters of each subtask model are not updated, but the parameters of the image classification model are updated directly.

[0059] To further clarify, in the above process, gradient updates are performed twice in a training epoch: once in the inner loop and once in the outer loop. The outer loop update is based on the average of the loss function from the last epoch of the inner loop. Experiments revealed that this training method is highly unstable. During the outer loop, because the weight parameters need to be generated multiple times through the network, gradients are backpropagated from the outer loop to the inner loop. Each layer of the network is backpropagated several times, which easily leads to gradient explosion or gradient decay. Furthermore, if the network structure is very deep, gradient explosion or gradient decay may occur after only one layer has been propagated. When the gradient values ​​become problematic, the parameter updates will naturally fail, resulting in unstable results. Specifically, since the update of the image classification model parameters depends on the loss value from the last step on all query sets, the parameters from the previous inner loops can only be implicitly optimized when backpropagating using this value. The losses they generate cannot be directly used for explicit optimization, thus the algorithm's stability is poor.

[0060] To address the aforementioned issues, one specific implementation involves changing the approach from a single-step to a multi-step process. Instead of updating the parameters in the image classification model after the final inner loop iteration, the approach updates the parameters after each inner loop iteration. The principle is to increase training time, sacrificing computational resources for algorithm stability, as more backpropagation iterations during training enhance stability. However, this method trades time for stability, resulting in high costs. Therefore, a preferred solution employs a single-step weighted loss optimization approach, shifting the timing of outer loop updates during the pre-training phase from after the inner loop iteration. The update changes the approach to calculating the loss at each step of the inner loop and updating the model using a weighted average of the losses from each step. This further enhances the model's stability and generalization performance. Specifically, after each inner loop iteration, the performance of the corresponding sub-task model is tested using the query set in the sub-task set under the meta-training set, obtaining the classification loss value of the sub-task model in the corresponding inner loop iteration. At this point, step S5 includes: assigning different weight values ​​to all iterations of the inner loop, calculating the weighted average of the classification loss values ​​of the sub-task model in each inner loop iteration, and updating the parameters in the image classification model to be trained based on the obtained weighted average. Specifically, it can be seen that the reason for the poor algorithm stability is that the losses updated in the previous inner loop iterations cannot participate in the optimization; the loss from the last iteration is used instead. Therefore, the loss values ​​of all inner loop iterations can be calculated separately, assigned different weights, and finally weighted and averaged before updating again. This not only allows the losses updated in the previous inner loop iterations to participate in the optimization but also reduces the computation time and the number of backpropagation iterations.

[0061] The flowchart for the second stage, the fine-tuning stage, is as follows: Figure 4 As shown, this stage primarily involves training on the test dataset, which is still divided into a support set and a query set. The goal of the training task in the first stage is to find a good hyperparameter setting. The second stage uses the pre-trained parameters for fine-tuning, allowing the model to quickly adapt to the new data distribution. The general algorithm flow is the same as the pre-training stage, except that the fine-tuning stage does not require parameter initialization; instead, it utilizes the parameters of the pre-trained image classification model. Furthermore, the fine-tuning stage does not require batching; it only needs to extract a sub-task from the meta-test set for learning, train the model using the support set of this task, and test the model using the query set. However, to avoid extreme cases, multiple sub-tasks are randomly selected from the meta-test set to fine-tune the pre-trained image classification model separately. Finally, the test results are averaged to avoid extreme situations. Since the query set of the test sub-task is used to test the model, and the labels are unknown to the model, there is no second gradient update in the fine-tuning stage; instead, the parameters are updated directly using the results of the first gradient calculation.

[0062] It should be noted that meta-learning provides a new approach and solution for few-shot learning. It aims to train the model's parameters to a suitable position through pre-training, and then fine-tune the parameters to an optimal position using a few training epochs with new tasks and data distributions. Although this process involves few-shot learning with a small support set in each sub-task, the number of sub-tasks still needs to be substantial to ensure the model's ability to adapt to new tasks. This results in a large number of epochs and requires significant computation. To address this issue, this invention performs pruning operations on the sub-task model sequentially to avoid unnecessary computations. This also reduces the model size after the pre-training phase, as well as the number and time of computations in the fine-tuning phase. Furthermore, it reduces overfitting during pre-training and improves the model's generalization ability.

[0063] Specifically, in one optional implementation, in step S3 above, channel pruning and weight pruning are performed sequentially on the sub-task model to reduce the number of parameters and computational load in the sub-task model without significantly sacrificing the accuracy of the original algorithm, thereby improving the efficiency of the model's inference process. Specifically, the sub-task model is a convolutional neural network, mainly including convolutional layers, BatchNorm, and activation layers; the method for pruning the sub-task model includes:

[0064] S31. By applying L1 norm penalty to the scaling factor of the BN layer in the subtask model, the scaling factor of the BN layer in the subtask model is trained to be sparse; the convolutional layer channels corresponding to the scaling factors whose absolute value is less than the preset channel pruning threshold (in this embodiment, the value is 30% after the descending order of expressive power) are removed to obtain the subtask model after channel convolution.

[0065] Specifically, this invention achieves model compression by pruning convolutional layer channels with poor expressive power in each layer. This embodiment selects the scaling factor of the Batch Normalization (BN) layer as the metric for pruning convolutional layer channels. Since the L1 norm penalty restricts the scaling factor of the BN layer to near 0, it easily identifies channels with poor expressive power in the model. Furthermore, regularization has a relatively small performance loss, and in some cases, it can even improve the model's generalization performance. Therefore, during training, L1 regularization is added to the scaling factor of the BN layer to achieve sparsity. This allows the identification of unimportant convolutional layer channels by ensuring the BN layer's scaling factor approaches 0. Specifically, through the formula... The scaling factor (denoted as γ parameter) of the Batch Normalization (BN) layer is derived to reflect the expressive power of the convolutional channels. Based on the magnitude of the γ parameter calculated using the L1 norm of the BN layer, convolutional kernels or channels corresponding to γ ​​parameters with smaller absolute values ​​are selected and removed. This process removes the unimportant convolutional kernels or their corresponding channels, and the remaining kernels are rearranged and combined to form new convolutional layer weights. The removal of selected channels will result in some accuracy loss, which will be compensated for in subsequent fine-tuning stages.

[0066] It should be noted that during the entire global pruning operation, if the set pruning rate is large enough, the pruning threshold may exceed the maximum value of parameter γ in a certain BN layer. Based on this consideration, the maximum value of parameter γ in each BN layer needs to be calculated before pruning, and the minimum value among them is set as the upper limit of the pruning threshold. Once the channels that need to be pruned in each layer are determined through calculation, the entire network can be pruned. Pruning is achieved by recombining the convolutional layer parameters and BN layer parameters. The criteria and process for determining which convolutional kernel channels to prune are as follows... Figure 5 The diagram shows the convolution kernel during the channel pruning process.

[0067] This process requires adjusting the pruning mask M of the input channel. in And the pruning mask M of this layer channel x The corresponding convolutional kernels and channels are pruned, and the pruning mask for each layer is composed of the BN layer parameter γ of the current convolutional layer and the pruning threshold. For example... Figure 5As shown, the leftmost layer is the input feature map of this layer. It can be seen that the second layer channel is judged to be the channel with a smaller parameter γ corresponding to the BN layer, which means it has poor expressive power and needs to be pruned. According to the convolution calculation process, the second row of the corresponding convolution kernel tensor also needs to be pruned, because the second layer channel of each convolution kernel corresponds to the second layer of the input feature map. Similarly, the rightmost layer is the output feature map of this layer. It can be seen that the first layer channel is also a channel with poor expressive power and needs to be pruned. The first column of the corresponding convolution kernel tensor also needs to be pruned.

[0068] S32. Apply weight pruning to the fully connected layer. For all neuron connections between adjacent layers in the subtask model after channel convolution, remove neuron connections whose L1 norm weight is less than the preset weight pruning threshold (in this embodiment, the value is taken as the bottom 20% of the expression ability in descending order) to complete the pruning operation of the subtask model.

[0069] It should be noted that step S31 uses a channel pruning compression method to perform structured pruning on the subtask model. This is the most effective coarse-grained pruning method, which can significantly reduce computation and thus achieve acceleration. Step S32, as a supplement to the pruning method, performs fine-grained pruning on the fully connected layers of the pruned model, which can complement channel pruning. The main pruning idea and steps are the same as channel pruning: first, a network needs to be pre-trained, then the weight of each neuron is evaluated, then the pruning threshold is calculated based on the pruning rate, and then the weights below the threshold are pruned. This process will result in some loss of accuracy, which is then compensated for through a fine-tuning stage.

[0070] In step S32, taking the x-th and x+1-th adjacent layers in the sub-task model after channel convolution as an example. Figure 6 As shown, calculate the weights of the connections between neurons in layer x and layer x+1, and set i x With k y The connection weights are recorded in the x-th row and y-th column of matrix f1. Next, the L1 norm of each neuron connection is calculated. The weight pruning threshold is calculated according to the set pruning rate. Weights below the weight pruning threshold are marked. The specific marking method is to set the corresponding position in matrix f2 to 0, and set the unmarked weights to 1. Matrix f2 is the mask. In the final pruning step, we only need to multiply the weights in matrix f1 with the corresponding mask in matrix f2.

[0071] Furthermore, during the training of deep learning networks, the computational complexity of multiplying and adding model parameters is enormous, often reaching millions. This necessitates specialized cloud computing platforms to meet real-time computing demands, which is unacceptable for products on edge intelligent devices. This invention employs model quantization to reduce computational load. Quantization is the process of converting the precision of parameters in the network from high to low precision. Quantization algorithms vary depending on the number of bits used, including binary quantization and ternary quantization. For example, converting a 32-bit floating-point number to an 8-bit integer (int8) reduces the model size to one-quarter of its original size. Quantization can result in smaller model size, lower power consumption, and faster computation speed. Taking four-bit quantization as an example (2-bit quantization), the 32-bit precision weights are quantized into four values: -1.0, 0, 1.5, and 2.0. In this process, the space occupied by each parameter is compressed from 32 bits to 2 bits, a reduction to one-sixteenth. This is the benefit of model quantization.

[0072] In an optional second embodiment, the training method of the above-mentioned image classification model based on meta-learning further includes: step S8 executed between step S5 and step S6, and step S9 executed between step S6 and step S7;

[0073] Step S8 includes: performing a quantization operation on the image classification model;

[0074] Step S9 includes: performing an inverse quantization operation on the image classification model.

[0075] Specifically, the image classification model parameters are converted from floating-point numbers to integers after quantization. Although the recovered values ​​have some error compared to the uncompressed values, the model's accuracy has proven to be robust against noise caused by compression. The flowchart of the model quantization calculation method in the meta-learning pre-training stage is shown below. Figure 7As shown, the model maps the range of maximum and minimum values ​​to an 8-bit numerical range. For model quantization tasks, the first step is usually to statistically determine the corresponding minimum and maximum values ​​in the input data, which are the weights or activation values ​​in the image classification model. The second step is to select a suitable quantization type, including the choice of quantization bit depth and quantization method, such as symmetric or asymmetric quantization. The third step is to calculate the quantization parameter offset (Zero point) and scaling factor (Scale) based on the selected quantization type, the statistically determined minimum and maximum values, to facilitate subsequent calculations. The fourth step is to quantize the parameters in the image classification model based on the calibration data. For example, if the selected quantization bit depth is 8 bits and the quantization method is symmetric quantization, the parameters are converted from FP32 to INT8. The final step is to verify the performance of the quantized model. If the negative impact of quantization is too large, i.e., the performance degrades significantly, it is necessary to try different quantization methods to calculate the Zero point and Scale parameters, and then repeat the above operations until the model's compression ratio and performance reach a relatively balanced state.

[0076] Typically, neural network parameters are represented using 32-bit floating-point numbers. However, in actual training, it has been found that such high precision is unnecessary. For example, integers of type uint (0-255) can be used to represent the precision originally represented by 32 bits, trading a small difference in precision for space. Furthermore, SGD processes generally only require 6-8 bits of precision. Therefore, proper quantization of the network can reduce the model's storage size while maintaining accuracy.

[0077] In this implementation, the weights in the image classification model will be quantized from 32 bits to 8 bits.

[0078] First, the minimum and maximum values ​​of the weights in the image classification model are calculated.

[0079] Then the scaling factor (Scale) is calculated, specifically as follows: Where n represents the number of bits to be quantized.

[0080] Next, the zero point after quantization is calculated, specifically: Z = Q max -R max ÷S; where Q max R is the maximum value after quantization. max This is the maximum value with full precision;

[0081] The weights in the image classification model are quantized based on the quantization mapping formula r = Round(S(qZ)); where r is the floating-point weight before quantization and q is the weight after quantization.

[0082] Finally, in the actual calculation process, dequantization is performed, specifically: R = (QZ) * S; where R is the floating-point weight value after dequantization; Q is the weight value after quantization.

[0083] This implementation quantizes the image classification model after pre-training, reducing the precision of the model parameters from FP32 to INT8 while ensuring model performance. This reduces the model size to 25% of its original size with almost no loss of result accuracy.

[0084] The flowcharts for the pre-training phase of both Implementation Method 1 and Implementation Method 2 are as follows: Figure 8 As shown, Figure 8 As shown, the pre-training phase consists of three parts: the first part involves training on the support set of the subtask set in the meta-training set; the second part involves pruning the model at fixed training intervals; and the third part involves testing on the query set of the subtask set in the meta-training set, calculating the loss, and updating the model. Finally, the model is quantized using 8-bit quantization, converting the full-precision weights to the int range. By pruning and quantizing the model, the computational cost and the size of the pre-trained model are significantly reduced. The specific process is as follows: Figure 9 As shown.

[0085] Example 2

[0086] An image classification method includes: inputting an image to be classified into an image classification model trained using the training method of the meta-learning-based image classification model provided in Embodiment 1 of the present invention, and obtaining an image classification result.

[0087] The relevant technical solutions are the same as in Embodiment 1, and will not be repeated here.

[0088] Example 3

[0089] An image classification system includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the image classification method provided in Embodiment 2 of the present invention when executing the computer program.

[0090] The relevant technical solutions are the same as in Embodiment 1, and will not be repeated here.

[0091] Example 4

[0092] A computer-readable storage medium includes a stored computer program, wherein when the computer program is run by a processor, it controls the device where the storage medium is located to execute the training method of the image classification model based on meta-learning provided in Embodiment 1 of the present invention and / or the image classification method provided in Embodiment 2 of the present invention.

[0093] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for training an image classification model based on meta-learning, characterized in that, The method comprises the following steps: S1, dividing the collected image samples with classification labels into meta-training sets and meta-test sets which are different in image categories; S2, randomly sampling N1 sub-tasks from a plurality of pre-prepared sub-tasks; dividing the meta-training sets to obtain sub-task sets corresponding to the N1 sub-tasks; each sub-task set comprises a support set and a query set; one sub-task corresponds to one sub-task model; N1 is an integer greater than 1; S3, training the corresponding sub-task model by using the support set in each sub-task set under the meta-training sets; determining whether the current inner loop iteration number is 1; if yes, performing a pruning operation on the sub-task model and turning to step S4; otherwise, directly turning to step S4; S4, repeating step S3 to perform inner loop iteration until the current inner loop iteration number reaches a first preset iteration number; S5, updating parameters in the image classification model to be trained based on the average value of the classification loss values of the sub-task models; the classification loss value of each sub-task model is obtained when the query set in each sub-task set under the meta-training sets is input into the corresponding sub-task model for performance testing; S6, repeating steps S2-S5 to perform outer loop iteration until the current outer loop iteration number reaches a second preset iteration number; S7, sampling N2 sub-tasks from a plurality of pre-prepared sub-tasks; dividing the meta-test sets to obtain sub-task sets corresponding to the N2 sub-tasks; fine-tuning the image classification model by using the support set in the sub-task sets corresponding to the N2 sub-tasks to obtain a trained image classification model; N2 is an integer greater than or equal to 1; Wherein, the sub-task model and the image classification model are both neural network-based models; In step S3, the sub-task model is sequentially subjected to channel pruning and weight pruning; The sub-task model is a convolutional neural network; the method for pruning the sub-task model comprises: S31. By applying a scaling factor to the BN layer in the subtask model Norm penalty is applied to sparsify the scaling factor of the subtask model based on the BN layer during training; channels of the convolutional layer corresponding to scaling factors whose absolute value is less than the preset channel pruning threshold are removed to obtain the subtask model after channel convolution. S32, adopting weight pruning in the full connection layer, removing neuron connection weights between all neurons in two adjacent layers in the sub-task model after channel convolution of the sub-task model neuron connections with norm values less than the preset weight pruning threshold, completing the sub-task model pruning operation.

2. The method of claim 1, wherein, Further comprising: Step S8 performed between step S5 and step S6, and step S9 performed between step S6 and step S7; Step S8 comprises: performing a quantization operation on the image classification model; Step S9 comprises: performing a dequantization operation on the image classification model. 3.The method of any one of claims 1-2, wherein, After each inner loop iteration, the performance of the corresponding sub-task model is tested by using the query set in the sub-task set under the meta-training sets, and the classification loss value of the sub-task model in the corresponding inner loop iteration round is obtained; At this time, step S5 comprises: assigning different weight values to all iteration rounds of the inner loop, calculating the weighted average value of the classification loss values of the sub-task models in each inner loop iteration round, and updating the parameters in the image classification model to be trained based on the obtained weighted average value. 4.The method of claim 1 or 2, wherein, In the sub-task set, the number of image samples in the support set is less than the number of image samples in the query set. 5.The method of claim 1 or 2, wherein, The data distribution of each sub-task set is the same.

6. An image classification method characterized by, Comprising: Inputting the image to be classified into the image classification model trained by the meta-learning-based image classification model training method of any one of claims 1-5 to obtain an image classification result.

7. An image classification system, characterized by Comprising: A memory storing a computer program and a processor, wherein the processor executes the computer program to perform the image classification method in claim 6.

8. A computer readable storage medium comprising a stored computer program, wherein, The computer program is run by the processor to control a device in which the storage medium is arranged to perform the training method of the meta-learning-based image classification model in any one of claims 1-5 and / or the image classification method in claim 6.

Images