A Few-Shot Image Classification Method Based on CGAN-MAML-ResNet

Abstract

This invention relates to the field of image classification, and particularly to a few-shot image classification method based on CGAN-MAML-ResNet. The method includes: acquiring original images; constructing and training a CGAN model; generating sample images using the trained CGAN model; dividing the dataset consisting of the sample images and the original images into training and testing sets, and constructing multiple classification meta-tasks; inputting the training set into a ResNet18 model for training, obtaining the feature parameters of each layer of the ResNet18 model after training; constructing and optimizing an improved ResNet18 model to obtain the optimal parameters; and training the target classification task based on the optimal parameters to obtain the classification result. The method described in this invention has the advantages of high accuracy and strong generalization ability.

Description

Technical Field

[0001] This invention relates to the field of image classification technology, and specifically to a few-shot image classification method based on CGAN-MAML-ResNet. Background Technology

[0002] In the field of image classification, traditional deep learning models typically rely on large-scale labeled datasets for training to achieve good classification performance. This method is time-consuming, labor-intensive, and costly. Few-Shot Learning (FSL) aims to address this issue by training the model using only a small number of labeled samples to achieve efficient identification of new categories. This technique reduces the cost of data acquisition and labeling, effectively improving model performance in data-scarce or emerging fields. For example, in medical image analysis, FSL can help doctors accurately diagnose diseases even with insufficient samples. FSL methods typically include training and testing phases: during training, the model is trained on a small dataset containing multiple tasks, learning how to extract effective features from a limited number of samples; during testing, the model is validated on new category data to evaluate its performance under few-shot conditions. This process often employs deep residual network structures to enhance feature extraction capabilities and improve classification accuracy.

[0003] Currently, research on few-shot image classification mainly focuses on optimizing model architecture, improving feature representation, and refining learning strategies. One of the most cutting-edge research areas is the combination of Generative Adversarial Networks (GANs) and Meta-Learning.

[0004] While research on few-shot image classification techniques has made some progress, areas for improvement remain: in certain tasks, the quality of generated samples significantly impacts model performance, necessitating more precise generative adversarial network (GAN) design; secondly, effectively combining different meta-learning strategies to adapt to diverse application scenarios is also a challenge. Therefore, there is an urgent need for a few-shot image classification method that can complete classification tasks with high accuracy using only a small number of sample images. Summary of the Invention

[0005] The present invention aims to propose a few-sample image classification method that can complete the classification task with high accuracy using only a small number of sample images, in order to solve the above-mentioned problems.

[0006] The steps of the method described in this invention include:

[0007] S1. Obtain the original images and the dataset for the object classification task; construct and train the CGAN (Conditional Generative Adversarial Network) model, input the original images into the trained CGAN model, and generate sample images, including:

[0008] S11. Generate a random noise vector, flatten the class label image of the original sample image and concatenate it with the random noise vector, and use the resulting composite tensor as the input information of the CGAN model generator;

[0009] S12. In the downsampling path of UNet, the encoder performs downsampling operations on the input information. The input information is converted into a 4-level feature map after passing through 4 layers of round convolution-pooling.

[0010] S13. In the upsampling path of UNet, feature maps from each level are fused to obtain the generated sample image;

[0011] S14. Optimize the discriminator and generator, including:

[0012] S141. Initialize the discriminator weights and generator weights;

[0013] S142. Train the discriminator and calculate the discriminator's discrimination probability for real samples and generated samples according to the discriminator weights;

[0014] S143. Calculate the gradient of the discriminator's loss function and update the discriminator's weights accordingly;

[0015] S144. Train the generator and calculate the discriminator's discrimination probability for the generator's generated samples based on the generator weights;

[0016] S145. Calculate the gradient of the generator's loss function and update the generator's weights accordingly;

[0017] Repeat steps S142 to S145 until the discrimination probability reaches 0.5, then exit the loop.

[0018] S2. Divide the dataset consisting of sample images and original images into training set E1 and test set T1, and divide the dataset of target classification task into training set E2 and test set T2. Construct a classification meta-task for E1 and T1.

[0019] S3. Input the training set E1 into the ResNet18 model for training, and obtain the feature parameters of each layer of the ResNet18 model after training, including:

[0020] S31. Feed the generated sample and the original image into the feature extraction layer of the ResNet18 model in sequence to obtain the feature map of the generated sample and the feature map of the original image.

[0021] S32. Multiply the feature map of the generated sample and the feature map of the original image along the feature dimension to obtain the fused feature map;

[0022] S33. Input the fused feature map into the ResNet18 classification layer to obtain the classification result; calculate the loss function of the model based on the classification result; when the loss function converges, complete the training of the model and obtain the feature parameters of each layer of the ResNet18 model after training.

[0023] S4. Improve the ResNet18 model based on the feature parameters of each layer. Train the improved ResNet18 model for the first time using the MAML algorithm with a classification meta-task to obtain the optimal parameters of the improved ResNet18 model, including:

[0024] S41. Freeze the parameters of the first convolutional layer and the first three residual blocks of the ResNet18 model to obtain the improved ResNet18 model. Use the feature parameters of the ResNet18 model as the initial parameters of the improved ResNet18 model.

[0025] S42. Use the classification meta-task to update the inner layers of the improved ResNet18 model;

[0026] S43. Use the classification meta-task to update the outer layer of the improved ResNet18 model, and update the global parameters of the model based on the inner and outer layer update results;

[0027] S44. Obtain the optimal parameters of the improved ResNet18 model after inner and outer layer updates;

[0028] S5. The improved ResNet18 model is trained a second time using the training set E2, and the optimal parameters from the first training are used as the initial parameters for the second training. The improved ResNet18 model after the second training is tested using T2 to obtain the classification results of the target classification task.

[0029] The beneficial effects of this invention are as follows:

[0030] This patent effectively addresses the problems of insufficient data and weak model generalization in few-sample image classification tasks by generating sample images using the CGAN model. By employing an improved ResNet18 model for feature extraction, it reduces the number of training parameters, lowers memory usage during computation, and improves extraction speed. Furthermore, by using the MAML algorithm to train the improved ResNet18 model, it effectively solves the problem of the model easily getting trapped in local optima when sample data is insufficient, and enables the model to quickly adapt to different types of data, thus improving its generalization ability. Attached Figure Description

[0031] Figure 1 This is a framework diagram of the method described in this invention;

[0032] Figure 2 This is a structural diagram of the CGAN model described in this invention;

[0033] Figure 3 This is a structural diagram of the ResNet18 model described in this invention;

[0034] Figure 4 This is a schematic diagram illustrating the first training of the improved ResNet18 model using a classification meta-task as described in this invention. Detailed Implementation

[0035] To make the objectives, technical solutions, features, and advantages of the present invention clearer, the present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0036] The framework diagram of the method described in this invention is as follows: Figure 1 As shown, the specific steps are as follows:

[0037] S1. Obtain the original images and the dataset for the object classification task; construct and train the CGAN model, input the original images into the trained CGAN model, and generate sample images:

[0038] Specifically, the generator of the conditional generative adversarial network adopts the UNet network architecture, and the discriminator adopts the PatchGAN network architecture. The structure diagram of the CGAN model is shown below. Figure 2 As shown.

[0039] S11. Generate a random noise vector, flatten the class label image of the original sample image and concatenate it with the random noise vector, and use the resulting composite tensor as the input information of the CGAN model generator;

[0040] Furthermore, the category label image of the sample image, i.e., the two-dimensional matrix of pixel values, is first used as conditional information. Then, random noise is generated, the label information is flattened and concatenated with the noise vector, and the composite tensor containing the conditional information and random noise is used as the input information of the generator.

[0041] S12. In the downsampling path of UNet, the encoder performs downsampling operations on the input information. The input information is converted into a 4-level feature map after passing through 4 layers of round convolution-pooling.

[0042] Specifically, the encoder in UNet's downsampling path consists of convolutional layers, a normalization function, and a ReLU activation function.

[0043] S13. In the upsampling path of UNet, feature maps from each level are fused to obtain the generated sample image;

[0044] Specifically, in the upsampling path of UNet, the decoder consists of transposed convolutional layers, a normalization function, and a ReLU activation function.

[0045] S14. Optimize the discriminator and generator, including:

[0046] S141. Initialize the discriminator weights and generator weights;

[0047] S142. Train the discriminator and calculate the discriminator's discrimination probability for real samples and generated samples according to the discriminator weights;

[0048] Specifically, [X] i ,T i ] and [X i ,S i Two sets of images are input into the discriminator, and the discriminator's discrimination probability is calculated to see if it is close to 1; the generated fake samples and their conditional information are input into the discriminator, and the discriminator's discrimination probability is calculated to see if it is close to 0; the expression for the discrimination probability is:

[0049]

[0050] Where P represents the discrimination probability, and N pix X represents the number of pixels in each image. i T i S i These represent the label image of the original sample image, the original sample image, and the image generated by the generator, respectively.

[0051] S143. Calculate the gradient of the discriminator's loss function and update the discriminator's weights accordingly;

[0052] Specifically, the loss function D of the discriminator is calculated. lossThe discriminator's weights are updated based on the gradient of the loss function. The discriminator's loss function D... loss The expression is:

[0053] D loss =Dr loss +D floss

[0054]

[0055] Among them, D rloss This indicates the use of labeled sample T. i True discriminator loss, D floss This indicates that the sample S generated using the generator is... i The pseudo-discriminator loss is given by D, where D represents the discriminator, and log(D[x]) represents the logarithmic probability that the discriminator D classifies x as a real image. i ,S i ]) represents the logarithmic value of the probability that the discriminator D classifies x as a fake image.

[0056] S144. Train the generator and calculate the discriminator's discrimination probability for the generator's generated samples based on the generator weights;

[0057] Specifically, in order for the discriminator to classify fake samples as real, the image category labels of the generated samples and the original samples are input into the discriminator, and the discriminator's discrimination probability P for these fake samples generated by the generator is calculated.

[0058] S145. Calculate the gradient of the generator's loss function and update the generator's weights accordingly;

[0059] Specifically, the loss function G of the generator is calculated. loss The generator's weights are updated based on the gradient of the loss function, where the generator's loss function G is... loss The expression is:

[0060] G loss =L bce +λ·P loss

[0061]

[0062] Among them, L bce This represents the binary cross-entropy loss, where λ represents the weighting factor, controlling the pixel-level loss relative to L. bce The impact of P loss N represents pixel-level loss. S D(T) represents the number of samples in the training batch. i ) and D(S i ) represent the discriminator D on T respectively. iand S i The prediction, l r A binary label that distinguishes between real and fake samples.

[0063] Furthermore, l r =1 indicates that the data sample is real, l r ×log(D(T i )) indicates a high probability that the discriminator will output the true image, log(D(T) i The )) term penalizes the model when the probability of the model predicting the true image is low; r =0 indicates that the data sample is false, (1-l r )×log(1-D(S i )) represents the low probability that the discriminator is encouraged to output a fake image, log(1-D(S) i The )) item penalizes the model when the probability of the model predicting a fake image is high.

[0064] The loop continues from S142 to S145 until the discrimination probability reaches 0.5, at which point the discriminator can no longer determine whether the generated image is real or fake, and then exits the loop.

[0065] Specifically, the discriminant and generator are judged by the numerical value of the discrimination probability. Reaching Nash equilibrium means that the discriminant can no longer distinguish between real and fake generated samples.

[0066] S2. Divide the dataset consisting of sample images and original images into training set E1 and test set T1, and divide the dataset of target classification task into training set E2 and test set T2. Construct a classification meta-task for E1 and T1.

[0067] Specifically, each meta-task has its own dedicated training set (support set) and test set (query set).

[0068] S3. Input the training set E1 into the ResNet18 model for training, and obtain the feature parameters of each layer of the ResNet18 model after training.

[0069] The structure diagram of the ResNet18 model is as follows: Figure 3 As shown, feature extraction from the samples in the training set further includes:

[0070] S31. Feed the generated sample and the original image into the feature extraction layer of the ResNet18 model in sequence to obtain the feature map of the generated sample and the feature map of the original image.

[0071] Specifically, the feature extraction layer includes a 7×7 convolutional layer and four residual blocks.

[0072] S32. Multiply the feature map of the generated sample and the feature map of the original image along the feature dimension to obtain the fused feature map.

[0073] S33. Input the fused feature map into the ResNet18 classification layer to obtain the classification result; calculate the loss function of the model based on the classification result; when the loss function converges, complete the training of the model and obtain the feature parameters of each layer of the ResNet18 model after training.

[0074] Specifically, the classification layer includes a maximum pooling layer and three fully connected layers, with the number of neurons in the fully connected layers being 1024, 512, and 5, respectively.

[0075] S4. Improve the ResNet18 model based on the feature parameters of each layer. Train the improved ResNet18 model for the first time using the MAML algorithm and a classification meta-task to obtain the optimal parameters of the improved ResNet18 model. Further:

[0076] S41. Freeze the parameters of the first convolutional layer and the first three residual blocks of the ResNet18 model to obtain the improved ResNet18 model. Use the feature parameters of the ResNet18 model as the initial parameters of the improved ResNet18 model.

[0077] S42. Use the classification meta-task to update the inner layers of the improved ResNet18 model;

[0078] Specifically, the support set E1 from the meta-task is obtained, the samples in the support set are input into the classification model to obtain the classification result, and the loss value of the model is calculated using the loss function. The inner layer updates the loss function. The expression is:

[0079]

[0080] Where N represents the total number of samples in the support set, x i Let represent the i-th input image in the support set, θ represent the trainable parameters of the network, and y i x represents i The corresponding category label, f(θ,x) i ) represents the network output, L MSE (y i ,f(θ,x i ) represents the mean squared error between the model output and the sample labels in the support set. The formula for calculating the mean squared error is:

[0081]

[0082] Calculate the descent gradient values ​​of the parameters in the few-shot classification model to obtain the new parameters of the few-shot classification model, completing the inner layer update; the expression for the inner loop parameter gradient update is:

[0083]

[0084] Where, θ ′ Let θ represent the gradient after the update, α represent the gradient before the update, and α represent the inner layer's learning rate. Indicates the loss function The gradient is obtained by taking the derivative with respect to the parameter θ.

[0085] S43. Use the classification meta-task to update the outer layer of the improved ResNet18 model, and update the global parameters of the model based on the inner and outer layer update results;

[0086] Specifically, the model parameters after the inner layer update are used as initial parameters, the query set T1 is obtained, and the samples from the query set are input into the model, using the loss function. The formula for calculating the loss value of the model is as follows:

[0087]

[0088] in, Let x represent the loss value of the model's internal update on the query set in the j-th task. k Let y represent the k-th input image in the support set, C represent the total number of samples in the query set, and y represent the k-th input image in the support set. k x represents k The corresponding category label, M, represents the number of tasks used to perform outer layer updates.

[0089] The network parameters are updated using the gradient descent method, which is expressed as follows:

[0090]

[0091] Where, θ ′ This represents the updated gradient, and β represents the learning rate when performing the outer update. Indicates passage The gradient is obtained by taking the derivative with respect to θ.

[0092] The Adam optimizer is used to optimize the few-shot classification model, completing the outer layer update.

[0093] S44. Obtain the optimal parameters of the improved ResNet18 model after inner and outer layer updates.

[0094] S5. The improved ResNet18 model is trained a second time using the training set E2, and the optimal parameters from the first training are used as the initial parameters for the second training. The improved ResNet18 model after the second training is tested using T2 to obtain the classification results of the target classification task.

[0095] Specifically, the obtained optimal parameters are used as initial parameters and passed to the support set of the target classification task for rapid training. This allows the model to achieve good performance on a small dataset of the target classification task with only one or two gradient updates, resulting in good classification results. This yields a few-shot classification model for the target classification task. The model is then tested on the query set of a new task to obtain the classification result for the target task. The loss function L for the few-shot classification model of the new task is... NEW The expression is:

[0096]

[0097] Where D represents the number of training samples in a batch, x W Let θ represent the W-th training sample, and y represent the parameters of the classification model. W x represents W The corresponding label, f(θ,x) W ) represents the output of the classification model, L MSE (f(θ,x W ),y W The value represents the mean squared error between the model output and the label.

[0098] Finally, it should be noted that the above description is only one embodiment of the present invention. For those skilled in the art, various changes, modifications, substitutions and variations can be conceived of these embodiments without departing from the principles and spirit of the present invention. The scope of protection of the present invention is defined by the appended claims and their equivalents, and all the above-mentioned behaviors should be covered within the scope of protection of the present invention.

Claims

1. A few-shot image classification method based on CGAN-MAML-ResNet, characterized in that, include: S1. Obtain the original images and the dataset for the object classification task; construct and train the CGAN model, input the original images into the trained CGAN model, and generate sample images; S2. Divide the dataset consisting of sample images and original images into training set E1 and test set T1, and divide the dataset of target classification task into training set E2 and test set T2. Construct classification meta-tasks for E1 and T1. S3. Input the training set E1 into the ResNet18 model for training, and obtain the feature parameters of each layer of the ResNet18 model after training; The ResNet18 model includes: a 7*7 convolutional layer, residual block 0, residual block 1, residual block 2, residual block 3, a 3*3 max pooling layer, a 1024 fully connected layer, a 512 fully connected layer, and a 5-connected layer, with the modules connected sequentially. Feature extraction of samples from training set E1 based on the ResNet18 model includes: S31. Feed the generated sample and the original image into the feature extraction layer of the ResNet18 model in sequence to obtain the feature map of the generated sample and the feature map of the original image. S32. Multiply the feature map of the generated sample and the feature map of the original image along the feature dimension to obtain the fused feature map; S33. Input the fused feature map into the ResNet18 classification layer to obtain the classification result; calculate the loss function of the model based on the classification result; when the loss function converges, complete the training of the model and obtain the feature parameters of each layer of the ResNet18 model after training. S4. Improve the ResNet18 model based on the feature parameters of each layer. Train the improved ResNet18 model for the first time using the MAML algorithm and a classification meta-task to obtain the optimal parameters of the improved ResNet18 model. The improvement of the ResNet18 model includes: freezing the 7*7 convolutional layer, residual block 0, residual block 1, and residual block 2 of the ResNet18 model; connecting the remaining unfrozen layers to form the improved ResNet18 model; and using the feature parameters of the ResNet18 model as the initial parameters of the improved ResNet18 model. The improved ResNet18 model includes, in sequence: a residual block, a 3*3 max-pooling layer, a 1024-size fully connected layer, a 512-size fully connected layer, and a 5-size fully connected layer. S5. The improved ResNet18 model is trained a second time using the training set E2, and the optimal parameters from the first training are used as the initial parameters for the second training. The improved ResNet18 model after the second training is tested using T2 to obtain the classification results of the target classification task.

2. The few-shot image classification method based on CGAN-MAML-ResNet according to claim 1, characterized in that, The CGAN model consists of a generator and a discriminator, with the generator using the UNet network architecture and the discriminator using the PatchGAN network architecture.

3. The few-shot image classification method based on CGAN-MAML-ResNet according to claim 2, characterized in that, The steps to build a CGAN model include: S11. Generate a random noise vector, flatten the class label image of the original sample image and concatenate it with the random noise vector, and use the resulting composite tensor as the input information of the CGAN model generator; S12. In the downsampling path of UNet, the encoder performs downsampling operations on the input information, which is then converted into a 4-level feature map after 4 layers of round convolution-pooling. S13. Perform upsampling on the 4-level feature maps, and fuse the upsampled feature maps of each level to obtain the generated sample image.

4. The few-shot image classification method based on CGAN-MAML-ResNet according to claim 1, characterized in that, When training the CGAN model, the discriminator and generator are optimized. The process of optimizing the discriminator and generator includes: Step 1. Initialize the discriminator weights and generator weights; Step 2. Train the discriminator and calculate the discriminator's discrimination probability for real samples and generated samples based on the discriminator weights; Step 3. Calculate the gradient of the discriminator's loss function and update the discriminator's weights accordingly; Step 4. Train the generator and calculate the discriminator's discrimination probability for the generator's generated samples based on the generator weights; Step 5. Calculate the gradient of the generator's loss function and update the generator's weights accordingly; Step 6. Repeat steps 2-5 until the discrimination probability is 0.5, then exit the loop.

5. The few-shot image classification method based on CGAN-MAML-ResNet according to claim 1, characterized in that, The first training and testing of the improved ResNet18 model using the classification meta-task includes: updating the inner layers of the improved ResNet18 model using the classification meta-task; updating the outer layers of the improved ResNet18 model using the classification meta-task; and updating the model's global parameters based on the inner and outer layer update results.

Images