Image classification method based on few-shot learning

Abstract

The application discloses a kind of based on few-sample learning's image classification method, including obtaining existing training dataset;Classify task model is constructed;Loss function is set and the classification task model is trained, and initial teacher model is obtained;Student model is constructed with the same structure as current teacher model and randomly initializes parameter;Student model is trained and the student model after training is used as current teacher model;Repeat the previous step until the set number of times, obtain feature extractor;Again obtain training dataset;Image classification initial model is constructed and image classification terminal model is obtained by training;Prune image classification terminal model obtains image classification model, and image classification based on few-sample learning is carried out using image classification model.The application can learn more transferable knowledge by the design and training of teacher model and student model;Image is learned by image classification model, and the support-query offset problem is solved;Therefore, the application has high reliability and good accuracy.

Description

Technical Field

[0001] This invention belongs to the field of image processing, specifically relating to an image classification method based on few-shot learning. Background Technology

[0002] Image classification technology plays a vital role in people's production and daily lives, such as facial recognition, automatically organizing photo albums, and identifying plant and animal species. Therefore, the accuracy and reliability of image classification schemes are of paramount importance.

[0003] Currently, traditional image classification schemes require a large number of labeled samples to train the image classification model so that it can learn to recognize different image categories. However, in the real world, obtaining a large number of labeled samples is extremely costly and difficult, especially for images in specific domains or rare categories. Therefore, image classification schemes based on few-shot learning have become a key research focus for researchers.

[0004] In few-shot learning, image classification models are first trained on a large dataset containing hundreds of thousands of images to acquire basic image feature extraction capabilities. Then, on a completely new dataset, they learn from only a small number of images for each category, thus achieving accurate classification of the new dataset. However, there is a bias in the data distribution between different images of the same category, known as the support-query shift problem. This problem prevents the image classification model from effectively mastering the general features and knowledge of different categories of images in the new dataset, thereby affecting the performance of the image classification model.

[0005] Currently, mainstream image classification schemes based on few-shot learning are all based on meta-learning. The meta-learning process typically includes two stages: meta-training and meta-testing. In the meta-training stage, the image classification model learns from a small subset of images in a large dataset to simulate few-shot scenarios, aiming to quickly adapt to new scenarios and tasks by utilizing transferable knowledge learned in the pre-training stage. In the meta-testing stage, the image classification model uses images from the large dataset that were not used for meta-training to test the model's performance and capabilities on new tasks.

[0006] However, current image classification schemes based on few-shot learning using meta-learning methods cannot effectively solve the support-query offset problem in the few-shot learning process, and their training process also has some problems. This makes the existing image classification schemes based on few-shot learning have poor reliability and accuracy. Summary of the Invention

[0007] The purpose of this invention is to provide an image classification method based on few-shot learning that can solve the support-query offset problem and has high reliability and accuracy.

[0008] The image classification method based on few-shot learning provided by this invention includes the following steps: S1. Obtain the existing training dataset; S2. Construct a classification task model based on a feature extractor and a first classifier; the feature extractor is used to extract features from the input image, and the first classifier is used to classify the input image according to the extracted features; S3. Using the training dataset obtained in step S1, set the loss function and train the classification task model constructed in step S2, and use the trained model as the initial teacher model. S4. Construct a student model with the same structure as the current teacher model, and randomly initialize the parameters of the student model; using the training dataset obtained in step S1, train the student model based on the loss function set in step S3 and the structural loss function between the student model and the teacher model, and use the trained student model as the current teacher model. S5. Repeat step S4 until the set number of times; obtain the feature extractor from the student model obtained from the last training. S6. Obtain the existing training dataset again; S7. Construct an initial image classification model including a feature processor, a feature modifier, a task adapter, and the feature extractor obtained in step S5; wherein the feature extractor is used to extract features from the input image, the feature processor is used to preprocess the extracted features, the task adapter is used to train the initial image classification model, and the feature modifier is used to repair the preprocessed features of the input and output the final classification result. S8. Using the training dataset obtained in step S6, train the initial image classification model constructed in step S7 to obtain the image classification terminal model; S9. Prune the image classification terminal model obtained in step S8 to obtain an image classification model, and use the obtained image classification model to perform image classification based on few-shot learning.

[0009] Step S2 specifically includes the following steps: The classification task model consists of a concatenated feature extractor and a first classifier; The feature extractor uses a ResNet12 network, comprising a first residual convolutional module, a second residual convolutional module, a third residual convolutional module, and a fourth residual convolutional module connected in series. The first, second, third, and fourth residual convolutional modules have identical structures. In the residual convolutional modules, the input feature data is divided into two paths. The first path of feature data is processed sequentially through the first, second, and third convolutional sub-modules to obtain a first sub-feature map. The second path of feature data is processed through a downsampling sub-module to obtain a second sub-feature map. The first and second sub-feature maps are then added bit-by-bit as the output of the residual convolutional module. The first convolutional sub-module includes a... A convolutional layer, a batch normalization layer, and a ReLU activation function layer; the second and third convolutional submodules each include a... A convolutional layer and a batch normalization layer; the downsampling submodule includes a A convolutional layer and a ReLU activation function layer; The first classifier is a fully connected layer network; The feature extractor is used to extract features from the input image; the first classifier is used to classify the input image based on the extracted features.

[0010] Step S3 specifically includes the following steps: In the training dataset obtained in step S1, the labels of the images are obtained and converted into 0-1 labels; Input the images in the training dataset into the classification task model constructed in step S2 to obtain the corresponding prediction results; Based on the obtained prediction results and the corresponding 0-1 labels, the first loss is calculated using the following formula. : In the formula N The total number of input images; For the first ii Does Zhang Tu belong to the [number]? jj A binary variable of class i, if the i-th ii Zhang Tu belongs to the first jj Each category ,otherwise ; For the first ii The prediction result of the first image jj A number; Based on the obtained prediction results and the corresponding 0-1 labels, the second loss is calculated using the following formula. : In the formula For the first ii Prediction results for the images; The feature center is the image category to which the input image belongs; the feature center is the average of the feature values ​​of all images in the corresponding category. This represents the L2 normal form of the matrix; The classification task model constructed in step S2 is trained based on the first loss and the second loss, and the trained model is used as the initial teacher model.

[0011] Step S4 specifically includes the following steps: The parameters of the current teacher model will not be modified. Construct a student model with the exact same structure as the current teacher model, and randomly initialize the parameters of the student model; The image is simultaneously input into both the current teacher model and the current student model; Based on the prediction results of the current teacher model and the 0-1 labels corresponding to the input image, calculate the first loss and the second loss of the current student model; Based on the prediction results of the current student model and the current teacher model, the structural loss of the current student model is calculated using the following formula. : In the formula This is the function for calculating cosine similarity. Let be the feature distance matrix of the current teacher model, where the _i_ is the _i_th ... i Line 1, Page j Elements in column 1 for , For the teacher model to the first i Prediction results for one image For the teacher model to the first j Prediction results for a single image; Let be the feature distance matrix of the current student model, where the _i ... i 2nd line j Elements in 2 columns for , For the student model to the first i Prediction results for 2 images For the student model to the first j Prediction results for two images; Let be the characteristic angle relation matrix of the current teacher model, where the _i_ is the _i_th ... i 3rd line j Elements in 3 columns for , For the teacher model to the first i Prediction results for 3 images, For the teacher model to the first jPrediction results for 3 images; Let be the characteristic angle relation matrix of the current student model, where the _i_ is the _i_th ... i 4th line j Elements in 4 columns for , For the student model to the first i Prediction results for 4 images For the student model to the first j Prediction results for 4 images; Using the training dataset obtained in step S1, the student model is trained based on the obtained first loss, second loss, and structural loss, and the trained student model is used as the current teacher model.

[0012] Step S6 specifically includes the following steps: Obtain the existing training dataset again; In the acquired training dataset, a number of images are randomly selected to construct the support set and query set. The random selection rule is as follows: N categories are randomly selected, and KK images are randomly selected for each category to construct the support set, and QQ images are randomly selected for each category to construct the query set. When inputting a model, it will support inputting both the set and the query set together.

[0013] Step S7 specifically includes the following steps: The constructed initial image classification model includes the feature extractor obtained in step S5, the newly constructed feature processor, the newly constructed feature modifier, and the newly constructed task adapter; The feature extractor is used to extract features from the input image and input the extracted feature image into the feature processor; the input image includes a support set image and a query set image; the extracted feature image includes a support set feature image and a query set feature image. The feature processor is used to perform preprocessing operations on the received feature image to obtain preprocessed features, and simultaneously inputs the preprocessed features into the feature corrector and the task adapter; the preprocessing operations include coarsening and scrambling operations. A task adapter is constructed based on convolution, instance attention, and spatial attention; the task adapter is used to train the initial image classification model based on the information entropy of the preprocessed features. A feature modifier is constructed based on spatial attention. The feature modifier is used to repair the preprocessed features based on their similarity. Then, the image classification initial model is trained based on the information entropy of the preprocessed features. Finally, the repaired features are used to classify the input image through a second classifier.

[0014] The feature processor specifically includes the following: Roughing operation: The input support set feature image is fed into the erase function; the erase function randomly generates a set number of random numbers, maps all random numbers back to the feature image coordinates, and uses the selected numbers as the pixels to be erased; for the erased pixels, the values ​​of the pixels in all channels are set to 0; after the erasure of all selected pixels is completed, the preprocessed support set feature image is obtained. Disrupt the operation: The input query set feature image is used as the object of the shuffling operation; The process of generating the shuffled images: Based on the number of images in each batch, generate an integer array of the same length as the number of images. This integer array includes the range... All certificates within the array, and the numbers in the array are in random order; each image uses the corresponding data in the integer array as the same area of ​​the image received by its own shuffled area; The process of generating a scrambled region: generating a region in The random parameter of the region is used as the percentage of the image scrambled area; two integers are randomly selected. xx and yy , xx Smaller than the image length yy Smaller than the image width, and coordinates As the center of the scrambled region; before generating the scrambled region, the coordinates of the upper left and lower right corners of the scrambled region are calculated and limited to the image range; finally, based on the coordinates... The final scrambled area is defined by the defined upper left and lower right coordinates. For each image, in a shuffled order, receive the shuffled region information of the corresponding image and overwrite the original information of the image itself; copy the label of each image, starting from a length of... One-dimensional tensor expansion to The three-dimensional tensor, in which The number of categories in the training set. H For the height of the feature image, W The width of the feature image; after expansion, the labels are processed according to the scrambling order and scrambling area to complete the information exchange between different labels: similar to the image scrambling method, each label receives the information of the scrambling area of ​​the corresponding label according to the scrambling order and overwrites its own original information. Finally, the preprocessed query set feature image is obtained.

[0015] The aforementioned task adapter specifically includes the following: The task adapter includes a projection head module, a first average pooling module, an instance attention module, a spatial attention module, and a second average pooling module. The projection head submodule comprises four convolutional submodules connected in series, each convolutional submodule including one... Convolutional layers and ReLU activation functions are used; the preprocessed query set feature image and the preprocessed support set feature image are input into the projection head submodule for processing. The query set features processed by the projection head submodule are processed by the first average pooling module to obtain the query set feature vector. The support set features processed by the projection head submodule are divided into three paths: the first path is processed by the instance attention module, the second path is processed by the spatial attention module, and the third path is used for direct calculation. The instance attention module consists of sequentially connected... The system includes a convolution submodule, an average pooling submodule, a ReLU activation function submodule, and a softmax activation function submodule. The first support set features are processed by the instance attention module and then multiplied with the third support set features to obtain a matrix image, thereby assigning weights to each support set feature. The spatial attention module consists of sequentially connected components. The convolutional submodule, the ReLU activation function submodule, and the softmax activation function submodule are used to process the second support set features through the spatial attention module. The result is then copied C times in the second dimension to obtain the attention image. The attention image is then multiplied element-wise with the obtained matrix image to obtain the element image, thereby assigning weights to each element. The element images are then processed by the second average pooling module to obtain the support set feature vectors; Based on the obtained query set feature vector and support set feature vector, the first information entropy is calculated using the following formula. : In the formula For the first k Zhang queries the categories of the image collection; For the first l Zhang supports the categories of images in the set; For the first k Features of the Zhang query set image after processing by the task adapter; For the first n Zhang supports the features of the image after processing by the task adapter.

[0016] The feature modifier specifically includes the following: The input to the feature modifier is the preprocessed query set feature image and the preprocessed support set feature image; First, the preprocessed support set feature image is repaired using the preprocessed query set feature image. Then, the repaired support set feature image is used to repair the preprocessed query set feature image, resulting in the repaired query set feature image. The total number of feature images in the query set after preprocessing is The total number of support set feature images after preprocessing is ,in , ; Each channel of each image is converted into a one-dimensional tensor. The preprocessed query set feature image and the preprocessed support set feature image are then multiplied pairwise, and spatial attention is calculated to obtain a similarity matrix R, with the shape of R as follows: ; For each preprocessed support set feature image, the correction process is expressed by the following formula: In the formula For the repaired support set i Features of the image; This represents the j-th row of the similarity matrix R; Features of the i-th image in the support set before repair; Will The repaired features are stacked to obtain the corrected support set features; Transpose the similarity matrix R; the transposed shape is: ; For each preprocessed query set feature image, the correction process is expressed by the following formula: In the formula The th similarity matrix R l Row data; The features of the k-th image in the query set before repair are obtained; after repair, the features of the query set after repair are obtained. Based on the revised support set features and the revised query set features, the second information entropy is calculated using the following formula. : In the formula Let m be the category of the query set image; Let n be the category of the nth support set image; The features of the m-th query set image after processing by the feature correction module; The features of the nth support set image after processing by the feature correction module; The corrected support set features and query set features are input into the overall classifier to obtain the final classification result; the overall classifier is a single classifier. Convolutional networks.

[0017] The training described in step S8 specifically includes the following steps: The overall loss function L is calculated using the following formula: In the formula The classification loss is the classification result output by the feature modifier and the corresponding label. This is the first weight value; This is the second weight value; Based on the calculated overall loss function, the backpropagation algorithm is used to train and optimize the model.

[0018] Step S9 specifically includes the following steps: Obtain the image classification terminal model obtained in step S8; In the image classification terminal model, the feature processor and task adapter are removed, and the output of the feature extractor is directly used as the input of the feature modifier, thus obtaining the image classification model; The obtained image classification model is then used for image classification based on few-shot learning.

[0019] The image classification method based on few-shot learning provided by this invention enables the model to learn more transferable knowledge through the design and training of teacher and student models in the first stage; then, by constructing an image classification model to learn and train on various types of images simultaneously, the support-query offset problem is solved; therefore, the method of this invention has high reliability and good accuracy. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the method flow of the present invention. Detailed Implementation

[0021] like Figure 1 The diagram shown is a flowchart of the method of the present invention: This image classification method based on few-shot learning disclosed in the present invention includes the following steps: S1. Obtain the existing training dataset; S2. Construct a classification task model based on a feature extractor and a first classifier; the feature extractor is used to extract features from the input image, and the first classifier is used to classify the input image according to the extracted features; specifically, it includes the following steps: The classification task model consists of a concatenated feature extractor and a first classifier; The feature extractor uses a ResNet12 network, comprising a first residual convolutional module, a second residual convolutional module, a third residual convolutional module, and a fourth residual convolutional module connected in series. The first, second, third, and fourth residual convolutional modules have identical structures. In the residual convolutional modules, the input feature data is split into two paths (i.e., it is copied to obtain two sets of input data). The first set of feature data is processed sequentially through the first, second, and third convolutional sub-modules to obtain a first sub-feature map. The second set of feature data is processed through a downsampling sub-module to obtain a second sub-feature map. The first and second sub-feature maps are added bit-by-bit as the output of the residual convolutional module. The first convolutional sub-module includes a... A convolutional layer, a batch normalization layer, and a ReLU activation function layer; the second and third convolutional submodules each include a... A convolutional layer and a batch normalization layer; the downsampling submodule includes a A convolutional layer and a ReLU activation function layer; The first classifier is a fully connected layer network; The feature extractor is used to extract features from the input image; the first classifier is used to classify the input image based on the extracted features. S3. Using the training dataset obtained in step S1, set the loss function and train the classification task model constructed in step S2, then use the trained model as the initial teacher model; specifically including the following steps: In the training dataset obtained in step S1, the labels of the images are obtained and converted into 0-1 labels; Input the images in the training dataset into the classification task model constructed in step S2 to obtain the corresponding prediction results; Based on the obtained prediction results and the corresponding 0-1 labels, the first loss is calculated using the following formula. : In the formula N The total number of input images; For the first ii Does Zhang Tu belong to the [number]? jj A binary variable of class i, if the i-th ii Zhang Tu belongs to the first jj Each category ,otherwise ; For the first ii The prediction result of the first image jj A number; Based on the obtained prediction results and the corresponding 0-1 labels, the second loss is calculated using the following formula. : In the formula For the first ii Prediction results for the images; The feature center is the image category to which the input image belongs; the feature center is the average of the feature values ​​of all images in the corresponding category. This represents the L2 normal form of the matrix; Based on the first loss and the second loss, the classification task model constructed in step S2 is trained, and the trained model is used as the initial teacher model. S4. Construct a student model with the same structure as the current teacher model, and randomly initialize the parameters of the student model; using the training dataset obtained in step S1, train the student model based on the loss function set in step S3 and the structural loss function between the student model and the teacher model, and use the trained student model as the current teacher model; specifically including the following steps: The parameters of the current teacher model will not be modified. Construct a student model with the exact same structure as the current teacher model, and randomly initialize the parameters of the student model; The image is simultaneously input into both the current teacher model and the current student model; Based on the prediction results of the current teacher model and the 0-1 labels corresponding to the input image, calculate the first loss and the second loss of the current student model; Based on the prediction results of the current student model and the current teacher model, the structural loss of the current student model is calculated using the following formula. : In the formula This is the function for calculating cosine similarity. Let be the feature distance matrix of the current teacher model, where the _i_ is the _i_th ... i Line 1, Page j Elements in column 1 for , For the teacher model to the first i Prediction results for one image For the teacher model to the first j Prediction results for a single image; Let be the feature distance matrix of the current student model, where the _i ... i 2nd line j Elements in 2 columns for , For the student model to the first i Prediction results for 2 images For the student model to the first j Prediction results for two images; Let be the characteristic angle relation matrix of the current teacher model, where the _i_ is the _i_th ... i 3rd line j Elements in 3 columns for , For the teacher model to the first i Prediction results for 3 images, For the teacher model to the first j Prediction results for 3 images; Let be the characteristic angle relation matrix of the current student model, where the _i_ is the _i_th ... i 4th line j Elements in 4 columns for , For the student model to the first i Prediction results for 4 images For the student model to the first j Prediction results for 4 images; Using the training dataset obtained in step S1, the student model is trained based on the obtained first loss, second loss, and structural loss, and the trained student model is used as the current teacher model. S5. Repeat step S4 until the set number of times; obtain the feature extractor from the student model obtained from the last training. S6. Obtain the existing training dataset again; specifically, this includes the following steps: Obtain the existing training dataset again; In the acquired training dataset, a number of images are randomly selected to construct the support set and query set. The random selection rule is as follows: N categories are randomly selected, and KK images are randomly selected for each category to construct the support set, and QQ images are randomly selected for each category to construct the query set. When inputting a model, it will support inputting both the set and the query set together. S7. Construct an initial image classification model including a feature processor, a feature modifier, a task adapter, and the feature extractor obtained in step S5; wherein the feature extractor is used to extract features from the input image, the feature processor is used to preprocess the extracted features, the task adapter is used to train the initial image classification model, and the feature modifier is used to repair the preprocessed input features and output the final classification result; specifically including the following steps: The constructed initial image classification model includes the feature extractor obtained in step S5, the newly constructed feature processor, the newly constructed feature modifier, and the newly constructed task adapter; The feature extractor is used to extract features from the input image and input the extracted feature image into the feature processor; the input image includes a support set image and a query set image; the extracted feature image includes a support set feature image and a query set feature image. The feature processor performs preprocessing operations on the received feature image to obtain preprocessed features, and simultaneously inputs the preprocessed features into the feature corrector and the task adapter. The preprocessing operations include coarsening and scrambling operations; specifically, they include the following: Roughing operation: The input support set feature image is fed into the erase function; the erase function randomly generates a set number of random numbers, maps all random numbers back to the feature image coordinates, and uses the selected numbers as the pixels to be erased; for the erased pixels, the values ​​of the pixels in all channels are set to 0; after the erasure of all selected pixels is completed, the preprocessed support set feature image is obtained. Disrupt the operation: The input query set feature image is used as the object of the shuffling operation; The process of generating the shuffled images: Based on the number of images in each batch, generate an integer array of the same length as the number of images. This integer array includes the range... All certificates within the array, and the numbers in the array are in random order; each image uses the corresponding data in the integer array as the same area of ​​the image received by its own shuffled area; The process of generating a scrambled region: generating a region in The random parameter of the region is used as the percentage of the image scrambled area; two integers are randomly selected. xx and yy , xx Smaller than the image length yy Smaller than the image width, and coordinates As the center of the scrambled region; before generating the scrambled region, the coordinates of the upper left and lower right corners of the scrambled region are calculated and limited to the image range; finally, based on the coordinates... The final scrambled area is defined by the defined upper left and lower right coordinates. For each image, in a shuffled order, receive the shuffled region information of the corresponding image and overwrite the original information of the image itself; copy the label of each image, starting from a length of... One-dimensional tensor expansion to The three-dimensional tensor, in which The number of categories in the training set. H For the height of the feature image, WThe width of the feature image; after expansion, the labels are processed according to the scrambling order and scrambling area to complete the information exchange between different labels: similar to the image scrambling method, each label receives the information of the scrambling area of ​​the corresponding label to cover its original information according to the scrambling order. Finally, the preprocessed query set feature image is obtained; A task adapter is constructed based on convolution, instance attention, and spatial attention. This task adapter is used to train the initial image classification model based on the information entropy of the preprocessed features. Specifically, it includes the following: The task adapter includes a projection head module, a first average pooling module, an instance attention module, a spatial attention module, and a second average pooling module. The projection head submodule comprises four convolutional submodules connected in series, each convolutional submodule including one... Convolutional layers and ReLU activation functions are used; the preprocessed query set feature image and the preprocessed support set feature image are input into the projection head submodule for processing. The query set features processed by the projection head submodule are processed by the first average pooling module to obtain the query set feature vector. The support set features processed by the projection head submodule are divided into three paths: the first path is processed by the instance attention module, the second path is processed by the spatial attention module, and the third path is used for direct calculation. The instance attention module consists of sequentially connected... The system includes a convolution submodule, an average pooling submodule, a ReLU activation function submodule, and a softmax activation function submodule. The first support set features are processed by the instance attention module and then multiplied with the third support set features to obtain a matrix image, thereby assigning weights to each support set feature. The spatial attention module consists of sequentially connected components. The convolutional submodule, the ReLU activation function submodule, and the softmax activation function submodule are used to process the second support set features through the spatial attention module. The result is then copied C times in the second dimension to obtain the attention image. The attention image is then multiplied element-wise with the obtained matrix image to obtain the element image, thereby assigning weights to each element. The element images are then processed by the second average pooling module to obtain the support set feature vectors; Based on the obtained query set feature vector and support set feature vector, the first information entropy is calculated using the following formula. : In the formula For the first k Zhang queries the categories of the image collection; For the first l Zhang supports the categories of images in the set; For the first k Features of the Zhang query set image after processing by the task adapter; For the first n Zhang supports the features of the image after processing by the task adapter; A feature modifier is constructed based on spatial attention. This feature modifier repairs preprocessed features based on their similarity. Then, the initial image classification model is trained using the information entropy of the preprocessed features. Finally, the repaired features are used to classify the input image through a second classifier. Specifically, this includes the following: The input to the feature modifier is the preprocessed query set feature image and the preprocessed support set feature image; First, the preprocessed support set feature image is repaired using the preprocessed query set feature image. Then, the repaired support set feature image is used to repair the preprocessed query set feature image, resulting in the repaired query set feature image. The total number of feature images in the query set after preprocessing is The total number of support set feature images after preprocessing is ,in , ; Each channel of each image is converted into a one-dimensional tensor. The preprocessed query set feature image and the preprocessed support set feature image are then multiplied pairwise, and spatial attention is calculated to obtain a similarity matrix R, with the shape of R as follows: ; For each preprocessed support set feature image, the correction process is expressed by the following formula: In the formula For the repaired support set i Features of the image; This represents the j-th row of the similarity matrix R; Features of the i-th image in the support set before repair; Will The repaired features are stacked to obtain the corrected support set features; Transpose the similarity matrix R; the transposed shape is: ; For each preprocessed query set feature image, the correction process is expressed by the following formula: In the formula The th similarity matrix R l Row data; The features of the k-th image in the query set before repair are obtained; after repair, the features of the query set after repair are obtained. Based on the revised support set features and the revised query set features, the second information entropy is calculated using the following formula. : In the formula Let m be the category of the query set image; Let n be the category of the nth support set image; The features of the m-th query set image after processing by the feature correction module; The features of the nth support set image after processing by the feature correction module; The corrected support set features and query set features are input into the overall classifier to obtain the final classification result; the overall classifier is a single classifier. Convolutional networks; S8. Using the training dataset obtained in step S6, train the initial image classification model constructed in step S7 to obtain the final image classification model; specifically including the following steps: The overall loss function L is calculated using the following formula: In the formula, v is the classification loss of the classification result output by the feature modifier and the corresponding label. In specific implementation, the classification result is the classification result of the query set data, and the corresponding label is the label of the query set. However, the classification model does not need the query set label, but only uses the query set label to calculate the loss function. This is the first weight value; This is the second weight value; Based on the calculated overall loss function, the backpropagation algorithm is used to train and optimize the model; During the training phase, the model is fed three types of data: support set images, support set labels, and query set images. These three types of data are the data that the model directly needs to apply and compute. The query set labels are only used to calculate the loss function during model training, and the model does not directly use the query set labels. S9. Prune the image classification terminal model obtained in step S8 to obtain an image classification model, and use the obtained image classification model to perform image classification based on few-shot learning; specifically including the following steps: Obtain the image classification terminal model obtained in step S8; In the image classification terminal model, the feature processor and task adapter are removed, and the output of the feature extractor is directly used as the input of the feature modifier, thus obtaining the image classification model; The obtained image classification model is then used for image classification based on few-shot learning. Since image classification based on few-shot learning involves a dataset, in practical applications, users need to select a small portion of images from the target dataset, label them, and use this portion as the support set, while using the remaining portion as the query set. During application, the support set and the corresponding number of query sets are input into the final image classification model. The image classification model can then classify the images in the query set based on the images in the input support set and their corresponding labels.

[0022] The image classification method based on few-shot learning provided by this invention is particularly suitable for endangered species classification, target tracking, and gesture recognition.

[0023] The following comparative example illustrates the effects of the present invention: The model trained using the few-shot learning method proposed in this invention will be compared with existing few-shot learning methods. The comparison will be conducted on two public datasets: the first dataset is the Mini-ImageNet dataset, containing 64 training classes and 20 test classes, with 600 images per class; the second dataset is the CIFAR-FS dataset, containing 64 training classes and 20 test classes, with 100 images per class. During the experiments, all methods will be trained and learned according to the proposer's method, with the number of support sets K selected as either 1 or 5, and the classification accuracy will be used as the evaluation criterion. All experimental results will be obtained on the test set.

[0024] The 2018 method was proposed by Bertinetto, L., Henriques, JF, Torr, PH, Vedaldi, A. in their 2018 paper "Meta-learning with differentiable closed-form solvers"; the 2019 method was proposed by Ravichandran, A., Bhotika, R., Soatto, S. in their 2019 paper "Few-shot learning with embedded class models and shot-free meta-training"; and the 2021 method was proposed by Ouali, Y., Hudelot, C., Tami, M. in their 2021 paper "Spatial contrastive learning for few-shot classification". The comparison results are shown in Table 1: Table 1. Schematic diagram of comparative experimental results

[0025] As can be seen from the data in Table 1, the method of the present invention achieved optimal results on two different datasets, outperforming other few-shot learning methods, thus demonstrating the superiority and feasibility of the method of the present invention.

[0026] The method of this invention can improve the learning effect of the model in the pre-training stage (student model and teacher model stage) and learn more transferable knowledge. In the subsequent second training stage, the model can integrate the information of the support set and the query set to repair the feature map after it has been enlarged or scrambled. In this process, the model needs to pay attention to the features of the support set and the query set at the same time, so it can solve the support-query drift problem and better learn the general features and knowledge of the image in the new task.

Claims

1. An image classification method based on few-shot learning, characterized in that... Includes the following steps: S1. Obtain the existing training dataset; S2. Construct a classification task model based on a feature extractor and a first classifier; the feature extractor is used to extract features from the input image, and the first classifier is used to classify the input image according to the extracted features; S3. Using the training dataset obtained in step S1, set the loss function and train the classification task model constructed in step S2, and use the trained model as the initial teacher model. S4. Construct a student model with the same structure as the current teacher model, and randomly initialize the parameters of the student model; using the training dataset obtained in step S1, train the student model based on the loss function set in step S3 and the structural loss function between the student model and the teacher model, and use the trained student model as the current teacher model. S5. Repeat step S4 until the set number of times; obtain the feature extractor from the student model obtained from the last training. S6. Obtain the existing training dataset again; S7. Construct an initial image classification model that includes a feature processor, a feature modifier, a task adapter, and the feature extractor obtained in step S5; The feature extractor is used to extract features from the input image, the feature processor is used to preprocess the extracted features, the task adapter is used to train the initial image classification model, and the feature modifier is used to repair the preprocessed input features and output the final classification result; specifically, it includes the following steps: The constructed initial image classification model includes the feature extractor obtained in step S5, the newly constructed feature processor, the newly constructed feature modifier, and the newly constructed task adapter; The feature extractor is used to extract features from the input image and input the extracted feature image into the feature processor; the input image includes a support set image and a query set image; the extracted feature image includes a support set feature image and a query set feature image. The feature processor is used to perform preprocessing operations on the received feature image to obtain preprocessed features, and simultaneously inputs the preprocessed features into the feature corrector and the task adapter; the preprocessing operations include coarsening and scrambling operations. Build a task adapter based on convolution, instance attention, and spatial attention; The task adapter is used to train the initial image classification model based on the information entropy of the preprocessed features; Constructing a feature modifier based on spatial attention; The feature modifier is used to repair the preprocessed features based on the similarity of the preprocessed features. Then, the image classification initial model is trained based on the information entropy of the preprocessed features. Finally, the repaired features are used to classify the input image through a second classifier. In specific implementation, the feature modifier includes the following: The input to the feature modifier is the preprocessed query set feature image and the preprocessed support set feature image; First, the preprocessed support set feature image is repaired using the preprocessed query set feature image. Then, the repaired support set feature image is used to repair the preprocessed query set feature image, resulting in the repaired query set feature image. The total number of feature images in the query set after preprocessing is The total number of support set feature images after preprocessing is ,in , ; Each channel of each image is converted into a one-dimensional tensor. The preprocessed query set feature image and the preprocessed support set feature image are then multiplied pairwise, and spatial attention is calculated to obtain a similarity matrix R, with the shape of R as follows: ; For each preprocessed support set feature image, the correction process is expressed by the following formula: In the formula Features of the i-th image in the repaired support set; This represents the j-th row of the similarity matrix R; Features of the i-th image in the support set before repair; Will The repaired features are stacked to obtain the corrected support set features; Transpose the similarity matrix R; the transposed shape is: ; For each preprocessed query set feature image, the correction process is expressed by the following formula: In the formula This represents the data in the l-th row of the similarity matrix R; Features of the k-th image in the query set before repair; After the correction is completed, the corrected query set features are obtained; Based on the revised support set features and the revised query set features, the second information entropy is calculated using the following formula. : In the formula Let m be the category of the query set image; Let n be the category of the nth support set image; The features of the m-th query set image after processing by the feature correction module; The features of the nth support set image after processing by the feature correction module; The corrected support set features and query set features are input into the overall classifier to obtain the final classification result; the overall classifier is a single classifier. Convolutional networks; S8. Using the training dataset obtained in step S6, train the initial image classification model constructed in step S7 to obtain the image classification terminal model; S9. Prune the image classification terminal model obtained in step S8 to obtain an image classification model, and use the obtained image classification model to perform image classification based on few-shot learning.

2. The image classification method based on few-shot learning according to claim 1, characterized in that... Step S2 specifically includes the following steps: The classification task model consists of a concatenated feature extractor and a first classifier; The feature extractor uses a ResNet12 network, comprising a first residual convolutional module, a second residual convolutional module, a third residual convolutional module, and a fourth residual convolutional module connected in series. The first, second, third, and fourth residual convolutional modules have identical structures. In the residual convolutional modules, the input feature data is divided into two paths. The first path of feature data is processed sequentially through the first, second, and third convolutional sub-modules to obtain a first sub-feature map. The second path of feature data is processed through a downsampling sub-module to obtain a second sub-feature map. The first and second sub-feature maps are then added bit-by-bit as the output of the residual convolutional module. The first convolutional sub-module includes a... A convolutional layer, a batch normalization layer, and a ReLU activation function layer; the second and third convolutional submodules each include a... A convolutional layer and a batch normalization layer; the downsampling submodule includes a A convolutional layer and a ReLU activation function layer; The first classifier is a fully connected layer network; The feature extractor is used to extract features from the input image; the first classifier is used to classify the input image based on the extracted features.

3. The image classification method based on few-shot learning according to claim 2, characterized in that... Step S3 specifically includes the following steps: In the training dataset obtained in step S1, the labels of the images are obtained and converted into 0-1 labels; Input the images from the training dataset into the classification task model constructed in step S2 to obtain the corresponding prediction results; Based on the obtained prediction results and the corresponding 0-1 labels, the first loss is calculated using the following formula. : In the formula, N is the total number of input images; Let be a binary variable indicating whether the i-th image belongs to the j-th class. If the i-th image belongs to the j-th class, then... ,otherwise ; is the jjth value of the prediction result for the ii-th image; Based on the obtained prediction results and the corresponding 0-1 labels, the second loss is calculated using the following formula. : In the formula The prediction result for the i-th image; The feature center is the image category to which the input image belongs; the feature center is the average of the feature values ​​of all images in the corresponding category. This represents the L2 normal form of the matrix; The classification task model constructed in step S2 is trained based on the first loss and the second loss, and the trained model is used as the initial teacher model.

4. The image classification method based on few-shot learning according to claim 3, characterized in that... Step S4 specifically includes the following steps: The parameters of the current teacher model will not be modified. Construct a student model with the exact same structure as the current teacher model, and randomly initialize the parameters of the student model; The image is simultaneously input into both the current teacher model and the current student model; Based on the prediction results of the current teacher model and the 0-1 labels corresponding to the input image, calculate the first loss and the second loss of the current student model; Based on the prediction results of the current student model and the current teacher model, the structural loss of the current student model is calculated using the following formula. : In the formula This is the function for calculating cosine similarity. Let i be the feature distance matrix of the current teacher model, where the element in the i1th row and j1st column is... for , This represents the teacher model's prediction result for the i1th image. This represents the teacher model's prediction result for the j1-th image; Let i be the feature distance matrix of the current student model, where the element in the i2th row and j2nd column is... for , This represents the prediction result of the student model for the i2th image. This represents the prediction result of the student model for the j2-th image; Let i be the characteristic angle relation matrix of the current teacher model, where the element in the i3th row and j3rd column is... for , This represents the teacher model's prediction result for the i3rd image. This represents the teacher model's prediction result for the j3rd image; The characteristic angle relation matrix of the current student model, where the element in the i4th row and j4th column is... for , This represents the prediction result of the student model for the i4th image. This represents the prediction result of the student model for the j4th image; Using the training dataset obtained in step S1, the student model is trained based on the obtained first loss, second loss, and structural loss, and the trained student model is used as the current teacher model.

5. The image classification method based on few-shot learning according to claim 4, characterized in that... Step S6 specifically includes the following steps: Obtain the existing training dataset again; In the acquired training dataset, a number of images are randomly selected to construct the support set and query set. The random selection rule is as follows: N categories are randomly selected, and KK images are randomly selected for each category to construct the support set, and QQ images are randomly selected for each category to construct the query set. When inputting a model, it will support inputting both the set and the query set together.

6. The image classification method based on few-shot learning according to claim 5, characterized in that... The feature processor specifically includes the following: Roughing operation: The input support set feature image is fed into the erase function; the erase function randomly generates a set number of random numbers, maps all random numbers back to the feature image coordinates, and uses the selected numbers as the pixels to be erased; for the erased pixels, the values ​​of the pixels in all channels are set to 0; after the erasure of all selected pixels is completed, the preprocessed support set feature image is obtained. Disrupt the operation: The input query set feature image is used as the object of the shuffling operation; The process of generating the shuffled images: Based on the number of images in each batch, generate an integer array of the same length as the number of images. This integer array includes the range... All certificates within the array, and the numbers within the array are in random order; Each image will use the corresponding data in the integer array as the same region of the image received by its own scrambled region; The process of generating a scrambled region: generating a region in The random parameter of the region is used as a percentage of the image scrambled area; two random integers xx and yy are selected, where xx is less than the image length and yy is less than the image width, and their coordinates are... As the center of the scrambled region; before generating the scrambled region, the coordinates of the upper left and lower right corners of the scrambled region are calculated and limited to the image area; finally, based on the coordinates... The final scrambled area is defined by the defined upper left and lower right coordinates. Each image is processed in a shuffled order, and the shuffled area information of the corresponding image is received and overwritten with the original information of the image itself. Copy the label of each image, starting from a length of [length missing]. One-dimensional tensor expansion to The three-dimensional tensor, in which H is the number of categories in the training set, H is the height of the feature image, and W is the width of the feature image. After expansion, the labels are processed according to the shuffled order and shuffled area to complete the information exchange between different labels: each label receives the information of the shuffled area of ​​the corresponding label according to the shuffled order to overwrite its original information. Finally, the preprocessed query set feature image is obtained.

7. The image classification method based on few-shot learning according to claim 6, characterized in that... The aforementioned task adapter specifically includes the following: The task adapter includes a projection head module, a first average pooling module, an instance attention module, a spatial attention module, and a second average pooling module. The projection head submodule comprises four convolutional submodules connected in series, each of which includes one... Convolutional layers and ReLU activation function layers; The preprocessed query set feature image and the preprocessed support set feature image are input into the projection head submodule for processing. The query set features processed by the projection head submodule are processed by the first average pooling module to obtain the query set feature vector. The support set features processed by the projection head submodule are divided into three paths: the first path is processed by the instance attention module, the second path is processed by the spatial attention module, and the third path is used for direct calculation. The instance attention module consists of sequentially connected... The system consists of a convolution submodule, an average pooling submodule, a ReLU activation function submodule, and a softmax activation function submodule. The first support set features are processed by the instance attention module and then multiplied with the third support set features to obtain a matrix image, thereby assigning weights to each support set feature. The spatial attention module consists of sequentially connected components. Convolutional submodule, ReLU activation function submodule, and softmax activation function submodule; After the second support set features are processed by the spatial attention module, the result is copied C times in the second dimension to obtain the attention image. The attention image is then multiplied element-wise with the obtained matrix image to obtain the element image, thereby assigning weights to each element. The element images are then processed by the second average pooling module to obtain the support set feature vectors; Based on the obtained query set feature vector and support set feature vector, the first information entropy is calculated using the following formula. : In the formula Let k be the category of the query set image. The category of the l-th support set image; The features of the k-th query set image after processing by the task adapter; The features of the nth support set image after processing by the task adapter.

8. The image classification method based on few-shot learning according to claim 7, characterized in that... The training described in step S8 specifically includes the following steps: The overall loss function L is calculated using the following formula: In the formula The classification loss is the classification result output by the feature modifier and the corresponding label. This is the first weight value; This is the second weight value; Based on the calculated overall loss function, the backpropagation algorithm is used to train and optimize the model; Step S9 specifically includes the following steps: Obtain the image classification terminal model obtained in step S8; In the image classification terminal model, the feature processor and task adapter are removed, and the output of the feature extractor is directly used as the input of the feature modifier, thus obtaining the image classification model; The obtained image classification model is then used for image classification based on few-shot learning.

Images