A small sample image classification method, device and storage medium

Abstract

This invention relates to a method, device, and storage medium for few-sample image classification. The method includes: extracting features from few-sample images using a feature extraction network to obtain feature vectors; constructing a parameter optimization learning model to transform the feature vectors into classifier parameters; training the classifier parameters of the parameter optimization learning model for a set number of rounds on mutually exclusive few-sample image training and testing sets to obtain a parameterized base learner; introducing new meta-parameters to optimize and update the parameterized base learner; and using the optimized meta-learning model to achieve few-sample image classification. Compared with existing technologies, this invention has the advantage of high classification accuracy.

Description

Technical Field

[0001] This invention relates to the field of few-sample image classification technology, and in particular to a few-sample image classification method, device and storage medium. Background Technology

[0002] With the development of science and technology, the application of computer vision-based image processing and image classification technologies is becoming increasingly widespread. Image classification is an important fundamental task in computer vision. Most existing image classification methods employ deep learning, using big data as a background, and learning through various neural networks to achieve superior model performance.

[0003] However, in practice, labeling a large number of labeled samples is a huge undertaking, and the number of usable samples is often limited. In many cases, it is desirable to enable neural network models to learn quickly using only a few samples; therefore, the few-sample problem has broad research prospects.

[0004] However, existing methods for studying small sample problems have limited effective sample sizes in small datasets, which may lead to overfitting. They may also suffer from poor feature extraction and low classification accuracy due to uneven sample distribution.

[0005] Therefore, there is an urgent need to design an image classification method suitable for small sample data. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art by providing a small sample image classification method, device and storage medium to solve problems such as low classification accuracy and overfitting due to insufficient training samples in small sample image classification problems.

[0007] The objective of this invention can be achieved through the following technical solutions:

[0008] According to a first aspect of the present invention, a few-sample image classification method is provided, the method comprising the following steps:

[0009] Step S1: Establish mutually exclusive training and test sets on the small sample image dataset to complete the data preprocessing process;

[0010] Step S2: Input the small sample image into the feature extraction network, and use the residual network module as the feature extractor to extract features from the input small sample image to obtain the feature vector;

[0011] Step S3: Introduce meta-parameters, use differentiable parameters to optimize the learning model, train the base learner parameters, and transform the feature vector into classifier parameters to complete the meta-training stage;

[0012] Step S4: Combine the trained classifier parameters with a linear classifier to predict the class, and obtain the prediction result of the classifier;

[0013] Step S5: After the parameterization process of the base learner is implemented, the classifier parameters are optimized and updated, new meta-parameters are introduced to train the meta-learner, and the meta-testing process is implemented.

[0014] Step S6: In the training of the meta-learner, the classification loss function is calculated based on the parameter optimization learning model, the prediction results of the classifier are updated, and steps S3-S6 are repeated to carry out the meta-training and meta-testing process to continuously optimize the parameters and obtain the final meta-learning model.

[0015] Step S7: Perform small sample image classification on the training and test sets based on the trained meta-learning model.

[0016] Preferably, the parameter optimization learning model is a few-sample image classification model based on Lasso regression.

[0017] Preferably, the mathematical expression of the few-sample image classification model based on Lasso regression is:

[0018]

[0019] In the formula, The optimal weight parameters output by the Lasso regression-based few-sample image classification model. Z This represents the dataset used for meta-training. X, Y These represent the input feature vector and output feature vector of the meta-training set samples, respectively. I To and X, Y The identity matrix of the same order; the latter term is the L1 regularization penalty term. λ This represents the regularization parameter.

[0020] Preferably, in the few-sample image classification model based on Lasso regression, the loss function obtained according to its parameter regularization property has the following form:

[0021]

[0022] In the formula, n is the sample size. X, Y These represent the sample input feature vector and the output feature vector, respectively. α 1 This represents the adjustable hyperparameters and the updated weight parameters. W ’ All of these are used as meta-parameters for training.

[0023] Preferably, the parameter optimization learning model is a few-sample image classification model based on elastic network regression.

[0024] Preferably, the mathematical expression of the few-sample image classification model based on elastic network regression is:

[0025]

[0026] In the formula, The optimal weight parameters output by the Lasso regression-based few-sample image classification model. Z This represents the dataset used for meta-training. X, Y Let I represent the input feature vector and output feature vector of the meta-training set samples, respectively. X, Y The identity matrices are of the same order, and the last two terms are the L1 regularization penalty terms and the L2 regularization penalty terms, respectively. λ This represents the regularization parameter.

[0027] Preferably, the loss function expression of the few-sample image classification model based on elastic network regression is:

[0028]

[0029] In the formula, n is the sample size. X, Y These represent the sample input feature vector and the output feature vector, respectively. α 2 、β This represents the adjustable hyperparameters and the updated weight parameters. W ’ All of these are used as meta-parameters for training.

[0030] Preferably, in step S5, the subgradient approach is used during parameter training, employing coordinate axis descent to update the classifier weight parameters while also considering non-differentiable points. W :

[0031]

[0032] in, For the first The weight parameters are updated in the next iteration; The dimension of the weight parameter matrix represents the set number of iterations.

[0033] According to a second aspect of the present invention, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement any of the methods described above.

[0034] According to a third aspect of the invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements any of the methods described herein.

[0035] Compared with the prior art, the present invention has the following advantages:

[0036] 1) This invention, through a meta-learning framework, employs two differentiable optimization algorithms to improve feature extraction and classification accuracy to a certain extent for small sample image classification tasks;

[0037] 2) The method of obtaining classifier parameters from feature vectors based on Lasso regression used in this invention can accelerate the convergence speed, improve the feature selection effect and maintain the sparsity of the model compared with other algorithms such as ridge regression;

[0038] 3) The method used in this invention to obtain classifier parameters from feature vectors based on elastic network regression, and the addition of L1 and L2 regularization terms, can effectively solve the overfitting problem, ensure the sparsity and stability of the model at the same time, and handle highly correlated multidimensional data. Attached Figure Description

[0039] Figure 1 This is a flowchart of the method in Example 1;

[0040] Figure 2 This is a flowchart of the method in Example 4. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0042] Example 1

[0043] like Figure 1 As shown in the figure, this embodiment presents a few-sample image classification method, which includes the following steps:

[0044] Step S1: Establish mutually exclusive training and test sets on the small sample image dataset to complete the data preprocessing process;

[0045] Step S2: Input the small sample image into the ResNet18 feature extraction network, and use the residual network module as the feature extractor to extract features from the input small sample image to obtain the feature vector;

[0046] Step S3: Introduce meta-parameters, use differentiable parameters to optimize the learning model, train the base learner parameters, and transform the feature vector into classifier parameters to complete the meta-training stage;

[0047] Step S4: Combine the trained classifier parameters with a linear classifier to predict the class, and obtain the prediction result of the classifier;

[0048] Step S5: After the parameterization process of the base learner is implemented, the classifier parameters are optimized and updated. New meta-parameters are introduced to train the meta-learner, and the meta-testing process is implemented. During parameter training, the subgradient idea is adopted, and the coordinate axis descent method is used to update the classifier weight parameters while taking into account non-differentiable points. W :

[0049]

[0050] in, For the first The weight parameters are updated in the next iteration; The dimension of the weight parameter matrix represents the set number of iterations.

[0051] Step S6: In the training of the meta-learner, the classification loss function is calculated based on the parameter optimization learning model, the prediction results of the classifier are updated, and steps S3-S6 are repeated to carry out the meta-training and meta-testing process to continuously optimize the parameters and obtain the final meta-learning model.

[0052] Step S7: Perform small sample image classification on the training and test sets based on the trained meta-learning model.

[0053] The electronic device of this invention includes a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) or loaded from a storage unit into random access memory (RAM). The RAM may also store various programs and data required for device operation. The CPU, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0054] Multiple components in the device are connected to the I / O interface, including: input units such as keyboards and mice; output units such as various types of displays and speakers; storage units such as disks and optical discs; and communication units such as network interface cards (NICs), modems, and wireless transceivers. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0055] The processing unit executes the various methods and processes described above, such as steps S1 to S7 of the methods. For example, in some embodiments, steps S1 to S7 of the methods may be implemented as a computer software program tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via ROM and / or a communication unit. When the computer program is loaded into RAM and executed by the CPU, one or more steps S1 to S7 of the methods described above may be performed. Alternatively, in other embodiments, the CPU may be configured to execute steps S1 to S7 of the methods by any other suitable means (e.g., by means of firmware).

[0056] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload programmable logic devices (CPLDs), and so on.

[0057] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0058] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0059] Example 2

[0060] In this embodiment, the parameter optimization learning model is a few-sample image classification model based on Lasso regression, and its mathematical expression is:

[0061]

[0062] In the formula, The optimal weight parameters output by the Lasso regression-based few-sample image classification model. Z This represents the dataset used for meta-training. X, Y These represent the input feature vector and output feature vector of the meta-training set samples, respectively. I To and X, Y The identity matrix of the same order; the latter term is the L1 regularization penalty term. λ This represents the regularization parameter.

[0063] For the Lasso regression method, the loss function obtained based on its parameter regularization property takes the following form:

[0064]

[0065] In the formula, n is the sample size. X, Y These represent the sample input feature vector and the output feature vector, respectively. α 1 This represents the adjustable hyperparameters and the updated weight parameters. W ’ All of these are used as meta-parameters for training.

[0066] The other settings in this embodiment are the same as in Embodiment 1.

[0067] Example 3

[0068] In this embodiment, the parameter optimization learning model is a few-sample image classification model based on elastic network regression, and its mathematical expression is:

[0069]

[0070] In the formula, The optimal weight parameters output by the Lasso regression-based few-sample image classification model. Z This represents the dataset used for meta-training. X, Y Let I represent the input feature vector and output feature vector of the meta-training set samples, respectively. X, Y The identity matrices are of the same order, and the last two terms are the L1 regularization penalty terms and the L2 regularization penalty terms, respectively. λ This represents the regularization parameter.

[0071] The loss function expression is:

[0072]

[0073] In the formula, n is the sample size. X, Y These represent the sample input feature vector and the output feature vector, respectively. α 2 、β This represents the adjustable hyperparameters and the updated weight parameters. W ’ All of these are used as meta-parameters for training.

[0074] The other settings in this embodiment are the same as in Embodiment 1.

[0075] Example 4

[0076] like Figure 2 As shown, this embodiment provides a few-sample image classification method that can be used to perform image classification experiments on the Omniglot few-sample image dataset. The method includes the following steps:

[0077] Step A1: Data preprocessing.

[0078] For small sample datasets, the dataset is divided as follows: On the Omniglot dataset, of the 1623 classes it contains, 1200 classes are used for model training to implement classifier parameterization; and 423 classes are used for model testing to update parameters and evaluate model performance.

[0079] Step A2: Training process of small sample image classifiers.

[0080] First, the format of the few-sample image classification task is specified: For a classification task, there are several training epochs. Each epoch adopts the N-way K-Shot task format, that is, N categories are selected as the number of categories in the training set, and K samples are used as the support set. In this embodiment, taking the 5-way 1-Shot task structure as an example, in the 20 samples contained in each class, 1 sample is selected as the support set, which is used as the sample to be matched; the other 15 samples are selected as the query set to complete the sample matching and classification process. Therefore, one epoch includes 15×5+1×5=80 samples. For 5-way 5-Shot, the same sample partitioning method as 5-way 1-shot is used, while for 20-way 1-Shot and 20-way 5-Shot experiments, 10 samples are selected as the support set, and the remaining 10 samples are used as the query set. The task format is consistent during sample testing with that during training. Samples are randomly input according to the classification task format. After passing through the ResNet18 feature extraction network, feature vectors are obtained, and the classification prediction results are output by the linear classifier.

[0081] The parameterization process is then implemented using Lasso regression or elastic network regression algorithms. The training process, employing the task format in S2, lasts for 60,000 epochs, with model validation performed on the test set every 1,000 epochs.

[0082] Step A3: Small sample image classification meta-testing process.

[0083] In the meta-testing phase, the task format is the same as during meta-training, requiring the introduction of meta-parameters for parameter updates. For the task format in S2, 1000 rounds of meta-parameter training are performed on the entire dataset, iteratively updating the parameters and validating the entire meta-learning model.

[0084] This embodiment can perform classification on small sample images, effectively reducing overfitting and achieving high classification accuracy.

[0085] The other settings in this embodiment are the same as in Embodiment 1.

[0086] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A few-sample image classification method, characterized in that, The method includes the following steps: Step S1: Establish mutually exclusive training and test sets on the small sample image dataset to complete the data preprocessing process; Step S2: Input the small sample image into the feature extraction network, and use the residual network module as the feature extractor to extract features from the input small sample image to obtain the feature vector; Step S3: Introduce meta-parameters, use differentiable parameters to optimize the learning model, train the base learner parameters, and transform the feature vector into classifier parameters to complete the meta-training stage; Step S4: Combine the trained classifier parameters with a linear classifier to predict the class, and obtain the prediction result of the classifier; Step S5: After the parameterization process of the base learner is implemented, the classifier parameters are optimized and updated, new meta-parameters are introduced to train the meta-learner, and the meta-testing process is implemented. Step S6: In the training of the meta-learner, the classification loss function of the learning model is optimized based on the parameters, the prediction results of the classifier are updated, and steps S3-S6 are repeated to carry out the meta-training and meta-testing process to continuously optimize the parameters and obtain the final meta-learning model. Step S7: Perform small sample image classification on the training set and test set based on the trained meta-learning model; The parameter optimization learning model is a few-sample image classification model based on Lasso regression, and its mathematical expression is: In the formula, The optimal weight parameters output by the Lasso regression-based few-sample image classification model. Z This represents the dataset used for meta-training. X, Y These represent the input feature vector and output feature vector of the meta-training set samples, respectively. I To and X, Y The identity matrix of the same order; the latter term is the L1 regularization penalty term. λ Indicates the regularization term parameter; In the few-sample image classification model based on Lasso regression, the loss function obtained based on its parameter regularization property takes the following form: In the formula, n is the sample size. X, Y These represent the sample input feature vector and the output feature vector, respectively. α 1 This represents the adjustable hyperparameters and the updated weight parameters. W ’ All of these are used as meta-parameters for training; Alternatively, the parameter optimization learning model may be a few-sample image classification model based on elastic network regression, and its mathematical expression is: In the formula, The optimal weight parameters output by the Lasso regression-based few-sample image classification model. Z This represents the dataset used for meta-training. X, Y Let I represent the input feature vector and output feature vector of the meta-training set samples, respectively. X, Y The identity matrices are of the same order, and the last two terms are the L1 regularization penalty terms and the L2 regularization penalty terms, respectively. λ Indicates the regularization term parameter; The loss function expression for the few-sample image classification model based on elastic network regression is: In the formula, n is the sample size. X, Y These represent the sample input feature vector and the output feature vector, respectively. α 2 、β This represents the adjustable hyperparameters and the updated weight parameters. W ’ All of these are used as meta-parameters for training; The subgradient approach is used during parameter training, employing coordinate descent to update the classifier weight parameters while also considering non-differentiable points. W : in, For the first The weight parameters are updated in the next iteration; The dimension of the weight parameter matrix represents the set number of iterations.

2. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the program, it implements the method as described in claim 1.

3. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in claim 1.

Images