A small sample image classification method based on dimension adaptation
By using the dimensional adaptive method in small sample image classification, class features are extracted and singular value decomposition is performed, and the loss functions of high-dimensional and low-dimensional feature constraints are combined, the problems of low recognition accuracy and high computational complexity are solved, and the problem of low-sample recognition is achieved. More efficient image recognition and generalization capabilities.
Patent Information
- Application Number
- CN202111566913.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-20
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2041-12-20
AI Technical Summary
The prior art is difficult to effectively utilize limited data sets in small sample image classification, resulting in low recognition accuracy and high computational complexity, especially in the absence of a large amount of labeled data.
Using a small sample image classification method based on dimensional adaptation, class features are extracted through convolutional neural network, singular value decomposition, dimensionality reduction to subspace, and high-dimensional and low-dimensional feature constraints are added to the loss function, and the network is updated using gradient descent parameters to improve identification accuracy and calculation efficiency.
It improves the accuracy and generalization ability of image classification, reduces the computing volume and running speed of the network, and enhances the recognition ability in small samples.
Smart Images

Figure CN114299495B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an image classification technology, which is applicable to the situation where a large amount of labeled data is lacking; in particular, it relates to a small sample image classification method based on dimension adaptation. Background Art
[0002] Currently, image recognition systems have been widely used in different fields, and the number of identifiable categories is increasing. However, at the same time, the cost and difficulty of labeling large data sets are increasing. Usually, training an image recognition network requires a large number of relatively balanced data sets. In practical applications, on the one hand, data labeling requires a lot of human resources, and on the other hand, it is very difficult to collect a large number of samples of the same category in some scenarios. Therefore, the small sample problem has become a hot topic in academic research in recent years.
[0003] In response to the problem of small sample recognition, researchers have proposed various algorithms. In summary, the current algorithms can be roughly divided into two categories: 1) Methods based on distance metrics. This method generally follows a paradigm, that is, first extracting features from the image through a feature extraction network, then using a classifier to calculate the distance between different features, and finally determining the classification result of the image based on the distance. 2) Methods based on parameter optimization. This method finds the best initial parameters for the network so that the network can find the optimal parameters even after several gradient descents in the absence of a large number of samples, thereby achieving the purpose of recognition. However, this type of method often involves the problem of quadratic gradient descent, which increases the complexity of the model. Summary of the invention
[0004] To solve the above problems, the present invention provides a small sample image classification method based on dimension adaptation; the network takes tasks as basic input, each task contains a support set and a test set, and uses a convolutional neural network to extract image features with category characteristics and combine features of the same category to obtain category features with category representativeness. The corresponding singular values and subspaces are obtained by performing singular value decomposition on the category features, and finally the category to which the sample belongs is determined by calculating the distance from the sample features to be identified to each subspace; this method improves accuracy, while reducing the amount of network calculation to a certain extent and improving the running speed.
[0005] The present invention discloses a small sample image classification method based on dimension adaptation.
[0006] Step 1: Preprocess the images in the training set data and the test set data, including image cropping, normalization, and rotation, etc. At the same time, perform data enhancement on the preprocessed images, and expand the number of samples based on the existing small sample data.
[0007] Step 2: Divide the preprocessed images by image category, and use the task as the basic input unit for network training.
[0008] Step 3: Process the image through a convolutional neural network to extract the features of each image in the same category, and then obtain representative and recognizable category features.
[0009] Step 4: Calculate the corresponding adaptive dimension based on the category features obtained in step 2, and obtain the adaptive feature subspace corresponding to each category of images accordingly.
[0010] Step 5: Improve the loss function and add constraints on high-dimensional and low-dimensional features to the loss function to provide more information and guidance for network training. That is, for the same category of sample images, calculate the distance of the network's middle layer features in three different dimensions and minimize it. Finally, take the weighted sum of the distances in the three dimensions as part of the overall loss function, and use the gradient descent method to update the network parameters.
[0011] Step 6: Input the image to be recognized into the feature extraction network to obtain the corresponding image features, and then calculate the distance between the feature and each feature subspace. The recognition result is determined as the category of the subspace closest to the image feature.
[0012] The present invention is further improved in that: the specific method of step 1 is:
[0013] Since the image sizes in the public dataset ImageNet are not uniform, this method will first preprocess the images in the dataset. First, the image length and width are uniformly cropped to C*W*H, where C is the number of channels of the image, and W and H are the pixel values of the image. In this method, C is 3, and W and H are both set to 84. Then each image is normalized to further improve the training performance. In normalization, the mean and standard deviation of each channel used are [0.485, 0.456, 0.406], [0.229, 0.224, 0.225]; then the image is rotated at different angles. The preprocessed image is a standard image.
[0014] The present invention is further improved in that: the specific method of step 2 is:
[0015] For the preprocessed images, this method divides the dataset into training set D base and the test set D novel , where the image categories contained in the training set and the test set are unrelated. Then D baseThe images in are further divided into several support sets and query sets, where the support set and the query set have the same image categories and the same number of categories.
[0016] Let each task in the training set The support set Query Set x and y represent the image and the label corresponding to the image respectively, where the support set contains N categories, and each category contains K pictures.
[0017] Let each task in the test set The support set Query Set x and y represent the image and the label corresponding to the image respectively, where the support set contains N categories, and each category contains K pictures.
[0018] The present invention is further improved in that: the specific method of step 3 is:
[0019] The task Input into the network, the support set Each image in is input into the feature extraction network In this paper, we can get a set of image features Then calculate the mean of the feature and subtract the mean μ from each sample to get
[0020] The present invention is further improved in that: the specific method of step 4 is:
[0021] Step 4.1: By By performing singular value decomposition SVD, we can get the left singular matrix U of this set of samples. s , the right singular value matrix V s And the singular value matrix ∑ s , where the main diagonal of the singular value matrix contains all the eigenvalues, and the eigenvectors in the singular value matrix are similar to the singular value matrix ∑ s There is a one-to-one correspondence between the singular values in . The singular value decomposition process is as follows:
[0022]
[0023] Step 4.2: Arrange the decomposed singular values in descending order, select the first m singular values and the first m vectors of the corresponding left singular value matrix as the projection transformation matrix for projection to the m-dimensional subspace.
[0024] Step 4.3: Determine the specific value of m by using the general cross validation (GCV) method to minimize the mean square error of the predicted value. Let X = {x1, x2, ..., x n} is a set of data with n samples, each sample has c dimensions (c ≥ m). Singular value decomposition is used to reduce it to m dimensions and then reconstruct it to c dimensions, and we get The mean square error is as follows:
[0025]
[0026] Furthermore, the first-order approximation formula of formula (2) can be used to simplify the calculation complexity as follows:
[0027]
[0028] By minimizing GCV, the optimal dimension m can be obtained, so that the network has better generalization ability while retaining the original information.
[0029] Step 4.4: Perform dimensionality reduction operation on the features of the images in the support set according to the corresponding dimension m calculated for each type of image. That is, arrange the singular values in descending order and take the first m singular values in the left singular matrix U s The corresponding vector is taken as the subspace after dimensionality reduction.
[0030] The present invention is further improved in that: the specific method of step 5 is:
[0031] On the basis of the original cross entropy loss function, constraints on the features of different layers of the network are added to provide more information and guidance for network training. Query Set For For each image in Then we can get the corresponding features as well as At the same time, you can get network and Features at different levels obtained in different convolutional layers Then the Euclidean distance between the features of two images at the same level can be calculated as follows:
[0032]
[0033]
[0034]
[0035] Furthermore, by adding the distances between the three different levels in a weighted summation manner, the overall loss of the feature can be obtained as follows:
[0036] L feat =αD E0 +βD E1 +γD E2 (7)
[0037] The overall loss of the network is as follows:
[0038] L=L cross_entropy +L feat (8)
[0039] The present invention is further improved in that: the specific method of step 6 is:
[0040] for Images in Its features can be obtained through the feature extraction network The subspace corresponding to each category has been obtained from the previous step 4.4, so we further calculate the subspace of each image in the query set The distance between a vector and a subspace can be obtained by the least squares method. Let the vector be β, then the distance between the vector and the subspace L(α1,α2,...,α s ) can be obtained by solving the following formula:
[0041]
[0042] Where X is the distance from the vector to the subspace.
[0043] By comparing the distances from the query set image to each category, the minimum distance can be used to determine The category to which it belongs.
[0044] Beneficial effects of the present invention:
[0045] 1. By reducing the dimension of features to subspace, the amount of computation in network training and application is reduced.
[0046] 2. Use general cross validation (GCV) to estimate the optimal dimension reduction m, making the network adaptive and improving the accuracy and generalization ability of the network at the same time.
[0047] 3. The improved loss function improves the network's ability to utilize limited information in small sample cases and provides more guidance for network training.
[0048] 4. In the process of general cross-validation, the first-order approximation is used to greatly reduce the amount of calculation and improve the computational efficiency of the network. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] Figure 1It is an overall flow chart of the small sample recognition method based on dimension adaptation provided by the present invention;
[0050] Figure 2 It is a specific flow chart of the small sample recognition method based on dimension adaptation provided by the present invention;
[0051] Figure 3 It is a demonstration diagram of the data enhancement technology used in the present invention;
[0052] Figure 4 It is the experimental result of the present invention on the Omniglot dataset;
[0053] Figure 5 These are the experimental results of the present invention on the mini-ImageNet dataset. DETAILED DESCRIPTION
[0054] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. It should be noted that the words "front", "rear", "left", "right", "upper" and "lower" used in the following description refer to directions in the accompanying drawings, and the words "inner" and "outer" refer to directions toward or away from the geometric center of a specific component, respectively.
[0055] like Figure 1-3 As shown, the steps of this embodiment are a small sample image classification method based on dimension adaptation.
[0056] Step 1: Preprocess the images in the training set data and the test set data, including image cropping, normalization, and rotation. At the same time, perform data enhancement on the preprocessed images, and expand the number of samples based on the existing small sample data;
[0057] The specific method of step 1 is:
[0058] Since the image sizes in the public dataset ImageNet are not uniform, this method will first preprocess the images in the dataset. First, the image length and width are uniformly cropped to C*W*H, where C is the number of channels of the image, and W and H are the pixel values of the image. In this method, C is 3, and W and H are both set to 84. Then, each image is normalized to further improve the training performance. In normalization, the mean and standard deviation of each channel used are [0.485, 0.456, 0.406], [0.229, 0.224, 0.225]; then the image is rotated at different angles; the preprocessed image is a standard image.
[0059] Step 2: Divide the preprocessed images by image category and use the task as the basic input unit for network training;
[0060] The specific method of step 2 is:
[0061] For the preprocessed images, this method divides the dataset into training set D base and the test set D novel , where the image categories contained in the training set and the test set are unrelated. Then D base The images in are further divided into several support sets and query sets, where the support set and the query set have the same image categories and the same number of categories;
[0062] Let each task in the training set The support set Query Set x and y represent the image and its corresponding label respectively, where the support set contains N categories, and each category contains K images;
[0063] Let each task in the test set The support set Query Set x and y represent the image and the label corresponding to the image respectively, where the support set contains N categories, and each category contains K pictures.
[0064] Step 3: Process the image through a convolutional neural network to extract the features of each image in the same category, and then obtain representative and recognizable category features;
[0065] The specific method of step 3 is:
[0066] The task Input into the network, the support set Each image in is input into the feature extraction network In this paper, we can get a set of image features Then calculate the mean of the feature and subtract the mean μ from each sample to get
[0067] Step 4: Calculate the corresponding adaptive dimension according to the category features obtained in step 2, and obtain the adaptive feature subspace corresponding to each category of images accordingly;
[0068] The specific method of step 4 is:
[0069] Step 4.1: By By performing singular value decomposition SVD, we can get the left singular matrix U of this set of samples. s , the right singular value matrix V s And the singular value matrix ∑ s , where the main diagonal of the singular value matrix contains all the eigenvalues, and the eigenvectors in the singular value matrix are similar to the singular value matrix ∑ s There is a one-to-one correspondence between the singular values in . The singular value decomposition process is as follows:
[0070]
[0071] Step 4.2: Arrange the decomposed singular values in descending order, select the first m singular values and the first m vectors of the corresponding left singular value matrix as the projection transformation matrix projected to the m-dimensional subspace;
[0072] Step 4.3: Determine the specific value of m by using the general cross validation (GCV) method to minimize the mean square error of the predicted value. Let X = {x1, x2, ..., x n} is a set of data with n samples, each sample has c dimensions (c ≥ m). Singular value decomposition is used to reduce it to m dimensions and then reconstruct it to c dimensions, and we get The mean square error is as follows:
[0073]
[0074] Furthermore, the first-order approximation formula of formula (2) can be used to simplify the calculation complexity as follows:
[0075]
[0076] By minimizing GCV, we can get the optimal dimension m, which enables the network to have better generalization ability while retaining the original information;
[0077] Step 4.4: Perform dimensionality reduction operations on the features of the images in the support set according to the corresponding dimension m calculated for each type of image; sort the singular values in descending order, and take the first m singular values in the left singular matrix U s The corresponding vector is taken as the subspace after dimensionality reduction.
[0078] Step 5: Improve the loss function and add constraints on high-dimensional and low-dimensional features to the loss function to provide more information and guidance for network training; that is, calculate and minimize the distance of the features of the middle layer of the network in three different dimensions for sample images of the same category; finally, take the weighted sum of the distances in the three dimensions as part of the overall loss function, and use the gradient descent method to update the network parameters;
[0079] The specific method of step 5 is:
[0080] On the basis of the original cross entropy loss function, constraints on the features of different layers of the network are added to provide more information and guidance for network training. Query Set For For each image in Then we can get the corresponding features as well as At the same time, you can get network and Features at different levels obtained in different convolutional layers Then the Euclidean distance between the features of two images at the same level can be calculated as follows:
[0081]
[0082]
[0083]
[0084] Furthermore, by adding the distances between the three different levels in a weighted summation manner, the overall loss of the feature can be obtained as follows:
[0085] L feat =αD E0 +βD E1 +γD E2 (7)
[0086] The overall loss of the network is as follows:
[0087] L=L cross_entropy +L feat (8).
[0088] Step 6: Input the image to be recognized into the feature extraction network to obtain the corresponding image features, and then calculate the distance between the feature and each feature subspace. The recognition result is determined as the category of the subspace closest to the image feature.
[0089] The specific method of step 6 is:
[0090] for Images in Its features can be obtained through the feature extraction network The subspace corresponding to each category has been obtained from the previous step 4.4, so we further calculate the subspace of each image in the query set The distance between the vector and the subspace can be obtained by the least squares method. Let the vector be β, then the distance between the vector and the subspace L(α1,α2,...,α s) can be obtained by solving the following formula:
[0091]
[0092] Where X is the distance from the vector to the subspace;
[0093] By comparing the distances from the query set image to each category, the minimum distance can be used to determine The category to which it belongs.
[0094] Based on the dimension-adaptive small sample image classification method, the residual network framework is used to realize the recognition task of small sample images. The experiment selected ConvNet-4 and ResNet-12 frameworks as the backbone network in the first step of image feature extraction. In the training of the network, the Omniglot and mini-ImageNet datasets were used to train the network. The Omniglot dataset contains 1623 handwritten text categories, each category contains 20 images; the samples in mini-ImageNet are selected from a part of ImageNet, which contains a total of 100 categories, each of which contains 600 images. This method uses 64 categories in the training set, and the validation set and test set use another 16 and 20 categories respectively. The irrelevance between categories simulates the situation where there is no large dataset available for training in the case of small samples.
[0095] Experiment: Based on the ConvNet-4 and ResNet-12 models, this method adds an improved loss function for dimensional constraints at each level and a classifier based on dimensional adaptive subspace, and shows good performance in experiments on the Omniglot and mini-ImageNet datasets, verifying the effectiveness of this method.
[0096] The technical means disclosed in the scheme of the present invention are not limited to the technical means disclosed in the above-mentioned implementation mode, but also include technical schemes composed of any combination of the above technical features.
Claims
1. A small sample image classification method based on dimension adaptation, characterized by: Step 1: Preprocess the images in the training set data and the test set data, including image cropping, normalization, and rotation; perform data enhancement on the preprocessed images, and expand the number of samples based on the existing small sample data; Step 2: Divide the preprocessed images by image category and use the task as the basic input unit for network training; Step 3: Process the image through a convolutional neural network to extract the features of each image in the same category, and then obtain representative and recognizable category features; Step 4: Calculate the corresponding adaptive dimension according to the category features obtained in step 2, and obtain the adaptive feature subspace corresponding to each category of images accordingly; the specific method of step 4 is: Step 4.1: By By performing singular value decomposition SVD, we can get the left singular matrix U of this set of samples. s , the right singular value matrix V s And the singular value matrix ∑ s , where the main diagonal of the singular value matrix contains all the eigenvalues, and the eigenvectors in the singular value matrix are similar to the singular value matrix ∑ s There is a one-to-one correspondence between the singular values in; the singular value decomposition process is as follows: Step 4.2: Arrange the decomposed singular values in descending order, select the first m singular values and the first m vectors of the corresponding left singular value matrix as the projection transformation matrix projected to the m-dimensional subspace; Step 4.3: Determine the specific value of m through the general cross-validation method to minimize the mean square error of the predicted value; let X = {x1, x2, ..., x n } is a set of data with n samples, each sample has c dimensions, where c ≥ m; after using singular value decomposition to reduce it to m dimensions and then reconstructing it to c dimensions, we get The mean square error is as follows: Furthermore, the first-order approximation formula of formula (2) can be used to simplify the calculation complexity as follows: By minimizing GCV, we can get the optimal dimension m, which enables the network to have better generalization ability while retaining the original information; Step 4.4: Perform dimensionality reduction operation on the features of the images in the support set according to the corresponding dimension m calculated for each type of image; that is, arrange the singular values in descending order and take the first m singular values in the left singular matrix U s The corresponding vector is used as the subspace after dimensionality reduction; Step 5: Improve the loss function and add constraints on high-dimensional and low-dimensional features to the loss function to provide more information and guidance for network training; that is, calculate and minimize the distance of the features of the middle layer of the network in three different dimensions for sample images of the same category; finally, take the weighted sum of the distances in the three dimensions as part of the overall loss function, and use the gradient descent method to update the network parameters; Step 6: Input the image to be recognized into the feature extraction network to obtain the corresponding image features, and then calculate the distance between the feature and each feature subspace. The recognition result is determined as the category of the subspace closest to the image feature.
2. The small sample image classification method based on dimension adaptation according to claim 1 is characterized in that: The specific method of step 1 is: Since the image sizes in the public dataset ImageNet are not uniform, this method will first preprocess the images in the dataset. First, the image length and width are uniformly cropped to C*W*H, where C is the number of channels of the image, and W and H are the pixel values of the image. In this method, C is 3, and W and H are both set to 84. Then, each image is normalized to further improve the training performance. In normalization, the mean and standard deviation of each channel used are [0.485, 0.456, 0.406], [0.229, 0.224, 0.225]; then the image is rotated at different angles; the preprocessed image is a standard image.
3. The small sample image classification method based on dimension adaptation according to claim 1 is characterized in that: The specific method of step 2 is: For the preprocessed images, this method divides the dataset into training set D base and the test set D novel , where the image categories contained in the training set and the test set are irrelevant; then D base The images in are further divided into several support sets and query sets, where the support set and the query set have the same image categories and the same number of categories; Let each task in the training set The support set Query Set x and y represent the image and its corresponding label respectively, where the support set contains N categories, and each category contains K images; Let each task in the test set The support set Query Set x and y represent the image and the label corresponding to the image respectively, where the support set contains N categories, and each category contains K pictures.
4. The small sample image classification method based on dimension adaptation according to claim 1 is characterized in that: The specific method of step 3 is: Task T i b Input into the network, the support set Each image in is input into the feature extraction network In this paper, we can get a set of image features Then calculate the mean of the feature and subtract the mean μ from each sample to get 5. The small sample image classification method based on dimension adaptation according to claim 1 is characterized in that: The specific method of step 5 is: On the basis of the original cross entropy loss function, constraints on the features of different layers of the network are added to provide more information and guidance for network training; let the support set in the training set Query Set For For each image in Then we can get the corresponding features as well as At the same time, you can get network and Features at different levels obtained in different convolutional layers Then the Euclidean distance between the features of two images at the same level can be calculated as follows: Furthermore, by adding the distances between the three different levels in a weighted summation manner, the overall loss of the feature can be obtained as follows: L feat =αD E0 +βD E1 +γD E2 (7) The overall loss of the network is as follows: L=L cross_entropy +L feat (8) 6. The small sample image classification method based on dimension adaptation according to claim 1 is characterized in that: The specific method of step 6 is: for Images in Through the feature extraction network, we can get Its characteristics The subspace corresponding to each category has been obtained from the previous step 4.4, so we further calculate the subspace of each image in the query set The distance between the vector and the subspace can be obtained by the least squares method. Let the vector be β, then the distance between the vector and the subspace L(α1,α2,...,α s ) can be obtained by solving the following formula: Where X is the distance from the vector to the subspace; By comparing the distances from the query set image to each category, the minimum distance can be used to determine The category to which it belongs.
