A training method for image classification model and image classification method

Abstract

The embodiment of the present invention provides an image classification model training method, which includes: S1, obtaining an original image data set including multiple image samples and its corresponding feature matrix; S2, performing dimensionality reduction processing on the feature matrix corresponding to the original image data set, and Gaussian modeling on the feature matrix after dimensionality reduction processing to obtain the Gaussian distribution of each eigenvalue in the feature matrix; S3, based on the Gaussian distribution of each eigenvalue in step S2, Gaussian sampling is performed on it in sequence, so that all eigenvalues ​​obtained by Gaussian sampling constitute a new feature matrix, and a new image data set is generated based on the new feature matrix; S4, using the new image data set and the original image data set to form an enhanced image training set to train the image classification model until convergence. This method improves the performance of the image classification model in the case of few samples.

Description

Technical Field

[0001] The present invention relates to the field of machine learning, and more specifically, to the field of image classification based on a training model of a small number of sample image training sets, and more specifically, to a training method for an image classification model and an image classification method. Background Art

[0002] In recent years, deep learning methods have demonstrated excellent performance in various pattern analysis and machine intelligence fields. At the same time, how to develop deep learning models with excellent performance in few-sample scenarios has attracted widespread attention and research due to its wide application needs. At present, the technologies used to solve this problem can be roughly divided into two categories: the first category is based on the improvement of neural network models and training methods; the second category is based on model training technology based on artificial synthetic data. The following is a detailed introduction to the two types of data enhancement technologies.

[0003] The first category of technologies mainly includes: model improvement technology based on transfer learning, model improvement technology based on contrastive learning, and model improvement technology based on big data pre-training. Among them, the model improvement technology based on transfer learning transfers effective prior knowledge from other related fields to the current few-sample data set to improve the problem of less effective information provided by the few-sample data set; the model improvement technology based on contrastive learning mines the high-level semantic features between the current few-sample data set and data from different fields, and learns through the strategy of stretching the distance between classes; the model improvement technology based on big data pre-training is based on a model trained on a large data set with rich existing modalities, and uses model fine-tuning, knowledge distillation and other technologies to make this model suitable for the current few-sample data. The common point of these technologies is to use additional information to improve the generalization performance of the model in the case of few samples, which usually requires additional data sets and model training steps.

[0004] The second category of technologies mainly includes: data enhancement technology based on geometric transformation, data enhancement technology based on resampling, and data enhancement technology based on deep generative models. Among them, data enhancement technology based on geometric changes is very simple and universal, and it can effectively expand the training data by scaling, rotating, and performing some other basic transformations; data enhancement technology based on resampling constructs new samples by using neighbor relationships in the low-dimensional embedding of data; data enhancement technology based on deep generative models generates training samples that approximate the distribution of a small number of sample data sets through deep generative models such as generative adversarial networks, variational autoencoders, and diffusion generative models. The common point of these technologies is to generate data similar to existing training data to increase the effective information content of the training data.

[0005] In the prior art, technologies based on improved neural network models and training methods usually require additional model training steps, which increases the burden of model development and deployment. The existing model training technologies based on artificial synthetic data have the following three main defects: 1) Data enhancement technologies based on geometric transformation are relatively simple in form and cannot significantly increase the effective information content of the data set (i.e., weak representation ability); 2) Data enhancement technologies based on resampling have unstable sampling methods when the data dimension is high, and require a large amount of calculation (i.e., high computational complexity); 3) Data enhancement technologies based on deep generative models require additional training of deep generative networks, which increases the difficulty of model development and deployment.

[0006] In summary, when training an image classification model based on a small number of image samples, using existing data augmentation technology to perform data augmentation on image data to develop an image classification model with excellent performance, there are problems such as weak representation ability of the enhanced image data, high technical computational complexity of the enhanced image data, and the need for additional model training steps, which increases the difficulty of model development and deployment. As a result, it is difficult to train an image classification model with excellent performance in all aspects using the enhanced image data. Summary of the invention

[0007] Therefore, the purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide an image classification model training method and an image classification method.

[0008] The objective of the present invention is achieved through the following technical solutions:

[0009] According to a first aspect of the present invention, there is provided a method for training an image classification model, wherein the image classification model is used to classify an input image based on image features, wherein the method comprises:

[0010] S1. Obtaining an original image data set including a plurality of image samples and a corresponding feature matrix thereof;

[0011] S2, performing dimensionality reduction processing on the feature matrix corresponding to the original image data set, and performing Gaussian modeling on the feature matrix after dimensionality reduction processing to obtain the Gaussian distribution of each eigenvalue in the feature matrix;

[0012] S3, performing Gaussian sampling on each eigenvalue in step S2 in sequence based on its Gaussian distribution, so that all eigenvalues ​​obtained by Gaussian sampling constitute a new feature matrix, and generating a new image data set based on the new feature matrix;

[0013] S4. Use the new image dataset and the original image dataset to form an enhanced image training set to train the image classification model until convergence.

[0014] In some embodiments of the present invention, in step S2, the feature matrix corresponding to the original image data set is subjected to dimensionality reduction processing in the following manner:

[0015] S21, performing decentralized processing on the feature matrix corresponding to the original image data set;

[0016] S22, decomposing the feature matrix corresponding to the decentralized original image data set to obtain a left singular matrix, a diagonal matrix and a right singular matrix, wherein the left singular matrix has a plurality of left singular eigenvectors, the diagonal matrix has a plurality of diagonal eigenvectors, each of the diagonal eigenvectors contains a diagonal non-zero eigenvalue, the right singular matrix has a plurality of right singular eigenvectors, and the left singular eigenvectors, the diagonal eigenvectors and the right singular eigenvectors have a one-to-one correspondence in position;

[0017] S23, selecting a preset number of diagonal eigenvectors from the diagonal matrix to form a reduced-dimensional diagonal matrix, the left singular eigenvectors corresponding to the selected diagonal eigenvectors form a reduced-dimensional left singular matrix, and the corresponding right singular eigenvectors form a reduced-dimensional right singular matrix;

[0018] S24. Based on the reduced dimensionality left singular matrix, the reduced dimensionality diagonal matrix and the reduced dimensionality right singular matrix, a feature matrix after dimensionality reduction processing is formed.

[0019] In some embodiments of the present invention, in step S21, the feature matrix corresponding to the original image data set is decentralized in the following manner:

[0020] The feature vector corresponding to each image sample in the feature matrix of the original image data set is obtained, the average value of the feature vectors of all image samples is calculated, and the feature matrix corresponding to the original image data set is decentralized based on the average value.

[0021] In some embodiments of the present invention, in step S22, a singular value decomposition method is used to decompose the feature matrix corresponding to the decentralized original image data set to obtain a left singular matrix, a diagonal matrix and a right singular matrix.

[0022] In some embodiments of the present invention, in step S2, Gaussian modeling is performed on the feature matrix after dimensionality reduction in the following manner to obtain the Gaussian distribution of each eigenvalue in the feature matrix:

[0023] S25, selecting a preset number of image samples from the original image data set multiple times, wherein the preset number of image samples selected each time is a sub-data set, and each sub-data set corresponds to a sub-feature matrix;

[0024] S26, performing decentralized processing on each sub-feature matrix;

[0025] S27, decomposing each decentralized sub-feature matrix to obtain a sub-left singular matrix, a sub-diagonal matrix, and a sub-right singular matrix;

[0026] S28, selecting a preset number of sub-diagonal eigenvectors in each sub-diagonal matrix to form a corresponding reduced-diagonal matrix;

[0027] S29. Perform Gaussian modeling on the diagonal non-zero eigenvalues ​​at the same position in all reduced-diagonal matrices to obtain the Gaussian distribution of the diagonal non-zero eigenvalues ​​at that position in the reduced-diagonal matrix.

[0028] In some embodiments of the present invention, in step S26, each sub-feature matrix is ​​decentralized in the following manner:

[0029] The eigenvectors corresponding to all image samples in each sub-feature matrix are obtained, the average value corresponding to the eigenvectors of all image samples in each sub-feature matrix is ​​calculated, and each sub-feature matrix is ​​decentralized based on the corresponding average value.

[0030] In some embodiments of the present invention, in step S27, a singular value decomposition method is used to decompose each decentralized sub-eigenmatrix to obtain a sub-left singular matrix, a sub-diagonal matrix and a sub-right singular matrix.

[0031] In some embodiments of the present invention, in step S29, the mean and variance of the Gaussian distribution of each diagonal non-zero eigenvalue in the dimension-reduced diagonal matrix are calculated in the following manner to implement Gaussian modeling:

[0032]

[0033]

[0034] i∈(1,k),h∈(1,l)

[0035] Among them, μ i,* represents the mean of the Gaussian distribution of the i-th diagonal non-zero eigenvalue in the reduced-dimensional diagonal matrix, λ i represents the i-th diagonal non-zero eigenvalue in the reduced-dimensional diagonal matrix, represents the i-th diagonal non-zero eigenvalue in the h-th reduced-diagonal matrix, represents the variance of the Gaussian distribution of the i-th diagonal non-zero eigenvalue in the reduced-dimensional diagonal matrix, is the preset variance, th is the preset threshold parameter, k is the number of diagonal non-zero eigenvalues, and l is the number of reduced-diagonal matrices.

[0036] In some embodiments of the present invention, in step S3, a new image data set is generated in the following manner:

[0037] S31, performing Gaussian sampling on the Gaussian distribution of each diagonal non-zero eigenvalue in the dimension-reduced diagonal matrix to obtain a sampling value at each diagonal non-zero eigenvalue position;

[0038] S32, replacing the diagonal non-zero eigenvalues ​​at the corresponding positions in the reduced-dimensional diagonal matrix with the sampled values ​​at each diagonal non-zero eigenvalue position to obtain a new diagonal matrix;

[0039] S33, forming an initial characteristic matrix based on the new diagonal matrix, the reduced-dimensional left singular matrix and the reduced-dimensional right singular matrix;

[0040] S34, adding the average value of the feature vectors of all image samples in the feature matrix corresponding to the original image data set to the initial feature matrix to generate a new feature matrix, and generating a new image data set based on the new feature matrix.

[0041] In some embodiments of the present invention, step S31 includes:

[0042] The Gaussian sampling is performed on the Gaussian distribution of each diagonal non-zero eigenvalue in the reduced-dimensional diagonal matrix by adopting a descending sampling method to obtain a sampling value of each diagonal non-zero eigenvalue position. The descending sampling method refers to sequentially performing Gaussian sampling on the Gaussian distribution of each diagonal non-zero eigenvalue in the reduced-dimensional diagonal matrix and making the sampling value of the subsequent diagonal non-zero eigenvalue smaller than the sampling value of the previous diagonal non-zero eigenvalue.

[0043] In some embodiments of the present invention, in step S31, Gaussian sampling is performed on a preset number of diagonal non-zero eigenvalues ​​that are ranked first in the following manner:

[0044] Gaussian sampling is performed within a preset range of the Gaussian distribution of each diagonal non-zero eigenvalue to obtain a sampling value of the diagonal non-zero eigenvalue.

[0045] According to a second aspect of the present invention, there is provided an image classification method, the method comprising:

[0046] Get the image to be classified;

[0047] The image classification model obtained by the training method described in the above embodiment is used to classify the image to be classified.

[0048] According to a third aspect of the present invention, there is provided an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the electronic device implements the steps of the method described in the first and second aspects.

[0049] Compared with the prior art, the advantages of the present invention are:

[0050] The data enhancement method proposed in the above embodiment of the present invention can efficiently generate training samples with high similarity to the few-sample category (compared with the existing data enhancement technology based on deep generative models, the present invention does not require additional model training, which reduces the difficulty of model development and deployment; compared with the existing data enhancement technology based on geometric transformation, it has stronger representation capabilities, can better explore the diversity of data modalities, and increase the effective information amount of training data), which is used to expand the training data of the few-sample category to improve the accuracy of the image classification model, thereby improving the performance of the image classification model in the few-sample case. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The embodiments of the present invention are further described below with reference to the accompanying drawings, in which:

[0052] Figure 1 A flowchart of a training method for an image classification model according to an embodiment of the present invention;

[0053] Figure 2 This is a schematic diagram of data enhancement of the MINST dataset according to the data enhancement method of an embodiment of the present invention;

[0054] Figure 3 1 is a flow chart of a training method for an image classification model based on a MINST dataset according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail by specific embodiments below. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0056] As described in the background technology, when an image classification model is trained based on a small number of image samples, the existing enhancement technology is used to perform data enhancement on the image data to develop an image classification model with excellent performance. However, there are problems such as weak representation ability of the enhanced image data, high technical computational complexity of the enhanced image data, and the need for additional model training steps, which increases the difficulty of model development and deployment. As a result, it is difficult to train an image classification model with excellent performance in all aspects using the enhanced image data.

[0057] The inventor has found through research that the defects of the prior art in terms of weak representation capability of enhanced image data and high computational complexity of enhanced image data technology are caused by the lack of explicit modeling of the main components in the random mode of high-dimensional image data. Based on the technology of model training of artificial synthetic data, the inventor has studied the low-dimensional embedding theory and the non-parametric generation process and found that the defects of the prior art can be solved by embedding the existing image data set into a low-dimensional space, thereby proposing a new training scheme for image classification models. In summary, the scheme of the present invention uses the low-dimensional representation of the image data set as its main component, and uses Gaussian processes to model the distribution and structure of its main components (that is, by modeling, each eigenvalue in the image sample obeys a Gaussian distribution), and then performs Gaussian sampling based on the Gaussian distribution of each eigenvalue to obtain enhanced training data, so as to improve the performance of the image classification model in scenarios with a small number of image samples. The present invention is described in detail below in conjunction with the accompanying drawings and embodiments.

[0058] According to one embodiment of the present invention, Figure 1 As shown, it shows a flow chart of the training method of the image classification model of the present invention, and the flow chart includes steps S1-S4. Each step is described in detail below.

[0059] In step S1, an original image data set including a plurality of image samples and a corresponding feature matrix are obtained, wherein the feature matrix is ​​composed of feature vectors of the plurality of image samples.

[0060] Schematically, the feature matrix corresponding to the original image dataset is X = {x1, x2, ..., x n}, where 1≤p≤n, n is the number of image samples, x p is the feature vector of the pth image sample.

[0061] In step S2, the feature matrix corresponding to the original image data set is subjected to dimensionality reduction processing, and Gaussian modeling is performed on the feature matrix after dimensionality reduction processing to obtain the Gaussian distribution of each eigenvalue in the feature matrix.

[0062] According to an embodiment of the present invention, in step S2, steps S21-S24 are executed to reduce the dimension of the feature matrix corresponding to the original image data set. Steps S21-S24 are described in detail below.

[0063] In step S21, the feature matrix corresponding to the original image data set is decentralized. According to one embodiment of the present invention, when the feature matrix corresponding to the original image data set is decentralized, the feature vector corresponding to each image sample in the feature matrix of the original image data set is obtained, the average value of the feature vectors of all image samples is calculated, and the feature matrix corresponding to the original image data set is decentralized based on the average value.

[0064] Schematically, the feature matrix X corresponding to the original image dataset is X = {x1, x2, ..., x n}, the decentralized processing process is as follows:

[0065] Calculate the average value of all image sample feature vectors in the feature matrix corresponding to the original image dataset:

[0066]

[0067] The average value Used to decentralize the feature matrix corresponding to the original image dataset:

[0068]

[0069] From the above description, it can be seen that decentralizing the original image data set can improve the quality of image samples and enhance the features of image samples, thereby reducing the interference of noise and background on the effective features of image samples.

[0070] In the step S22, the feature matrix corresponding to the decentralized original image data set is decomposed to obtain a left singular matrix, a diagonal matrix and a right singular matrix, wherein the left singular matrix has a plurality of left singular eigenvectors, the diagonal matrix has a plurality of diagonal eigenvectors, each diagonal eigenvector contains a diagonal non-zero eigenvalue, the right singular matrix has a plurality of right singular eigenvectors, and the left singular eigenvectors, the diagonal eigenvectors and the right singular eigenvectors have a one-to-one correspondence in position. According to one embodiment of the present invention, a singular value decomposition (SVD) method is used to decompose the feature matrix corresponding to the decentralized original image data set to obtain a left singular matrix, a diagonal matrix and a right singular matrix.

[0071] Schematically, the singular value decomposition method is used to decompose the feature matrix X corresponding to the decentralized original image data set to obtain the left singular matrix, diagonal matrix and right singular matrix in the following form:

[0072] X=UΣV T , the rank of X is r

[0073] U=(u1,u2,…,u k …,u r )

[0074] Σ=diag(λ1,λ2,…,λ k …,λ r )

[0075] V=(v1,v2,…,vk …,v r )

[0076] Among them, U is an m×r dimensional left singular matrix, (u1,u2,…,u k ,…,u r ) are the r left singular eigenvectors in the left singular matrix U, Σ is an r×r dimensional diagonal matrix, (λ1,λ2,…,λ k ,…,λ r ) are the r diagonal non-zero eigenvalues ​​in the diagonal matrix Σ, V is an n×r dimensional right singular matrix, (v1,v2,…,v k ,…,v r ) are the r right singular eigenvectors in the right singular matrix V, and U T U=I r×r , V T V=I r×r , where I r×r is the r-dimensional identity matrix, λ1≥λ2≥…≥λ k ≥…≥λ r .

[0077] In step S23, a preset number of diagonal eigenvectors are selected from the diagonal matrix to form a reduced-dimensional diagonal matrix, and the left singular eigenvectors corresponding to the selected diagonal eigenvectors form a reduced-dimensional left singular matrix, and the corresponding right singular eigenvectors form a reduced-dimensional right singular matrix.

[0078] In step S24, a feature matrix after dimensionality reduction processing is composed based on the reduced dimensionality left singular matrix, the reduced dimensionality diagonal matrix and the reduced dimensionality right singular matrix.

[0079] Schematically, the present invention takes the selection of k diagonal eigenvectors in the diagonal matrix Σ as an example, and the eigenmatrix after the dimensionality reduction process is:

[0080] X(k)=U(k)Σ(k)V(k) T , the rank of X(k) is k

[0081] U(k)=(u1,u2,…,u k )

[0082] Σ(k)=diag(λ1,λ2,…,λ k )

[0083] V(k)=(v1,v2,…,v k )

[0084] Among them, U(k) is a reduced dimension left singular matrix of m×k dimensions, (u1,u2,…,u k) are the k left singular eigenvectors in the reduced dimension left singular matrix U(k), Σ(k) is a k×k dimensional reduced dimension diagonal matrix, (λ1,λ2,…,λ k ) are the k diagonal non-zero eigenvalues ​​in the reduced-diagonal matrix Σ(k), V(k) is the n×k-dimensional reduced-diagonal right singular matrix, (v1,v2,…,v k ) are the k right singular eigenvectors in the reduced-dimensional right singular matrix V(k).

[0085] According to an embodiment of the present invention, in step S2, steps S25-S29 are executed to perform Gaussian modeling on the feature matrix after dimensionality reduction to obtain the Gaussian distribution of each eigenvalue in the feature matrix. Steps S25-S29 are described in detail below.

[0086] In the step S25, a preset number of image samples are selected multiple times from the original image data set, and the preset number of image samples selected each time is a sub-data set, and each sub-data set corresponds to a sub-feature matrix.

[0087] Schematically, q image samples are randomly selected with replacement from the original image data set, and the extraction process is repeated l times independently, thereby obtaining l m×q dimensional sub-feature matrices X1, X2, ..., X h ,…,X l . Take the h-th sub-feature matrix X h For example, the sub-feature matrix X h ={x1,x2,…,x q}, where 1≤j≤q, q is the number of image samples, x j is the feature vector of the jth image sample. The composition of other sub-feature matrices is the same as X h Similarly, I won’t go into details here.

[0088] In step S26, each sub-feature matrix is ​​decentralized. According to one embodiment of the present invention, when each sub-feature matrix is ​​decentralized, the feature vectors corresponding to all image samples in each sub-feature matrix are obtained, the average value corresponding to the feature vectors of all image samples in each sub-feature matrix is ​​calculated, and each sub-feature matrix is ​​decentralized based on the corresponding average value.

[0089] Schematically, for the h-th sub-feature matrix X h ={x1,x2,…,x q The process of decentralization is as follows:

[0090] Calculate the h-th sub-feature matrix X h The average value corresponding to the feature vectors of all image samples in :

[0091]

[0092] The average value Decentralize the h-th sub-feature matrix:

[0093]

[0094] In step S27, each decentralized sub-eigenmatrix is ​​decomposed to obtain a sub-left singular matrix, a sub-diagonal matrix, and a sub-right singular matrix. According to one embodiment of the present invention, a singular value decomposition method is used to decompose each decentralized sub-eigenmatrix to obtain a sub-left singular matrix, a sub-diagonal matrix, and a sub-right singular matrix.

[0095] Schematically, the h-th sub-feature matrix X after decentralization is decomposed into h Decomposition is performed to obtain the following sub-left singular matrix, sub-diagonal matrix, and sub-right singular matrix:

[0096] X h =U h Σ h V h T , the rank of X is r

[0097]

[0098]

[0099]

[0100] Among them, U h is the hth m×r-dimensional left singular sub-matrix, is the hth left singular sub-matrix U h The r left singular eigenvectors in Σ h is the hth r×r-dimensional subdiagonal matrix, is the hth subdiagonal matrix Σ h The r diagonal non-zero eigenvalues ​​in V h is the hth q×r-dimensional sub-right singular matrix, is the hth sub-right singular matrix V h The r sub-right singular eigenvectors in (k).

[0101] In step S28, a preset number of sub-diagonal eigenvectors are selected from each sub-diagonal matrix to form a corresponding reduced-dimensional sub-diagonal matrix.

[0102] Schematically, the present invention uses the h-th sub-diagonal matrix Σ hTake k sub-diagonal eigenvectors as an example, the sub-left singular eigenvectors corresponding to the selected k sub-diagonal eigenvectors form a reduced-dimension sub-left singular matrix, and the corresponding k sub-right singular eigenvectors form a reduced-dimension sub-right singular matrix, as follows:

[0103] X h (k) = U h (k)Σ h (k)V h (k) T , X h The rank of (k) is k

[0104]

[0105]

[0106]

[0107] Among them, U h (k) is the hth m×k-dimensional reduced-dimension left singular matrix, is the hth reduced-dimension left singular matrix U h The k left singular eigenvectors in (k), Σ h (k) is the hth k×k-dimensional reduced subdiagonal matrix, is the h-th dimension-reduced subdiagonal matrix Σ h (k) has k non-zero diagonal eigenvalues, V h (k) is the hth q×k-dimensional reduced right singular matrix, is the hth reduced-dimension right singular matrix V h The k right singular eigenvectors in (k).

[0108] In step S29, Gaussian modeling is performed on the diagonal non-zero eigenvalues ​​at the same position in all reduced-diagonal matrices to obtain the Gaussian distribution of the diagonal non-zero eigenvalues ​​at the position in the reduced-diagonal matrix. According to one embodiment of the present invention, the following method is performed to calculate the mean and variance of the Gaussian distribution of each diagonal non-zero eigenvalue in the reduced-diagonal matrix to achieve Gaussian modeling:

[0109]

[0110]

[0111] i∈(1,k),h∈(1,l)

[0112] Among them, μ i* represents the mean of the Gaussian distribution of the i-th diagonal non-zero eigenvalue in the reduced-dimensional diagonal matrix, λ i represents the i-th diagonal non-zero eigenvalue in the reduced-dimensional diagonal matrix, represents the i-th diagonal non-zero eigenvalue in the h-th reduced-diagonal matrix, represents the variance of the Gaussian distribution of the i-th diagonal non-zero eigenvalue in the reduced-dimensional diagonal matrix, is the preset variance, th is the preset threshold parameter (th>0, and its function is to prevent the mode from being too concentrated due to too small variance), k is the number of diagonal non-zero eigenvalues, and l is the number of reduced-diagonal matrices.

[0113] Schematically, Gaussian modeling is achieved by calculating the mean and variance of the Gaussian distribution of k diagonal non-zero eigenvalues ​​in the reduced-dimensional diagonal matrix Σ(k), and the Gaussian distribution of the diagonal non-zero eigenvalues ​​at that position in the reduced-dimensional diagonal matrix Σ(k) is obtained as follows:

[0114]

[0115] From the above description, it can be seen that by combining the Monte Carlo method (i.e., randomly selecting multiple image samples from the image dataset with replacement, and repeating this extraction process independently for each time), the low-dimensional representation of the image dataset is used as its main component. By modeling the complex data augmentation process as a Gaussian process about the principal component, it has the characteristics of high efficiency, high fidelity and high diversity.

[0116] In step S3, Gaussian sampling is performed on each eigenvalue in step S2 based on its Gaussian distribution, so that the Gaussian sampling of all eigenvalues ​​constitutes a new feature matrix, and a new image data set is generated based on the new feature matrix.

[0117] According to an embodiment of the present invention, in step S3, steps S31-S34 are executed to generate a new image data set. Steps S31-S34 are described in detail below.

[0118] In the step S31, Gaussian sampling is performed on the Gaussian distribution of each diagonal non-zero eigenvalue in the reduced-diagonal matrix to obtain the sampling value of each diagonal non-zero eigenvalue position. According to one embodiment of the present invention, Gaussian sampling is performed on the Gaussian distribution of each diagonal non-zero eigenvalue in the reduced-diagonal matrix to obtain the sampling value of each diagonal non-zero eigenvalue position by using a descending sampling method, and the descending sampling method refers to sequentially performing Gaussian sampling on the Gaussian distribution of each diagonal non-zero eigenvalue in the reduced-diagonal matrix and making the sampling value of the next diagonal non-zero eigenvalue less than the sampling value of the previous diagonal non-zero eigenvalue. According to one embodiment of the present invention, when Gaussian sampling is performed on a preset number of diagonal non-zero eigenvalues ​​that are ranked first, Gaussian sampling is performed within a preset range of the Gaussian distribution of each diagonal non-zero eigenvalue to obtain the sampling value of the diagonal non-zero eigenvalue.

[0119] Schematically, in the order 1≤i≤k, the i-th diagonal non-zero eigenvalue λ in the reduced-diagonal matrix Σ(k)i According to Gaussian distribution Sampling is performed, where the top C diagonal non-zero eigenvalues ​​are sampled by ±2σ i* The truncated Gaussian distribution sampling is used. Since the components corresponding to the larger diagonal non-zero eigenvalues ​​are more important, in order to ensure the similarity between the generated image dataset and the original image dataset, the larger the C, the closer the modality of the generated image dataset is to the original image dataset. In addition, when λ i+1 When sampling, multiple independent repeated samplings are performed until Where 1≤i≤k-1, we get

[0120] In step S32, the sample value at each diagonal non-zero eigenvalue position replaces the diagonal non-zero eigenvalue at the corresponding position in the reduced-dimensional diagonal matrix to obtain a new diagonal matrix.

[0121] Schematically, the new diagonal matrix is:

[0122]

[0123] In the step S33, an initial characteristic matrix is ​​composed based on the new diagonal matrix, the reduced-dimensional left singular matrix and the reduced-dimensional right singular matrix.

[0124] Schematically, the initial feature matrix is:

[0125]

[0126] In step S34, the average value of the feature vectors of all image samples in the feature matrix corresponding to the original image data set is added to the initial feature matrix to generate a new feature matrix, and a new image data set is generated based on the new feature matrix.

[0127] Schematically, the new feature matrix is:

[0128]

[0129] From the above description, it can be seen that the singular value decomposition and Monte Carlo method (i.e., randomly selecting image samples with replacement multiple times to form multiple sub-datasets) are used to simulate the Gaussian distribution (principal component distribution) of the image data set. While reducing the dimension of the image data set to extract the principal components, the potential randomness of the principal components is maintained, thereby overcoming the defects of high complexity and weak representation ability in existing data enhancement technologies (i.e., compared with existing data enhancement technologies based on deep generative models, the present invention does not require additional model training, which reduces the difficulty of model development and deployment; compared with existing data enhancement technologies based on geometric transformations, it has stronger representation capabilities, can better mine the diversity of data modalities, and increase the effective information content of training data).

[0130] In step S4, the new image data set and the original image data set are used to form an enhanced image training set to train the image classification model until convergence.

[0131] When training a deep neural network for image classification tasks, effective learning methods for few-sample categories are one of the research hotspots in this field. Therefore, in practical applications, based on the training method of the image classification model in the above embodiment, a corresponding image classification model is obtained. According to one embodiment of the present invention, an image classification method is proposed, which includes: obtaining an image to be classified; and classifying the image to be classified using the image classification model obtained by the training method described in the above embodiment.

[0132] In order to verify the effectiveness of the method of obtaining a new image dataset by performing data enhancement on the original image dataset through steps S2-S3 in the above embodiment of the present invention, the inventors conducted the following experiments:

[0133] The data enhancement method of steps S2-S3 in the above embodiment of the present invention is used to enhance the MNIST data set, and some of the effects are as follows: Figure 2 As shown. Figure 2 It can be seen that the left side is part of the image samples in the "number 2" class in the original MNIST, and the right side is the image samples of "number 2" generated by the method of the present invention. By comparison, it can be seen that the data enhancement method in the embodiment of the present invention, while ensuring that the main recognition attributes of the generated image sample class remain unchanged, generates image samples that are significantly different from the original image data in terms of morphology and contrast. Therefore, the data enhancement method proposed in the embodiment of the present invention effectively increases the diversity and difference of image samples while ensuring the recognition within the class, and can effectively improve the performance of the image classification model with small samples.

[0134] In order to verify whether the MINST dataset is enhanced by the data enhancement method of steps S2-S3 in the above embodiment of the present invention, and whether the new image dataset and the original image dataset are used to form an enhanced image training set for training the image classification model can improve the model performance, the inventors further conducted the following experiments:

[0135] like Figure 3As shown, the MINST dataset is data enhanced by the above embodiment of the present invention. That is, as can be seen from the figure, the data enhancement method of steps S2-S3 in the embodiment of the present invention is used to enhance the category 1 in the MINST dataset, and the enhanced dataset and the original dataset (the original dataset of category 1, the enhanced dataset, and the original dataset of category 2) are used to train the image classification model AlexNet. It should be noted that the image classification model AlexNet used in the present invention is only one embodiment, and other deep learning models other than AlexNet can also be used, which is not specifically limited here.

[0136] The experimental results are shown in Table 1. The accuracy of image classification model classification using the image data enhanced by the data enhancement method of steps S2-S3 in the above embodiment of the present invention reaches 94.5%, which is much better than no data enhancement (91.0%) and the traditional data enhancement method (92.4%). Therefore, after the traditional data enhancement method (for example, flipping, cropping and rotating) and the data enhancement method in the embodiment of the present invention are used to enhance the MNIST data set, they are used to train the image classification model respectively. The experimental results show that the image classification accuracy obtained by the image classification model training method in the above embodiment of the present invention is higher.

[0137] Table 1

[0138] Accuracy No data augmentation 91.0％ Traditional Data Augmentation 92.4％ Data enhancement of the present invention 94.5％

[0139] To sum up, the data enhancement method in the above embodiment of the present invention can efficiently generate training samples with high similarity to the few-sample category (compared with the existing data enhancement technology based on deep generative models, the present invention does not require additional model training, which reduces the difficulty of model development and deployment; compared with the existing data enhancement technology based on geometric transformation, it has stronger representation capabilities, can better mine the diversity of data modalities, and increase the effective information amount of training data), which is used to expand the training data of the few-sample category to improve the accuracy of the image classification model, thereby improving the performance of the image classification model in the few-sample case.

[0140] It should be noted that although the above describes the various steps in a specific order, it does not mean that the various steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently or even in a different order as long as the required functions can be achieved.

[0141] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0142] Computer readable storage medium can be a tangible device that holds and stores instructions used by an instruction execution device. Computer readable storage medium can include, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a protruding structure in a groove on which instructions are stored, and any suitable combination thereof.

[0143] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A training method for an image classification model, wherein the image classification model is used to classify an input image based on image features, characterized in that: The method comprises: S1. Obtaining an original image data set including a plurality of image samples and a corresponding feature matrix thereof; S2, performing dimensionality reduction processing on the feature matrix corresponding to the original image data set to obtain a feature matrix after dimensionality reduction processing consisting of a reduced dimensionality left singular matrix, a reduced dimensionality diagonal matrix, and a reduced dimensionality right singular matrix, and performing Gaussian modeling on the reduced dimensionality diagonal matrix to obtain a Gaussian distribution of each non-zero eigenvalue in the reduced dimensionality diagonal matrix; S3, performing Gaussian sampling on the Gaussian distribution of each non-zero eigenvalue in the reduced-dimensionality diagonal matrix obtained in step S2 in sequence, so that all eigenvalues ​​obtained by Gaussian sampling constitute a new reduced-dimensionality diagonal matrix, and constructing a new feature matrix based on the new reduced-dimensionality diagonal matrix to generate a new image data set; S4. Use the new image dataset and the original image dataset to form an enhanced image training set to train the image classification model until convergence.

2. The method according to claim 1, characterized in that In step S2, the feature matrix corresponding to the original image data set is subjected to dimensionality reduction processing in the following manner: S21, performing decentralized processing on the feature matrix corresponding to the original image data set; S22, decomposing the feature matrix corresponding to the decentralized original image data set to obtain a left singular matrix, a diagonal matrix and a right singular matrix, wherein the left singular matrix has a plurality of left singular eigenvectors, the diagonal matrix has a plurality of diagonal eigenvectors, each of the diagonal eigenvectors contains a diagonal non-zero eigenvalue, the right singular matrix has a plurality of right singular eigenvectors, and the left singular eigenvectors, the diagonal eigenvectors and the right singular eigenvectors have a one-to-one correspondence in position; S23, selecting a preset number of diagonal eigenvectors from the diagonal matrix to form a reduced-dimensional diagonal matrix, the left singular eigenvectors corresponding to the selected diagonal eigenvectors form a reduced-dimensional left singular matrix, and the corresponding right singular eigenvectors form a reduced-dimensional right singular matrix; S24. Based on the reduced dimensionality left singular matrix, the reduced dimensionality diagonal matrix and the reduced dimensionality right singular matrix, a feature matrix after dimensionality reduction processing is formed.

3. The method according to claim 2, characterized in that In step S21, the feature matrix corresponding to the original image data set is decentralized in the following manner: The feature vector corresponding to each image sample in the feature matrix of the original image data set is obtained, the average value of the feature vectors of all image samples is calculated, and the feature matrix corresponding to the original image data set is decentralized based on the average value.

4. The method according to claim 3, characterized in that: In the step S22, a singular value decomposition method is used to decompose the feature matrix corresponding to the decentralized original image data set to obtain a left singular matrix, a diagonal matrix and a right singular matrix.

5. The method according to claim 4, characterized in that In step S2, Gaussian modeling is performed in the following manner to obtain the Gaussian distribution of each non-zero eigenvalue in the reduced-dimensional diagonal matrix: S25, selecting a preset number of image samples from the original image data set multiple times, wherein the preset number of image samples selected each time is a sub-data set, and each sub-data set corresponds to a sub-feature matrix; S26, performing decentralized processing on each sub-feature matrix; S27, decomposing each decentralized sub-feature matrix to obtain a sub-left singular matrix, a sub-diagonal matrix, and a sub-right singular matrix; S28, selecting a preset number of sub-diagonal eigenvectors in each sub-diagonal matrix to form a corresponding reduced-diagonal matrix; S29. Perform Gaussian modeling on the diagonal non-zero eigenvalues ​​at the same position in all reduced-diagonal matrices to obtain the Gaussian distribution of the diagonal non-zero eigenvalues ​​at that position in the reduced-diagonal matrix.

6. The method according to claim 5, characterized in that In step S26, each sub-feature matrix is ​​decentralized in the following manner: The eigenvectors corresponding to all image samples in each sub-feature matrix are obtained, the average value corresponding to the eigenvectors of all image samples in each sub-feature matrix is ​​calculated, and each sub-feature matrix is ​​decentralized based on the corresponding average value.

7. The method according to claim 5, characterized in that In step S27, a singular value decomposition method is used to decompose each decentralized sub-eigenmatrix to obtain a sub-left singular matrix, a sub-diagonal matrix and a sub-right singular matrix.

8. The method according to claim 5, characterized in that In step S29, the mean and variance of the Gaussian distribution of each diagonal non-zero eigenvalue in the reduced-dimensional diagonal matrix are calculated in the following manner to implement Gaussian modeling: ， in, Represents the first The mean of a Gaussian distribution with diagonal non-zero eigenvalues, Represents the first diagonal non-zero eigenvalues, Indicates The first diagonal non-zero eigenvalues, Represents the first The variance of a Gaussian distribution with diagonal nonzero eigenvalues, is the preset variance, is the preset threshold parameter, is the number of non-zero eigenvalues ​​on the diagonal, is the number of reduced-diagonal sub-matrices.

9. The method according to claim 5, characterized in that In step S3, a new image data set is generated in the following manner: S31, performing Gaussian sampling on the Gaussian distribution of each diagonal non-zero eigenvalue in the dimension-reduced diagonal matrix to obtain a sampling value at each diagonal non-zero eigenvalue position; S32, replacing the diagonal non-zero eigenvalues ​​at the corresponding positions in the reduced-dimensional diagonal matrix with the sampled values ​​at each diagonal non-zero eigenvalue position to obtain a new diagonal matrix; S33, forming an initial characteristic matrix based on the new diagonal matrix, the reduced-dimensional left singular matrix and the reduced-dimensional right singular matrix; S34, adding the average value of the feature vectors of all image samples in the feature matrix corresponding to the original image data set to the initial feature matrix to generate a new feature matrix, and generating a new image data set based on the new feature matrix.

10. The method according to claim 9, characterized in that The step S31 comprises: The Gaussian sampling is performed on the Gaussian distribution of each diagonal non-zero eigenvalue in the reduced-dimensional diagonal matrix by adopting a descending sampling method to obtain a sampling value of each diagonal non-zero eigenvalue position. The descending sampling method refers to sequentially performing Gaussian sampling on the Gaussian distribution of each diagonal non-zero eigenvalue in the reduced-dimensional diagonal matrix and making the sampling value of the subsequent diagonal non-zero eigenvalue smaller than the sampling value of the previous diagonal non-zero eigenvalue.

11. The method according to claim 10, characterized in that In step S31, Gaussian sampling is performed on a preset number of diagonal non-zero eigenvalues ​​that are ranked first in the order as follows: Gaussian sampling is performed within a preset range of the Gaussian distribution of each diagonal non-zero eigenvalue to obtain a sampling value of the diagonal non-zero eigenvalue.

12. An image classification method, characterized in that: The method comprises: Get the image to be classified; An image classification model obtained by the training method as described in any one of claims 1 to 11 is used to classify the image to be classified.

13. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program can be executed by a processor to implement the steps of any method described in claims 1-11 and 12.

14. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the steps of any method as claimed in claims 1-11 and 12.

Images