An image classification method based on weakly supervised autoencoder

Abstract

An image classification method based on a weakly supervised autoencoder, specifically: classifying unlabeled images and... k A labeled reference image is input into the input layer of the autoencoder model; the mean squared error between the reconstructed output value and the original input value of the original autoencoder model is calculated as the mean squared error term; the sum of squared weights between the input layer and the hidden layer is calculated as the weight decay term; the energy function of the entire autoencoder model is calculated as the energy regularization term; the soft multi-label consistency similarity metric for discriminative vision is calculated as the soft multi-label learning term; the loss function of the autoencoder model is minimized using gradient descent within a specified number of iterations to obtain the optimal parameter weights and biases of the autoencoder model, and the autoencoder model is pre-trained; test data is input into the autoencoder model that has been adjusted to the optimal value, and then the features extracted from the hidden layer are input into the classifier for classification to obtain the classification result.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of deep learning and computer vision, and particularly relates to an image classification method based on a weakly supervised autoencoder. BACKGROUND

[0002] In recent years, deep learning has attracted much attention due to its ability to automatically learn data features. The research of deep learning has a profound influence on the development direction of machine learning. Auto-encoder (AE) is one of the important unsupervised algorithms in deep learning, which can extract useful features from a large amount of unlabeled data. The purpose of AE is to get the same expected output as the input data of the network as much as possible, so as to learn better abstract features. It has the characteristics of simple structure, simple training process and excellent generalization performance [Hinton GE, Salakhutdinov RR (2006) Reducing the Dimensionality of Data with Neural Networks. Science 313(5786): 504-507.]. Rumelhart et al. proposed the concept of auto-encoder [Rumelhart DE, Hinton GE, Williams RJ (1986) Learning Representations by Back Propagating Errors. Nature 323(6088): 533-536.], and Bourrard et al. explained the auto-encoder in detail [Bourlard H, Kamp Y (1988) Auto-association by multilayer perceptrons and singular value decomposition. Biological Cybernetics 59(4-5): 291-294.]. Recently, many scholars have proposed various improved algorithms of auto-encoder. In order to obtain high-dimensional and sparse abstract feature representation, Ng introduced sparse auto-encoder (SAE) by adding sparsity constraint on the hidden layer, which forced the network to use fewer neural nodes to extract effective features [Ng A (2011) Sparse auto-encoder. CS294A Lecture Notes 72(1): 1-19.].Vincent et al. proposed a denoising autoencoder (DAE) that uses noisy samples to reconstruct noise-free samples, making the extracted abstract features more robust [Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagoi PA (2010) Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion. Journal of Machine Learning Research 11(12):3371-3408.]. Rifai et al. invented a contractive autoencoder (CAE) that implements local spatial contraction to suppress small perturbations in input samples [Rifai S, Vincent P, Muller X, Glorot X, Bengio Y (2011) Contractive auto-encoders: Explicit invariance during feature extraction. Proceedings of the 28th International Conference on Machine Learning. Bellevue, WA, USA 833-840.]. Kingma et al. designed a variational autoencoder (VAE) that can serve as a generative model for high-dimensional complex data such as images [Nash C, Williams CKI (2017) The shape variational autoencoder: A deep generative model of part-segmented 3D objects. Computer Graphics Forum 36(5): 1-12].At present, AE and its improved algorithms are widely used in data classification [Lan R, Li Z, Liu Z, Gu T, Luo X (2018) Hyperspectral image classification using k-sparse denoising autoencoder and spectral-restricted spatial characteristics. Applied Soft Computing 74:693-708.], anomaly detection [Ribeiro M, Lazzaretti AE, Lopes HS (2017) A study of deep convolutional auto-encoders for anomaly detection in videos. Pattern Recognition Letters 105 (APR.1): 13-22.], pattern recognition [Lore KG, Akintayo A, Sarkar S (2017) LLNet: A Deep Autoencoder Approach to Natural Low-light Image Enhancement. Pattern Recognition 61:650-662.], and data generation [Xu J, Li H, Hou S (2022) Autoencoder-guided GAN for minority-class cloth-changing gait data generation. Digital Signal Processing 128:103608.] and other fields.

[0003] AE has good feature learning ability and can process a large amount of unlabeled data, saving manpower and resources. However, due to the lack of label information assistance, the learning process of AE is too difficult, and it completely relies on itself to learn, summarize and summarize. Therefore, due to its unsupervised learning characteristics, AE still has a certain gap between its performance in practical applications and supervised learning. Therefore, in order to improve the accuracy of feature extraction, supervised learning technology is constantly developing. However, the reality is that labeled data only accounts for a small part of explosive data, and due to the high cost of data annotation process, many tasks are difficult to obtain strong supervision information. To some extent, this situation restricts the application of supervised learning. Therefore, weakly supervised learning has become a future research direction for researchers.

[0004] Weakly supervised learning is mainly proposed for learning problems with incomplete or ambiguous supervised information. The purpose of weakly supervised learning is to use both unlabeled samples and a small amount of labeled samples to train the learning model, so that the learned model or model parameters are more accurate. Zhou briefly introduced weakly supervised learning and elaborated three types of weak supervision: incomplete supervision, inaccurate supervision, and inaccurate supervision [Zhou ZH (2018) A Brief Introduction to Weakly Supervised Learning. National Science Review 5: 44-53.]. Bilen et al. introduced a weakly supervised deep detection network, which modified the deep convolutional neural network to run at the image region level while performing region selection and classification [Bilen H, Vedaldi A (2016) Weakly Supervised Deep Detection Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2846-2854.]. Zhang et al. proposed a new weakly supervised learning model that added attention mechanisms and consistency regularization in the EfficientNet framework and used data augmentation techniques on the original data, which can make full use of these unlabeled data [Zhang XB, Li ZM, Zhang Q, Yin ZG, Lu ZJ, Li Y (2023) A new weakly supervised deep neural network for recognizing Alzheimer’s disease. Computers in Biology and Medicine 163: 107079.]. Although there have been some studies on weakly supervised learning methods, we found that there are few studies on weakly supervised learning methods for simple model structure autoencoders, and there is no discussion on the lower limit of the number of labels required for weakly supervised learning for classification.

[0005] The present application proposes a weakly supervised autoencoder WSAE based on k labeled samples, which mainly integrates soft multi-label learning and energy regularization terms. By comparing the features of k labeled samples and unlabeled samples with soft multi-label consistency, a standard function is constructed to force the intra-class distance to be small and the inter-class distance to be large, which makes the feature learning more discriminative. At the same time, an energy function is introduced to constrain the model to converge to a stable equilibrium state, which can better fit the data and optimize the feature learning ability of the network, thereby improving the classification accuracy. SUMMARY

[0006] Image classification is a hot topic in computer vision tasks. As a simple unsupervised network model, autoencoder can learn features and apply them to classification. However, due to the lack of prior knowledge of autoencoder, its performance in practical application has great limitations. On the contrary, although supervised learning can learn accurately under the guidance of accurate labels, it is difficult to obtain strong supervision information in many tasks due to the high cost of data annotation. In this paper, we propose a weakly supervised autoencoder (WSAE) through energy regularization and soft multi-label learning. Weakly supervised learning not only reduces the workload of manual annotation, but also introduces supervision information, greatly improving the performance of autoencoder. Specifically, we first explore the lower limit value of the training data that needs to be labeled in weakly supervised learning. Then the soft multi-label is used to judge the visual consistency of the labeled samples and the unlabeled samples, so as to effectively increase the difference between the features of different classes and reduce the change of the features within the class, so as to obtain more discriminative feature representation and higher classification accuracy. In addition, energy regularization is introduced into the model to fit the probability distribution of the data, so that the smaller the energy is, the more concentrated the probability distribution is, and the higher the classification accuracy is.

[0007] Technical scheme of the application:

[0008] Step 1: input the unlabeled image and k labeled reference images into the input layer of the autoencoder model as the training data set of the autoencoder model;

[0009] Step 2: calculate the mean square error between the reconstruction output value of the original autoencoder model and the original input value of step 1 as the mean square error term J AE of the autoencoder model loss function;

[0010] Step 3: calculate the sum of squares of the weights between the input layer and the hidden layer as the weight decay term J WD of the autoencoder model loss function;

[0011] Step 4: calculate the energy function of the entire autoencoder model as the energy regularization term J ER of the autoencoder model loss function;

[0012] Step 5: calculate the soft multi-label consistency similarity metric in the visual judgment as the soft multi-label learning term J SML of the autoencoder model loss function;

[0013] Step 6: minimize the loss function of the autoencoder model by using gradient descent method within a specified number of iterations to obtain the optimal parameter weights and biases of the autoencoder model, and pre-train the autoencoder model;

[0014] Step 7: input the test data into the self-encoding model adjusted to the optimum, then input the features extracted in the hidden layer into the classifier for classification to obtain the classification result.

[0015] Further, the self-encoder model comprises an input layer, an output layer and a hidden layer.

[0016] Setting parameters n unlabeled image samples in d-dimensional space, m labeled reference samples, wherein p i is one of the samples of each class of data in the database, and a i is the label of each data set. In the weakly supervised self-encoder WSAE of soft multi-label learning, a sigmoid activation function is used, which can be expressed as

[0017] wherein the parameters of the model are θ={W e ,b e ,W d ,b d}, W e and b e are the weights and biases between the input layer and the hidden layer of the self-encoder model, and W d and b d are the weights and biases between the hidden layer and the output layer of the self-encoder model. Then, y i =f(x i )=s(W e x i +b e ) represents the output result of the hidden layer in the self-encoder model, and z i =g(y i )=g(f(x i ))=s(W d (s(W e x i +b e ))+b d ) is the reconstruction output result of the output layer in the self-encoder model.

[0018] Further, in step 2, the mean square error term J AE in the loss function of the self-encoder model is calculated according to the formula:

[0019]

[0020] wherein x i represents the i-th unlabeled input sample, and z i represents the reconstruction output result of the i-th sample.

[0021] Further, in step 3, the weight decay term J WD The calculation formula is:

[0022]

[0023] where d is the number of nodes of the input layer of the autoencoder model; l is the number of nodes of the hidden layer of the autoencoder model, W e represents the network weight value connecting the input layer and the hidden layer nodes.

[0024] Further, in step 4, the energy regularization term J ER in the autoencoder model loss function is calculated as follows:

[0025] The reconstruction data of the autoencoder model is directly represented by r(X), which can be represented as r(X) = W d (s(W e X+b e ))+b d .

[0026] Then, the energy function of the autoencoder can be represented as

[0027] E(X) = ∫(r(X)-X)dX = ∫r(X)dX-∫XdX (3)

[0028] First, calculate the first part of formula (3), which can be represented as

[0029] ∫r(X)dX = ∫(W d (s(W e X+b d ))+b d )dX = W d ∫s(W e X+b d )dX + ∫b d dX (4)

[0030] The energy function can be further represented as

[0031] E(X) = W d ∫(s(W e X+b e ))dX + ∫(b d -X)dX (5)

[0032] In order to obtain the integral of the first part of formula (5), we define an auxiliary variable τ = W e X+b e , whose differential can be represented as

[0033]

[0034] We can also express formula (6) as dX = W e -T dτ. So, formula (5) can be expressed as

[0035]

[0036] Encoding weight W e and decoding weight W d There is a specified relationship, that is, W d = W e T Then we can get W d W e -T = I, where I is the unit matrix. Then, the energy regularization term can be expressed as

[0037]

[0038] Further, in step 5, the calculation method of the soft multi-label learning item J SML of the auto-encoder model loss function is:

[0039] Soft multi-label learning is to attach soft multi-labels to the unlabeled image dataset by comparing the unlabeled image dataset and the labeled reference sample, and to learn more discriminative feature representation by using soft multi-label to judge the visual similarity consistency, so as to improve the classification accuracy. In soft multi-label learning, the distance between similar images should be as close as possible, and the distance between dissimilar images should be as far as possible.

[0040] In the algorithm, the unit norm constraint is forced to be used, that is, ||f(•)||2 = 1, The cosine similarity between a pair of features f(x i ) and f(x j ) is simplified to their inner product f(x i ) T f(x j ), where x i ,x j are any pair of unlabeled images, and the same is true for the reference sample. Therefore, by comparing the features f(X) learned from the unlabeled image dataset with the labeled reference sample , the soft multi-label function can be obtained as follows:

[0041]

[0042] Where h (u)represents the u-th dimension of the soft multi-label h. When comparing unlabeled images, if two images have not only similar visual features but also highly similar comparison features, i.e., as similar as any other reference person, it is considered that the two images are very likely to belong to the same class. Therefore, the soft multi-label consistency similarity measure based on L1 distance can be represented as:

[0043]

[0044] Randomly extract two images x i ,x j from the unlabeled dataset, and perform soft multi-label h i ,h j consistency comparison. If A(h i ,h j )≥T, it is considered that the two images are not only similar in visual features but also similar in comparison features, and the two images are positive pairs. Otherwise, it is considered that the two images are negative pairs. Where T is the threshold of soft multi-label consistency, which can be determined by multiple experiments. Positive pairs are represented as X s and X t , and negative pairs are represented as X r and X g .

[0045] In order to obtain an automatic encoder with more discriminative features, reduce the distance between positive pairs, and separate negative pairs, so that the distance between images with high similarity is closer, and the distance between images with low similarity is farther. Therefore, the soft multi-label learning constraint term can be represented as:

[0046]

[0047] In the formula, N p and N n represent the number of positive pair sets and negative pair sets, respectively.

[0048] Further, in step 6, the loss function calculation formula of the autoencoder model is:

[0049] J WSAE =J AE +λJ WD +βJ SML +γJ ER (12)

[0050] Where λ, β and γ are parameters for balancing different penalty terms of J WD , J SML and J ER .

[0051] The beneficial effects of the present application: Although extensive research has been conducted on various weakly supervised methods and models, few weakly supervised studies of autoencoders have been used for image classification, and the lower limit of the number of labels required for weakly supervised learning has not been demonstrated. To address this deficiency, the present application proposes a weakly supervised autoencoder through energy regularization and soft multi-label learning. The main idea of the present application is to provide a model with a supervised signal with less label information, to punish the distance between learned features and their class centers, and to make the inter-class dispersion and intra-class compactness as small as possible. And inspired by the energy function in the restricted Boltzmann machine (RBM), the energy function is used as a measure of the state of the entire system, and the network is trained towards small energy, which ensures the stability of the network model and improves the feature extraction capability. In order to verify the effectiveness of our method, we apply WSAE to multiple benchmark datasets, including Fashion MNIST, USPS, Pendigits, Spambase, PageBlocks, ISOLET, MNIST and CIFAR-10. We also compare WSAE with AE, SAE, VAE, DC-VAE, LSRA, ERAE and SAME. The experimental results show that our method is always significantly better than the other several methods. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 Learning goal of soft multi-label learning.

[0053] Figure 2 WSAE model structure diagram. DETAILED DESCRIPTION

[0054] 1. Theoretical analysis of weakly supervised learning based on k labeled samples

[0055] Weakly supervised learning aims to train a good machine learning model with as few labels as possible, but what is the lower limit? This is a theoretical problem worth exploring.

[0056] Proposition 1 In our invention, we use S(k) to represent the input sample of k classes (k is a natural number greater than 1), and M(k) represents the corresponding weakly supervised learning model. Then the model can M(k) correctly classify the multi-class sample S(k) with at least k-1 labeled samples, and each class only needs to be labeled with one labeled sample.

[0057] Inductive reasoning can be used to infer:

[0058] 1) When k = 2, we use S(2) to represent the binary classification data, and use it to train the weakly supervised learning model M(2). At this time, M(2) can correctly classify the data S(2) with at least one labeled sample. In an ideal case, we only need one labeled sample to completely classify the binary classification data in the correct way.

[0059] 2) When 2 < k = n, the k-classification problem can be divided into k-1 binary classification problems. From step 1), we know that the model M(k) can correctly classify the data S(k) using at least k-1 labeled samples.

[0060] 3) When k = n+1, based on the above steps 1) and 2), it can be concluded that Proposition 1 is correct.

[0061] In the weakly supervised learning method using labeled samples, the lower limit value of the size of the labeled sample is k-1, which is crucial for correctly classifying all samples. In the present invention, we use k labeled samples for weakly supervised learning. Here, we choose a value k that is 1 larger than the lower limit value, in order to facilitate the construction of the model and the reasoning of the algorithm, while ensuring the effectiveness of the weakly supervised learning. The weakly supervised learning theory based on k labeled samples is the inheritance and development of Proposition 1.

[0062] 2. Soft multi-label learning

[0063] Soft multi-label learning is to attach soft multi-labels to the unlabeled image dataset by comparing it with the labeled reference samples, and to learn more discriminative feature representations by using soft multi-labels to judge visual similarity and consistency, thereby improving classification accuracy. Due to the difference within the class and the similarity between the classes, if the distance measurement method is not appropriate, the distance between the images within the class may be larger than the distance between the images of different classes. Therefore, in order to learn a feature representation with good discrimination, it is necessary to make the distance between similar images as close as possible, and the distance between dissimilar images as far as possible. Figure 1 The learning goal of soft multi-label learning is shown.

[0064] Setting parameters Let S = {x1, x2, …, xn} be a set of n unlabeled image samples in d-dimensional space, Let S = {x1, x2, …, xn} be a set of n unlabeled image samples in d-dimensional space, i is one of the samples of each class in the database, and a i is the label of each class of each dataset; in the weakly supervised autoencoder WSAE of soft multi-label learning, the sigmoid activation function is used, which can be represented as

[0065] where the parameters of the model are θ = {W e ,be W d b d},W e and b e are the weights and biases between the input layer and the hidden layer of the autoencoder model, W d and b d are the weights and biases between the hidden layer and the output layer of the autoencoder model; then, y i = f(x i ) = s(W e x i +b e ) represents the output result of the hidden layer in the autoencoder model, z i = g(y i ) = g(f(x i )) = s(W d (s(W e x i +b e ))+b d ) is the reconstruction output result of the output layer in the autoencoder model.

[0066] In order to improve the efficiency of feature learning, we force the use of unit norm constraint in the algorithm, i.e. ||f(·)||2=1, The cosine similarity between a pair of features f(x i ) and f(x j ) is simplified to their inner product f(x i ) T f(x j ), where x i ,x j are any pair of unlabeled images, and the same is true for the reference samples; therefore, by comparing the features f(X) learned from the unlabeled image dataset with the labeled reference samples , we can get the soft multi-label function:

[0067]

[0068] where h (u) represents the u-th dimension of the soft multi-label h; when comparing unlabeled images, two pictures not only have similar visual features, but also have highly similar comparison features, i.e. they are equally similar to any other reference person, then it is considered that the two pictures are very likely to belong to the same class; therefore, the soft multi-label consistency similarity based on L1 distance can be represented as:

[0069]

[0070] Randomly extract two images x i ,xj and its soft multi-label h i ,h j is performed; if A(h i ,h j )≥T, it is considered that the two pictures are positive pairs except that the visual features are similar, and vice versa. Where T is the threshold of soft multi-label consistency, which can be determined by multiple experiments, and T is 0.9 in this algorithm. We denote the positive pair as X s and X t , and the negative pair as X r and X g . In order to obtain an automatic encoder with more discriminative features, reduce the distance between positive pairs, and separate negative pairs, so that the distance between images with high similarity is closer, and the distance between images with low similarity is farther. Therefore, the soft multi-label learning constraint term can be expressed as

[0071]

[0072] In the formula, N p and N n respectively represent the number of positive pair sets and negative pair sets.

[0073] 3. Energy regularization

[0074] The encoding and decoding process of the autoencoder can be defined as a dynamic system. Inspired by the energy function of the restricted Boltzmann machine, the present application also uses an energy function to describe the state of the entire system. The more orderly the system is, the more concentrated the probability distribution is, and the smaller the energy of the system is. Conversely, the more disordered the system is or the more uniform the distribution is, the greater the energy is. The minimum value of the energy function corresponds to the most stable state of the system. This is similar to the law of nature, the smaller the energy, the more stable the system.

[0075] The energy function usually needs to calculate the mean square error between the input data and the reconstructed data. Therefore, we use r(X) to represent the reconstructed data, and it can be expressed as r(X) = W d (s(W e X+b e ))+b d .

[0076] Then, the energy function of the autoencoder can be expressed as

[0077] E(X) = ∫(r(X)-X)dX = ∫r(X)dX-∫XdX (16)

[0078] First, calculate the first part of formula (16), which can be expressed as

[0079] ∫r(X)dX = ∫(W d (s(W e X+b d ))+b d )dX = W d ∫s(W e X+b d )dX + ∫b d dX (17)

[0080] The energy function can be further expressed as

[0081] E(X) = W d ∫(s(W e X+b e ))dX + ∫(b d -X)dX (18)

[0082] To obtain the integral of the first part in equation (18), an auxiliary variable τ = W e X+b e is defined, whose differential can be expressed as

[0083]

[0084] Equation (19) can also be expressed as dX = W e -T dτ; therefore, equation (18) can be expressed as

[0085]

[0086] The encoding weight W e and the decoding weight W d have a specified relationship, i.e., W d = W e T ; then W d W e -T = I, where I is an identity matrix; then the energy regularization term can be expressed as

[0087]

[0088] 4. Soft multi-label learning and energy function constraint semi-supervised auto-encoder

[0089] In summary, the loss function of our method WSAE can be further expressed as

[0090] J WSAE = J AE + λJ WD + βJ SML + γJ ER(21)

[0091] where λ, β and γ are parameters of different penalty terms of J WD , J SML and J ER , respectively. The first term is the mean square error between the output value and the original input value, which can be directly given by the following formula

[0092]

[0093] The weight decay term punishes the sum of the square of the weights between the input layer and the hidden layer to prevent overfitting, and its calculation formula is

[0094]

[0095] In the present application, formula (15) is used to calculate the soft multi-label learning term, which punishes the distance between the learned features and their corresponding class centers as much as possible to make the intra-class compact and the inter-class dispersed. The energy regularization term can be calculated by formula (20), which is a measure to describe the state of the entire system. By minimizing this constraint, a stable model can be trained and a better representation of the data can be obtained. Subsequently, the parameters of WSAE are updated by minimizing the objective function in formula (21) using the gradient descent method.

[0096] The present application proposes a weakly supervised autoencoder WSAE based on k labeled samples. The overall framework of the proposed WSAE is shown in Figure 2 The network is a three-layer neural network, including an input layer, a hidden layer and an output layer. In order to improve the feature extraction performance of the learning model, a reconstruction error term, a weight decay term, a soft multi-label learning term and an energy regularization term are added to the network model. Among them, the reconstruction error term J AE is the average reconstruction error of the input data and the output data of all samples, and its purpose is to try to reconstruct the output data that is the same as the input data. The weight decay term J WD prevents overfitting of the model to a certain extent. In addition to the two conventional regularization terms, the method also introduces a soft multi-label learning term and an energy regularization term. The purpose of the soft multi-label learning term J SML is to provide a supervision signal, which can effectively pull the extracted features of the same class together and increase the feature difference between different classes, so as to learn good discriminative information. The energy regularization term J ER is used to describe the energy value of the system. When the energy value reaches the minimum value, the system enters a stable state. J ERThe smaller the value is, the more accurately the probability distribution of the data is fitted, the data is better expressed, the feature extraction capability of the network is improved, and the classification precision of the network is improved. The WSAE is a weakly supervised learning method, which only uses k labeled samples (where k is the number of classes of each data set), and uses the four regularization terms above to jointly constrain the model during the training process, and finally inputs the extracted features into a softmax classifier for classification application.

[0097] In order to verify the superiority of the WSAE method, we carried out multiple experiments on eight standard data sets, which are: Fashion-MNIST, USPS, Pendigits, Glass, Pageblocks, ISOLET, MNIST and CIFAR-10. In order to ensure the fairness of the experiment, we repeated the experiment 100 times on each data set. The experimental results are shown in Table 1. It can be seen that the WSAE method proposed in the present application is superior to other representation learning algorithms in all cases. Compared with other methods, the classification accuracy result of WSAE is also relatively stable. At the same time, the classification accuracy of the energy regularized autoencoder ERAE and the autoencoder SAME using soft multi-label learning is significantly higher than that of the standard autoencoder AE, which also proves the effectiveness of the energy constraint and the soft multi-label learning constraint.

[0098] Table 1 Comparison of classification accuracy on each data set

[0099]

[0100]

Claims

1. An image classification method based on a weakly supervised autoencoder, characterized in that, Step 1: Input the unlabeled image and k labeled reference images into the input layer of the autoencoder model as the training dataset for the autoencoder model; Step 2: Calculate the mean squared error between the reconstructed output value of the original autoencoder model and the original input value from Step 1, and use it as the mean squared error term J in the autoencoder model loss function. AE ; Step 3: Calculate the sum of squared weights between the input layer and the hidden layer as the weight decay term J in the loss function of the autoencoder model. WD ; Step 4: Calculate the energy function of the entire autoencoder model as the energy regularization term J of the autoencoder model loss function. ER ; Step 5: Calculate the soft multi-label learning term J, which is used as the loss function of the autoencoder model, to discriminate the visual soft multi-label consistency similarity metric. SML ; Step 6: Minimize the loss function of the autoencoder model using gradient descent within the specified number of iterations to obtain the optimal parameter weights and biases of the autoencoder model, and pre-train the autoencoder model. Step 7: Input the test data into the optimized autoencoder model, and then input the features extracted from the hidden layer into the classifier for classification to obtain the classification result.

2. The image classification method based on a weakly supervised autoencoder according to claim 1, characterized in that, The autoencoder model includes an input layer, an output layer, and a hidden layer; Setting parameters Given n unlabeled image samples in d-dimensional space, Let there be m labeled reference samples, where p i It is one sample of each type of data in the database, and a i These are the various labels for each dataset; In the weakly supervised autoencoder (WSAE) for soft multi-label learning, the sigmoid activation function can be used, which can be expressed as: The parameters of the model are θ={W e ,b e W d ,b d }, W e and b e W represents the weights and biases between the input layer and the hidden layer of the autoencoder model. d and b d It refers to the weights and biases between the hidden and output layers of the autoencoder model; then, y i =f(x) i ) = s(W e x i +b e ) represents the output of the hidden layer in the autoencoder model, z i =g(y i )=g(f(x i ))=s(W d (s(W e x i +b e ))+b d ) represents the reconstructed output of the output layer in the autoencoder model.

3. The image classification method based on a weakly supervised autoencoder according to claim 1, characterized in that, In step 2, the mean square error term J in the loss function of the autoencoder model AE The calculation formula is: Where, x i Let z represent the i-th unlabeled input sample. i This represents the reconstruction output of the i-th sample.

4. The image classification method based on a weakly supervised autoencoder according to claim 1, characterized in that, In step 3, the weight decay term J in the autoencoder model loss function WD The calculation formula is: Where d is the number of nodes in the input layer of the autoencoder model; l is the number of nodes in the hidden layer of the autoencoder model; and W... e This represents the network weight value connecting the input layer and the hidden layer nodes.

5. The image classification method based on a weakly supervised autoencoder according to claim 1, characterized in that, In step 4, the energy regularization term J in the autoencoder model loss function ER The calculation process is as follows: We can directly use r(X) to represent the reconstructed data of the autoencoder model, and it can be expressed as r(X) = W d (s(W e X+b e ))+b d ; Therefore, the energy function of the autoencoder can be expressed as: E(X)=∫(r(X)-X)dX=∫r(X)dX-∫XdX (3) First, calculate the first part of formula (3), which can be expressed as: ∫r(X)dX=∫(W d (s(W e X+b d ))+b d )dX=W d ∫s(W e X+b d )dX+∫b d dX (4) The energy function can be further expressed as E(X)=W d ∫(s(W e X+b e ))dX+∫(b d -X)dX (5) To obtain the integral of the first part in formula (5), the auxiliary variable τ = W is defined. e X+b e Its differential can be expressed as Formula (6) can also be expressed as dX = W e -T dτ; Therefore, formula (5) can be expressed as Encoding weight W e and decoding weight W d There is a defined relationship, namely W d =W e T Then we can get W. d W e -T =I, where I is the identity matrix; then, the energy regularization term can be expressed as 6. The image classification method based on a weakly supervised autoencoder according to claim 1, characterized in that, In step 5, the soft multi-label learning term J of the autoencoder model loss function SML The calculation method is as follows: Soft multi-label learning involves comparing unlabeled image datasets with labeled reference samples, assigning soft multi-labels to the unlabeled image datasets, and using soft multi-labels to determine visual similarity consistency, thereby learning more discriminative feature representations and improving classification accuracy. In soft multi-label learning, it is necessary to make the distance between similar images as close as possible and the distance between dissimilar images as far as possible. The algorithm enforces the unity norm constraint, i.e., ||f(·)||²=1, ||a i ||2=1, A pair of features f(x) i ) and f(x j The cosine similarity between x and y is simplified to their inner product f(x) i ) T f(x j ), where x i ,x j For any pair of unlabeled images, and the same applies to the reference sample; therefore, by combining the features f(X) learned from the unlabeled image dataset with the labeled reference sample... By comparing, we can obtain the soft multi-label function: Where h (u) Let represent the u-th dimension of the soft multi-label h; when comparing unlabeled images, if two images not only have similar visual features but also highly similar comparative features, that is, they are as similar as any other reference image, then the two images are considered to be very likely to belong to the same class; therefore, the soft multi-label consistency similarity measure based on L1 distance can be expressed as: Randomly select two images x from the unlabeled dataset i ,x j and its soft multi-tag h i ,h j Perform soft multi-label consistency comparison; if A(h i ,h j If X ≥ T, then the two images are considered a positive pair if they are similar not only in visual features but also in comparative features; otherwise, they are considered a negative pair. Here, T is the threshold for soft multi-label consistency, which can be determined through multiple experiments. A positive pair is represented as X. s and X t The negative image is represented as X. r and X g ; To obtain an autoencoder with more discriminative features, the distance between positive pairs of images is reduced, and negative pairs of images are separated, so that images with high similarity are closer together, while images with low similarity are farther apart; therefore, the soft multi-label learning constraint can be expressed as: In the formula, N p and N n These represent the number of positive pairs and the number of negative pairs, respectively.

7. The image classification method based on a weakly supervised autoencoder according to claim 1, characterized in that, In step 6, the loss function of the autoencoder model is calculated using the following formula: J WSAE =J AE +λJ WD +βJ SML +γJ ER (12) Where λ, β, and γ are the equilibrium J WD J SML and J ER The parameters for different penalty items.

Images