A training method and device of an image classification model, a computer device and a medium

Abstract

The present application relates to the technical field of image classification, and particularly relates to a training method and device of an image classification model, equipment and a medium. The method obtains an auto-encoding feature vector and a reconstructed image feature vector through an unsupervised image reconstruction model, obtains K feature vector clustering clusters of the auto-encoding feature vector, determines a noise probability transition matrix for representing noise confusion information between various image categories, obtains a category probability estimation vector according to a supervised image classification model, determines model loss by combining the noise probability transition matrix to perform model training, obtains a trained supervised image classification model to perform image classification, measures the distribution of noise labels and non-noise labels through the feature vector clustering clusters, obtains the noise probability transition matrix to represent the noise confusion information between the image categories, and measures the loss of the supervised image classification model, reduces the influence of noise labels on the supervised image classification model, and improves the accuracy of the supervised image classification model.

Description

Technical Field

[0001] This invention relates to the field of image classification technology, and in particular to a training method, apparatus, computer equipment, and medium for an image classification model. Background Technology

[0002] In recent years, training deep learning models with massive amounts of labeled data has become a common technical approach. However, in real-world image classification scenarios, the widespread presence of label noise in image label datasets causes image classification models to learn incorrect label information during training, reducing the classification accuracy of image classification models on noisy labeled datasets.

[0003] When training an image classification model based on noisy labeled data, the classification accuracy of the image classification model is generally improved by improving the loss function of the training model. However, this method often introduces other data assumptions, such as data sample balance. These distribution assumptions limit the applicability and reliability of improving the loss function, resulting in low accuracy of the image classification model on large-scale noisy image datasets.

[0004] Therefore, how to improve the accuracy of image classification models based on large-scale noisy image datasets has become an urgent problem to be solved. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a training method, apparatus, computer equipment, and medium for an image classification model to solve the problem of low accuracy of existing image classification models on large-scale noisy image datasets.

[0006] In a first aspect, embodiments of the present invention provide a method for training an image classification model, the method comprising:

[0007] Obtain the images to be classified and their category labels in the training set, and input each image to be classified into an unsupervised image reconstruction model to obtain the autoencoded feature vector and the reconstructed image feature vector of each image to be classified.

[0008] Cluster the autoencoded feature vectors of all the images to be classified to obtain K feature vector clusters. Based on the image to be classified and its category label corresponding to each autoencoded feature vector in each feature vector cluster, determine the noise probability transition matrix used to characterize the noise confusion information between each image category, where K is a positive integer.

[0009] Each image to be classified is input into a supervised image classification model to obtain a class probability estimation vector for each image to be classified.

[0010] Based on each image to be classified and the reconstructed image feature vector of each image to be classified, the unsupervised model loss of the unsupervised image reconstruction model is determined, and based on the noise probability transition matrix, the class probability estimation vector of each image to be classified and the class label, the supervised model loss of the supervised image classification model is determined.

[0011] The unsupervised image reconstruction model is trained based on the unsupervised model loss, and the supervised image classification model is trained based on the supervised model loss until the unsupervised model loss and the supervised model loss converge, resulting in a trained unsupervised image reconstruction model and a trained supervised image classification model. The trained supervised image classification model is then used for image classification.

[0012] Optionally, determining the supervised model loss of the supervised image classification model based on the noise probability transition matrix, the class probability estimation vector of each image to be classified, and the class label includes:

[0013] Determine the noise probability transition matrix in the t-th iteration of training;

[0014] In the training iterations from the tth to the (t+T)th iteration, the supervised model loss of the supervised image classification model is determined based on the noise probability transition matrix, the category probability estimation vector of each image to be classified, and the category label in the training iteration t, where t is the preset number of iterations, T is the preset interval between iterations, and t and T are positive integers.

[0015] Determine the noise probability transition matrix in the (t+T+1)th training iteration;

[0016] During the training iterations from t+T+1 to t+2T, the supervised model loss of the supervised image classification model is determined based on the noise probability transition matrix, the category probability estimation vector of each image to be classified, and the category label in the training iteration t+T+1.

[0017] This process continues until the supervised model loss, as determined in a certain iteration of training, converges.

[0018] Secondly, embodiments of the present invention provide a training apparatus for an image classification model, the training apparatus for the image classification model comprising:

[0019] An unsupervised reconstruction module is used to obtain the images to be classified and their category labels in the training set, and input each image to be classified into the unsupervised image reconstruction model to obtain the autoencoded feature vector and the reconstructed image feature vector of each image to be classified.

[0020] The matrix determination module is used to cluster the autoencoded feature vectors of all the images to be classified to obtain K feature vector clusters. Based on the image to be classified and its category label corresponding to each autoencoded feature vector in each feature vector cluster, the module determines the noise probability transition matrix used to characterize the noise confusion information between image categories, where K is a positive integer.

[0021] A supervised classification module is used to input each image to be classified into a supervised image classification model to obtain a class probability estimation vector for each image to be classified.

[0022] The loss determination module is used to determine the unsupervised model loss of the unsupervised image reconstruction model based on each image to be classified and the reconstructed image feature vector of each image to be classified, and to determine the supervised model loss of the supervised image classification model based on the noise probability transition matrix, the class probability estimation vector of each image to be classified and the class label.

[0023] The model training module is used to train the unsupervised image reconstruction model based on the unsupervised model loss and the supervised image classification model based on the supervised model loss until the unsupervised model loss and the supervised model loss converge, thereby obtaining the trained unsupervised image reconstruction model and the trained supervised image classification model, and using the trained supervised image classification model to perform image classification.

[0024] Optionally, the loss determination module mentioned above includes:

[0025] The first matrix determination submodule is used to determine the noise probability transition matrix in the t-th iteration of training.

[0026] The second model loss calculation submodule is used to determine the supervised model loss of the supervised image classification model in the training from the tth iteration to the t+Tth iteration, based on the noise probability transition matrix, the class probability estimation vector of each image to be classified, and the class label in the training of the tth iteration, where t is the preset number of iterations, T is the preset interval between iterations, and t and T are positive integers.

[0027] The second matrix determination submodule is used to determine the noise probability transition matrix in the (t+T+1)th iteration of training.

[0028] The third model loss calculation submodule is used to determine the supervised model loss of the supervised image classification model in the training from the (t+T+1)th to the (t+2T)th iteration, based on the noise probability transition matrix, the class probability estimation vector of each image to be classified, and the class label in the training from the (t+T+1)th iteration.

[0029] An iterative submodule is used to repeat this process until the supervised model loss determined in a certain iteration of training converges.

[0030] Thirdly, embodiments of the present invention provide a computer device, the computer device including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the training method of the image classification model as described in the first aspect.

[0031] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the training method for the image classification model as described in the first aspect.

[0032] The beneficial effects of this invention compared to existing technologies are as follows: Each image to be classified is input into an unsupervised image reconstruction model to obtain an autoencoded feature vector and a reconstructed image feature vector. The autoencoded feature vectors of all images to be classified are clustered to obtain K feature vector clusters. Based on the image to be classified and its category label corresponding to each autoencoded feature vector in each feature vector cluster, a noise probability transition matrix is ​​determined to characterize the noise confusion information between image categories. Each image to be classified is input into a supervised image classification model to obtain a category probability estimation vector for each image to be classified. Based on each image to be classified and its reconstructed image feature vector, the unsupervised model loss of the unsupervised image reconstruction model is determined. Based on the noise probability transition matrix, the category probability estimation vector of each image to be classified, and its category label, the supervised model loss of the supervised image classification model is determined. The unsupervised image reconstruction model is trained based on the unsupervised model loss, and the supervised model loss is used to train the supervised image classification model. The image classification model is trained until the unsupervised model loss and the supervised model loss converge, resulting in a trained unsupervised image reconstruction model and a trained supervised image classification model. The trained supervised image classification model is then used for image classification. The distribution of noise labels and non-noise labels in the images to be classified is measured by clustering the autoencoded feature vectors corresponding to each autoencoded feature vector in the feature vector clusters. The noise probability transition matrix effectively represents the noise confusion information between different image categories. The unsupervised image reconstruction model is trained based on the similarity between the reconstructed image feature vector and the image to be classified, improving the accuracy of the noise probability transition matrix. Furthermore, the noise probability transition matrix is ​​combined with the similarity between the category probability estimation vector and the category label to measure the supervised model loss of the supervised image classification model, thus reducing the influence of noise labels in the noisy training set on the supervised image classification model and improving its accuracy. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a schematic diagram of an application environment for a training method for an image classification model provided in Embodiment 1 of the present invention;

[0035] Figure 2 This is a flowchart illustrating a training method for an image classification model provided in Embodiment 1 of the present invention;

[0036] Figure 3 This is a schematic diagram of the structure of a training device for an image classification model provided in Embodiment 3 of the present invention;

[0037] Figure 4 This is a schematic diagram of the structure of a computer device provided in Embodiment 4 of the present invention. Detailed Implementation

[0038] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0039] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0040] It should also be understood that the term “and / or” as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0041] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0042] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0043] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0044] The embodiments of this invention can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0045] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0046] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0047] To illustrate the technical solution of the present invention, specific embodiments are described below.

[0048] The image classification model training method provided in Embodiment 1 of this invention can be applied to, for example... Figure 1In this application environment, the client communicates with the server. The client includes, but is not limited to, handheld computers, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cloud computing devices, and personal digital assistants (PDAs). The server can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0049] See Figure 2 This is a flowchart illustrating a training method for an image classification model provided in Embodiment 1 of the present invention. The above-described image classification model training method can be applied to... Figure 1 For clients using this model, the training method for the image classification model may include the following steps:

[0050] Step S201: Obtain the images to be classified and their category labels in the training set, and input each image to be classified into the unsupervised image reconstruction model to obtain the autoencoded feature vector and the reconstructed image feature vector of each image to be classified.

[0051] In this embodiment, the training set is a large-scale image dataset that suffers from label noise in real-world image classification scenarios. It includes a large number of images to be classified and a category label for each image, representing the image category. The specific image categories can be set according to the actual situation.

[0052] The goal of image classification is to extract and analyze features from the images to be classified, and to determine the image category of the images to be classified.

[0053] In this embodiment, feature extraction and category feature representation learning are performed on the images to be classified based on the unsupervised image reconstruction model to obtain the autoencoded feature vector of each image to be classified.

[0054] Then, the unsupervised image reconstruction model decodes the autoencoded feature vector to determine the reconstructed image feature vector for each image to be classified. This reconstructed image feature vector serves as the training basis for the unsupervised image reconstruction model. The accuracy of the unsupervised image reconstruction model in representing the features of the images to be classified can be measured by the similarity between the reconstructed image feature vector and the corresponding image to be classified.

[0055] In this embodiment, the image classification task can be in scenarios such as gait analysis, video surveillance, and sports science, classifying human postures in images to be classified, with corresponding category labels such as stationary, walking, running, squatting, jumping, etc.; it can also be in scenarios such as product audience analysis and population aging analysis, classifying facial attributes in images to be classified, with corresponding category labels such as male, female, child, youth, middle-aged, elderly, etc.; it can also be in scenarios such as teaching evaluation, product sales, and personnel interviews, classifying human emotions in images to be classified, with corresponding category labels such as happy, nervous, sad, disgusted, bored, etc.; and it can also be in scenarios such as renting, decorating, and buying a house, classifying room styles in images to be classified, with corresponding category labels such as pastoral, minimalist, classical, new Chinese, Mediterranean, Southeast Asian, etc.

[0056] This embodiment takes the classification of human poses in images as an example. The images to be classified in the training set include human pose images of various postures, and the category labels are various human pose categories such as stationary, walking, running, squatting, and jumping. Since it is difficult to avoid labeling errors when annotating the category labels, some human pose images will be labeled as incorrect human pose categories, making the training set in this embodiment a noisy human pose image dataset.

[0057] When training the unsupervised image reconstruction model, each human pose image in the training set is input into the unsupervised image reconstruction model to perform feature extraction and class feature representation learning, so as to obtain the autoencoded feature vector and the reconstructed image feature vector of each human pose image, which serve as the training basis for the unsupervised image reconstruction model.

[0058] Optionally, the unsupervised image reconstruction model includes an encoder and a decoder;

[0059] Each image to be classified is input into an unsupervised image reconstruction model to obtain the autoencoded feature vector and the reconstructed image feature vector for each image to be classified, including:

[0060] Each image to be classified is input into the encoder to obtain the autoencoded feature vector of each image to be classified.

[0061] The autoencoded feature vector of each image to be classified is input into the decoder to obtain the reconstructed image feature vector of each image to be classified.

[0062] The unsupervised image reconstruction model includes an encoder and a decoder. The encoder is used to extract features from the input of the unsupervised image reconstruction model and output a feature vector. The decoder is used to decode the feature vector output by the encoder to obtain the reconstructed input.

[0063] In this embodiment, each image to be classified is input into the encoder for feature extraction to obtain the autoencoded feature vector of each image to be classified. Then, the autoencoded feature vector of each image to be classified is input into the decoder for feature decoding to obtain the reconstructed image feature vector of each image to be classified. This serves as the training basis for the unsupervised image reconstruction model. The accuracy of the feature representation of the unsupervised image reconstruction model for the image to be classified can be measured by the similarity between the reconstructed image feature vector and the corresponding image to be classified.

[0064] In this embodiment, the encoder in the unsupervised image reconstruction model extracts features from each image to be classified, obtaining an autoencoded feature vector for each image. The decoder in the unsupervised image reconstruction model then decodes each autoencoded feature vector to obtain a reconstructed image feature vector for each image to be classified. This reconstructed image feature vector serves as the training basis for the unsupervised image reconstruction model, thereby improving the model's ability to represent features of images to be classified.

[0065] The steps described above—obtaining the images to be classified in the training set, inputting each image to be classified into an unsupervised image reconstruction model, and obtaining the autoencoded feature vector and the reconstructed image feature vector for each image to be classified—perform representation learning of the category features of the images to be classified, providing a feature foundation for image classification tasks.

[0066] Step S202: Cluster the autoencoded feature vectors of all images to be classified to obtain K feature vector clusters. Based on the images to be classified and their category labels corresponding to each autoencoded feature vector in each feature vector cluster, determine the noise probability transition matrix used to characterize the noise confusion information between image categories.

[0067] In this model, the unsupervised image reconstruction model learns the representation of category features of the images to be classified, and obtains the autoencoded feature vector of each image to be classified. Correspondingly, the autoencoded feature vectors of images to be classified belonging to the same image category are quite similar.

[0068] In this embodiment, the autoencoded feature vectors of all images to be classified are clustered to obtain K feature vector clusters, where K is a positive integer. When there are no noise labels in the training set, the category labels of the images to be classified corresponding to the autoencoded feature vectors in each feature vector cluster are consistent. When there are noise labels in the training set, the category labels of the images to be classified corresponding to the autoencoded feature vectors in each feature vector cluster may contain noise labels, which may make the category labels of the images to be classified corresponding to the autoencoded feature vectors in each feature vector cluster not completely consistent.

[0069] Therefore, this embodiment determines the number of images belonging to each category label in each feature vector cluster based on the images to be classified and their category labels corresponding to each autoencoded feature vector in each feature vector cluster, thereby measuring the distribution of all category labels in the images to be classified, including the distribution of noise labels and the distribution of non-noise labels, and thus characterizing the noise confusion information between each image category, and obtaining the noise probability transition matrix used to characterize the noise confusion information between each image category.

[0070] Specifically, in this embodiment, the noise probability transition matrix is ​​a K*K dimensional matrix, which corresponds to K image categories. Each row dimension corresponds to one image category, each column dimension corresponds to one image category, each feature vector cluster corresponds to one row dimension, and the number of images to be classified corresponding to each category label in each feature vector cluster corresponds to the column value of the row dimension.

[0071] For example, the k-th (k = 1, 2, ..., K) feature vector cluster corresponds to the j-th (j = 1, 2, ..., K) row dimension, and the number of images to be classified corresponding to the 1st, 2nd, ..., Kth category labels in the k-th feature vector cluster corresponds to the 1st, 2nd, ..., Kth column values ​​in the j-th row dimension.

[0072] This noise probability transition matrix, based on the eigenvector clusters of the autoencoded eigenvectors, characterizes the distribution of all category labels in the images to be classified according to the number of images belonging to each category label in each eigenvector cluster. The k-th eigenvector cluster corresponds to the j-th row dimension and an image category. Category labels in the k-th eigenvector cluster that do not belong to that image category can be considered noise labels, and the number of images to be classified corresponding to these noise labels can be used to characterize the distribution of noise labels in the k-th eigenvector cluster. Conversely, category labels in the k-th eigenvector cluster that belong to that image category can be considered non-noise labels, and the number of images to be classified corresponding to these non-noise labels can be used to characterize the distribution of non-noise labels in the k-th eigenvector cluster. Therefore, this noise probability transition matrix simultaneously characterizes the distribution of both noise and non-noise labels, thus representing the noise confusion information between different image categories.

[0073] In one embodiment, the K-means clustering algorithm is used to cluster the category feature information of all images to be classified, where K is the total number of image categories. Specifically, K initial feature cluster centers are randomly selected from all category feature information. By calculating the distance between each category feature information and each initial feature cluster center, each category feature information is assigned to the nearest initial feature cluster center. After each category feature information is assigned, the feature cluster center is recalculated and updated based on the existing category feature information in the feature vector cluster. The above clustering process is repeated until a preset clustering termination condition is met. Then, each feature cluster center and the category feature information assigned to that feature cluster center form a feature vector cluster, resulting in K feature vector clusters. The preset clustering termination condition can be set according to the actual situation. For example, it can be that no objects or a minimum number of objects are reassigned to different feature cluster centers, or that no or a minimum number of feature cluster centers are changed.

[0074] In this embodiment, taking the classification of human poses in images to be classified as an example, correspondingly, the autoencoded feature vectors of human pose images belonging to the same human pose are relatively similar. Therefore, the autoencoded feature vectors of all human pose images are clustered to obtain K feature vector clusters. Then, based on the human pose image and its category label corresponding to each autoencoded feature vector in each feature vector cluster, the number of human pose images belonging to each category label in each feature vector cluster is determined to measure the distribution of all category labels in the human pose image, including the distribution of noise labels and the distribution of non-noise labels, thereby characterizing the noise confusion information between each human pose category and obtaining the noise probability transition matrix used to characterize the noise confusion information between each human pose category.

[0075] Optionally, based on the image to be classified and its category label corresponding to each autoencoded feature vector in each feature vector cluster, the noise probability transition matrix used to characterize the noise confusion information between image categories is determined, including:

[0076] For any feature vector cluster, determine the number of images for each category label based on the image to be classified and its category label corresponding to each autoencoded feature vector;

[0077] The category of feature vector clusters is determined based on the number of images for each category label;

[0078] Based on the category of the eigenvector cluster, determine the row dimension of the eigenvector cluster in the noise probability transition matrix;

[0079] Based on the row dimension of the feature vector cluster in the noise probability transition matrix and the number of images for each category label, determine the column value of the row dimension of the feature vector cluster in the noise probability transition matrix.

[0080] Based on the column values ​​of the row dimension of all feature vector clusters in the noise probability transition matrix, the noise probability transition matrix used to characterize the noise confusion information between each image category is determined.

[0081] The noise probability transition matrix is ​​a K*K dimensional matrix, where each feature vector cluster corresponds to a row dimension, and each row dimension corresponds to an image category.

[0082] Due to the presence of label noise in the training set, each feature vector cluster may contain a small number of noisy labels and a large number of category labels corresponding to the image categories. Therefore, based on the images to be classified and their category labels corresponding to each autoencoded feature vector in each feature vector cluster, the number of images for each category label in each feature vector cluster can be counted, and the category label with the largest number of images can be determined as the category of the corresponding feature vector cluster, where the category belongs to one of the image categories.

[0083] Then, the row dimension of each feature vector cluster in the noise probability transition matrix can be determined. Then, based on the column dimension corresponding to each category label, the number of images of each category label in each feature vector cluster is taken as the column value of each category label in the corresponding row dimension. Thus, each row dimension in the noise probability transition matrix can be obtained. Finally, the noise probability transition matrix is ​​obtained based on the K row dimensions of the noise probability transition matrix.

[0084] For example, to count the number of images for each category label in the k-th feature vector cluster, let the number of images for the s-th (s = 1, 2, ..., K) category label be denoted as X. ks Then the number of images X can be determined. k1 X k2 , ..., X kK The category label corresponding to the maximum value in the matrix is ​​the category of the u-th image (u = 1, 2, ..., K) corresponding to this category label. The u-th image category corresponds to the u-th row of the noise probability transition matrix. Then the number of images X... k1 X k2 , ..., X kK These correspond to the column values ​​of the 1st, 2nd, ..., Kth columns of the uth row of the noise probability transition matrix.

[0085] The clusters of all eigenvectors corresponding to the u-th row of the noise probability transition matrix are counted. The number of images in the 1st, 2nd, ..., Kth columns of the u-th row are summed to obtain the sum of the number of images in the K-th row. The sum of the number of images in the K-th row is normalized. The sum of the number of images in the K-th row is determined as the column values ​​of the 1st, 2nd, ..., Kth columns of the u-th row of the noise probability transition matrix. The u-th row of the noise probability transition matrix is ​​obtained accordingly.

[0086] Further determine the K rows of the noise transition matrix to obtain the noise probability transition matrix.

[0087] This embodiment determines the category of feature vector clusters based on the number of images for each category label, and then determines the row dimension of the feature vector clusters in the noise probability transition matrix. Based on each row dimension, noise labels and non-noise labels are determined. The column values ​​of the row dimension of the feature vector clusters in the noise probability transition matrix are determined based on the number of images for each category label, and then the noise probability transition matrix is ​​determined. The distribution of noise labels and non-noise labels is measured based on the number of images for each category label, which effectively characterizes the noise confusion information between each image category.

[0088] Optionally, the training process of the unsupervised image reconstruction model also includes:

[0089] Determine whether the training of the unsupervised image reconstruction model meets the preset training conditions;

[0090] If the training of the unsupervised image reconstruction model does not meet the preset training conditions, continue to train the unsupervised image reconstruction model until it meets the preset training conditions. Then, cluster the autoencoded feature vectors of all images to be classified to obtain K feature vector clusters.

[0091] If the training of the unsupervised image reconstruction model meets the preset training conditions, the autoencoded feature vectors of all images to be classified are directly clustered to obtain K feature vector clusters.

[0092] In this context, iterative training allows for the analysis and learning of the images to be classified and their class labels, optimizing the parameters of the unsupervised image reconstruction model and improving its accuracy in representing features of the images. However, this also results in lower reliability of the unsupervised image reconstruction model in the early stages of iterative training. Furthermore, image classification models involve numerous training parameters, high computational intensity, and demanding requirements for both hardware and software conditions.

[0093] Therefore, the preset training condition in this embodiment is that the number of iterations of the unsupervised image reconstruction model reaches the target number of iterations. When the training of the unsupervised image reconstruction model meets the preset training condition, the autoencoded feature vectors of all images to be classified are clustered to obtain K feature vector clusters, so as to determine the noise probability transition matrix used to characterize the noise confusion information between each image category. While improving the accuracy of the noise probability transition matrix, the computational intensity in model training can be reduced, thereby improving the efficiency of model training.

[0094] This embodiment takes into account that the accuracy of the unsupervised image reconstruction model gradually increases with the number of training iterations, and the limitations on hardware and software conditions when the model computational intensity is high. It sets preset training conditions to supervise the training of the unsupervised image reconstruction model. When the training of the unsupervised image reconstruction model meets the preset training conditions, it starts to cluster the autoencoded feature vectors of all images to be classified, and obtains K feature vector clusters. This improves the accuracy of the noise probability transition matrix, reduces the computational intensity in model training, and improves the efficiency of model training.

[0095] The above steps involve clustering the autoencoded feature vectors of all images to be classified to obtain K feature vector clusters. Based on the images to be classified and their category labels corresponding to each autoencoded feature vector in each feature vector cluster, a noise probability transition matrix is ​​determined to characterize the noise confusion information between image categories. After clustering the autoencoded feature vectors to obtain feature vector clusters, the distribution of noise labels and non-noise labels in the images to be classified is measured based on the images to be classified and their category labels corresponding to each autoencoded feature vector in the feature vector clusters. The noise probability transition matrix effectively characterizes the noise confusion information between image categories.

[0096] Step S203: Input each image to be classified into the supervised image classification model to obtain the class probability estimation vector of each image to be classified.

[0097] The supervised image classification model is used to extract and analyze features of the images to be classified, and to determine the category probability estimation vector of each image to be classified. The category probability estimation vector consists of the probability that the corresponding image to be classified belongs to each image category. It can be used to characterize the image category of each image to be classified. The loss is calculated based on the category probability estimation vector and the category label, so as to perform iterative training of supervised image classification under the supervision of the category label and improve the accuracy of the category probability estimation vector.

[0098] In one implementation, the supervised image classification model is a residual convolutional neural network structure. That is, based on the convolutional layer, pooling layer, fully connected layer and discriminant layer of the traditional convolutional neural network, residual blocks are added to solve the problems of gradient vanishing and accuracy degradation in deep networks. This allows the supervised image classification model to become deeper and deeper, which not only ensures the accuracy of the supervised image classification model, but also improves the training efficiency of the supervised image classification model.

[0099] In this embodiment, taking the classification of human poses in images to be classified as an example, each human pose image in the training set is input into a supervised image classification model for feature extraction and feature analysis to determine the category probability estimation vector of each human pose image. This category probability estimation vector consists of the probability that the corresponding human pose image belongs to each human pose category, and can be used to characterize the human pose category of each human pose image.

[0100] The steps described above, which input each image to be classified into a supervised image classification model to obtain the category probability estimation vector of each image to be classified, involve feature extraction and feature analysis of the images to be classified, resulting in the category probability estimation vector of each image to be classified. This vector is used to characterize the image category of the image to be classified and provides a predictive basis for image classification tasks.

[0101] Step S204: Based on each image to be classified and its reconstructed image feature vector, determine the unsupervised model loss of the unsupervised image reconstruction model; based on the noise probability transition matrix, the class probability estimation vector of each image to be classified, and the class label, determine the supervised model loss of the supervised image classification model.

[0102] In this embodiment, the feature vector of each image to be classified is first determined. The unsupervised model loss of the unsupervised image reconstruction model is measured based on the similarity between the feature vector of each image to be classified and the feature vector of the corresponding reconstructed image. Correspondingly, the higher the similarity between the feature vector of the image to be classified and the feature vector of the corresponding reconstructed image, the smaller the unsupervised model loss of the unsupervised image reconstruction model, and the higher the feature representation accuracy of the unsupervised image reconstruction model for the image to be classified.

[0103] Therefore, the unsupervised image reconstruction model is trained using the unsupervised model loss to improve the accuracy of feature representation of the images to be classified, thereby improving the accuracy of the autoencoded feature vector. Since the noise transition matrix is ​​obtained based on the autoencoded feature vector of the unsupervised image reconstruction model, its accuracy can also be improved.

[0104] The category probability estimation vector can be used to predict the image category of the corresponding image to be classified, and the category label is used to represent the image category of the image to be classified. Therefore, in this embodiment, the supervised model loss of the supervised image classification model is measured based on the similarity between the category probability estimation vector and the category label of each image to be classified. Correspondingly, the higher the similarity between the category probability estimation vector and the category label of the image to be classified, the smaller the supervised model loss of the supervised image classification model, and the higher the accuracy of the supervised image classification model.

[0105] Meanwhile, this embodiment combines the noise probability transition matrix with the class probability estimation vector and class label to measure the loss of the supervised image classification model. It trains the supervised image classification model based on the autoencoding information of the unsupervised image reconstruction model and the noise confusion information of the training set, thereby reducing the impact of noise labels in the noisy training set on the supervised image classification model and improving the accuracy of the supervised image classification model.

[0106] In one embodiment, to facilitate similarity calculation, the corresponding category probability vector is first determined based on the category label of the image to be classified. The number of elements in the category probability vector is consistent with the total number of image categories. The element value at the position corresponding to the category label is 1, and the element value at other positions is 0.

[0107] For example, let N be the total number of images to be classified, and let X be the feature vector of the i-th (i = 1, 2, ..., N) image to be classified. i Let f(X) be the reconstructed image feature vector of the i-th image to be classified. i Then, based on the feature vector X of the i-th image to be classified... i The reconstructed image feature vector f(X) of the i-th image to be classified i The unsupervised sub-loss for the i images to be classified is calculated as follows:

[0108] l i1 =|X i -f(X i )|

[0109] In the formula, l i1 X is the unsupervised sub-loss for the i-th image to be classified. i Let f(X) be the feature vector of the i-th image to be classified. i ) is the reconstructed image feature vector of the i-th image to be classified.

[0110] Then, calculate the unsupervised sub-loss for all images to be classified, and sum the unsupervised sub-losses for all images to be classified. This sum is determined as the unsupervised model loss of the unsupervised image reconstruction model, which is used to train the unsupervised image reconstruction model. The unsupervised model loss is then:

[0111]

[0112] In the formula, L1 is the unsupervised model loss, l i1 Let be the unsupervised sub-loss for the i-th image to be classified, and N be the total number of images to be classified.

[0113] The i-th image to be classified corresponds to the s-th category label. Let P be the category probability vector of the i-th image to be classified. i0 Then the class probability vector P i0 = [0, ..., 1, ..., 0], where the s-th element has a value of 1 and the other elements have a value of 0.

[0114] Let A denote the noise transfer matrix, and P denote the class probability estimation vector of the i-th image to be classified. i1 For the i-th image to be classified, the noise transition matrix A and the class probability estimation vector P are then combined. i1 and category probability vector P i0 The cross-entropy between the values ​​is combined to obtain the supervised sub-loss for the i-th image to be classified:

[0115]

[0116] In the formula, l i2 Let A be the supervised sub-loss for the i-th image to be classified, and let A be the noise transition matrix. i c 1 represents the class probability estimation vector P of the i-th image to be classified. i1 The c-th element value in p i c 0 represents the class probability vector P of the i-th image to be classified. i0 The c-th element value in the array, where K is the total number of image categories.

[0117] Then, calculate the supervised sub-loss for all images to be classified, and sum the supervised sub-losses for all images to be classified. This sum is used to determine the supervised model loss of the supervised image classification model, which is then used to train the supervised image classification model. The supervised model loss is:

[0118]

[0119] In the formula, L2 is the supervised model loss, l i2 Let be the supervised loss for the i-th image to be classified, and N be the total number of images to be classified.

[0120] In this embodiment, taking the classification of human pose images as an example, the feature vector of each human pose image is first determined. The unsupervised model loss of the unsupervised image reconstruction model is measured based on the similarity between the feature vector of each human pose image and the feature vector of the corresponding reconstructed image. The unsupervised image reconstruction model is then trained based on the unsupervised model loss to improve the accuracy of the unsupervised image reconstruction model in representing the features of human pose images, thereby improving the accuracy of the autoencoded feature vector and, consequently, the accuracy of the noise transfer matrix.

[0121] Then, since the category probability estimation vector can be used to predict the human pose category of the corresponding human pose image, and the category label is used to represent the human pose category of the human pose image, this embodiment combines the noise probability transition matrix with the similarity between the category probability estimation vector and the category label of each human pose image to measure the supervised model loss of the supervised image classification model. The supervised image classification model is trained based on the autoencoding information of the unsupervised image reconstruction model and the noise confusion information of the training set, thereby reducing the impact of noise labels in the noisy human pose image dataset on the supervised image classification model and improving the accuracy of the supervised image classification model.

[0122] Optionally, before the unsupervised image reconstruction model meets the preset training conditions, the supervised model loss of the supervised image classification model is determined based on the noise probability transition matrix, the class probability estimation vector of each image to be classified, and the class label.

[0123] Obtain the preset noise probability transition identity matrix;

[0124] Based on the preset noise probability transition unit matrix, the class probability estimation vector of each image to be classified, and the class label, the supervised model loss of the supervised image classification model is determined.

[0125] In this process, the unsupervised image reconstruction model and the supervised image classification model are trained simultaneously. Before the unsupervised image reconstruction model meets the preset training conditions, the autoencoded feature vectors of all images to be classified are not clustered, and the noise probability transition matrix is ​​not determined based on the autoencoded feature vectors.

[0126] In order not to affect the training progress of the supervised image classification model, this embodiment pre-defines a noise probability transition unit matrix, which is used to combine with each image to be classified and the reconstructed image feature vector, class probability estimation vector and class label of each image to determine the model loss for training the unsupervised image reconstruction model and the supervised image classification model.

[0127] Meanwhile, in order not to affect the training results, the preset noise probability transition identity matrix is ​​an identity matrix, that is, the elements on the diagonal from the top left corner to the bottom right corner of the preset noise probability transition identity matrix are all 1, and the elements in other positions are all 0.

[0128] In this embodiment, before the unsupervised image reconstruction model meets the preset training conditions, the noise probability transition matrix is ​​not determined based on the autoencoded feature vector. Instead, a preset noise probability transition identity matrix is ​​set to determine the model loss, so that the training progress of the supervised image classification model is not affected, thus improving the training efficiency of the supervised image classification model.

[0129] Optionally, after the unsupervised image reconstruction model meets the preset training conditions, the supervised model loss of the supervised image classification model is determined based on the noise probability transition matrix, the class probability estimation vector of each image to be classified, and the class label.

[0130] Determine the noise probability transition matrix in the t-th iteration of training;

[0131] In the training from the tth iteration to the t+Tth iteration, the supervised model loss of the supervised image classification model is determined based on the noise probability transition matrix, the class probability estimation vector and class label of each image to be classified in the training of the tth iteration, where t is the preset number of iterations, T is the preset interval between iterations, and t and T are positive integers.

[0132] Determine the noise probability transition matrix in the (t+T+1)th training iteration;

[0133] During the training iterations from t+T+1 to t+2T, the supervised model loss of the supervised image classification model is determined based on the noise probability transition matrix, the class probability estimation vector of each image to be classified, and the class label in the training iteration t+T+1.

[0134] This process continues until the supervised model loss, determined in a certain iteration of training, converges.

[0135] In this embodiment, considering that unsupervised image reconstruction models and supervised image classification models have a large number of training parameters and high computational intensity, and place high demands on the hardware and software conditions for model training, the noise probability transition matrix is ​​updated in stages to reduce the computational intensity during the training process.

[0136] Specifically, once the noise probability transition matrix in the t-th iteration of training is determined, the noise probability transition matrix in the t-th iteration of training is used to calculate the supervised model loss in all iterations from the t-th to the (t+T)-th iterations. Correspondingly, in the (t+1)-th to (t+T)-th iterations of training, it is not necessary to cluster the autoencoded feature vectors to determine the corresponding noise probability transition matrix.

[0137] Then, the noise probability transition matrix in the (t+T+1)th iteration of training is determined. In the (t+T+1)th to (t+2T)th iterations of training, the noise probability transition matrix in the (t+T+1)th iteration of training is used to calculate the supervised model loss.

[0138] The specific values ​​of the initial iteration training number t for calculating the noise probability transition matrix and the iteration interval T for updating the noise probability transition matrix can be set according to the actual situation. In this embodiment, t = 30 and T = 30 are set according to the actual situation.

[0139] Similarly, the noise probability transition matrix is ​​updated in stages to calculate the loss of the supervised model.

[0140] This embodiment reduces the computational intensity and improves the training efficiency of the supervised image classification model by setting the initial number of iterations t for calculating the noise probability transition matrix and the iteration interval T for updating the noise probability transition matrix, while ensuring the accuracy of the noise probability transition matrix.

[0141] The above steps determine the unsupervised model loss of the unsupervised image reconstruction model based on each image to be classified and its reconstructed image feature vector, and determine the supervised model loss of the supervised image classification model based on the noise probability transition matrix, the class probability estimation vector of each image to be classified, and the class label. The unsupervised model loss of the unsupervised image reconstruction model is measured by the similarity between the reconstructed image feature vector and the image to be classified. The supervised model loss of the supervised image classification model is measured by combining the noise probability transition matrix with the similarity between the class probability estimation vector and the class label. This reduces the influence of noisy labels in the noisy training set on the supervised image classification model and improves the accuracy of the supervised image classification model.

[0142] Step S205: Train the unsupervised image reconstruction model based on the unsupervised model loss, and train the supervised image classification model based on the supervised model loss, until the unsupervised model loss and the supervised model loss converge, to obtain the trained unsupervised image reconstruction model and the trained supervised image classification model, and use the trained supervised image classification model to perform image classification.

[0143] Among them, model loss can measure the accuracy of the corresponding model in the corresponding training iteration, and the smaller the model loss, the higher the accuracy of the corresponding model in the corresponding training iteration.

[0144] Therefore, in the iterative training of the unsupervised image reconstruction model, the unsupervised image reconstruction model is trained based on the unsupervised model loss until the unsupervised model loss converges, resulting in a well-trained unsupervised image reconstruction model, which is used to provide more accurate autoencoded feature vectors and noise probability transition matrices for supervised image classification models.

[0145] In the iterative training of the supervised image classification model, the supervised image classification model is trained based on the supervised model loss until the supervised model loss converges, resulting in a trained supervised image classification model. This model is then used to classify the image based on the class probability estimation vector of the image to be classified, thereby improving the accuracy of image classification.

[0146] This embodiment takes the classification of human poses in images to be classified as an example. The unsupervised image reconstruction model is trained based on the unsupervised model loss, and the supervised image classification model is trained based on the supervised model loss until the unsupervised model loss and the supervised model loss converge, thus obtaining the trained unsupervised image reconstruction model and the trained supervised image classification model. The trained supervised image classification model is then used to perform the human pose classification task.

[0147] The above steps involve training an unsupervised image reconstruction model using unsupervised model loss, training a supervised image classification model using supervised model loss, and continuing until both unsupervised and supervised model losses converge, resulting in a trained unsupervised image reconstruction model and a trained supervised image classification model. The trained supervised image classification model is then used for image classification. The unsupervised model loss provides the trained unsupervised image reconstruction model with more accurate autoencoder feature vectors and noise probability transition matrices for the supervised image classification model; the supervised model loss provides the trained supervised image classification model, improving the accuracy of image classification.

[0148] This embodiment obtains an autoencoded feature vector and a reconstructed image feature vector by inputting each image to be classified into an unsupervised image reconstruction model. K feature vector clusters are obtained by clustering the autoencoded feature vectors of all images to be classified. Based on the image to be classified and its class label corresponding to each autoencoded feature vector in each feature vector cluster, a noise probability transition matrix is ​​determined to characterize the noise confusion information between image classes. Each image to be classified is then input into a supervised image classification model to obtain a class probability estimation vector for each image. The unsupervised model loss of the unsupervised image reconstruction model is determined based on each image to be classified and its reconstructed image feature vector. The supervised model loss of the supervised image classification model is determined based on the noise probability transition matrix, the class probability estimation vector, and the class label of each image to be classified. The unsupervised image reconstruction model is trained based on the unsupervised model loss, and the supervised image classification model is trained based on the supervised model loss. The process continues until both the unsupervised and supervised model losses converge, resulting in a trained unsupervised image reconstruction model and a trained supervised image classification model. The trained supervised image classification model is then used for image classification. The distribution of noise labels and non-noise labels in the images to be classified is measured by clustering the autoencoded feature vectors corresponding to each autoencoded feature vector in the feature vector clusters. The noise probability transition matrix effectively represents the noise confusion information between different image categories. The unsupervised image reconstruction model is trained based on the similarity between the reconstructed image feature vector and the image to be classified, improving the accuracy of the noise probability transition matrix. Furthermore, the noise probability transition matrix is ​​combined with the similarity between the category probability estimation vector and the category label to measure the supervised model loss of the supervised image classification model, thus reducing the influence of noise labels in the noisy training set on the supervised image classification model and improving its accuracy.

[0149] Embodiment 2 of the present invention provides an image classification method. This image classification method uses the supervised image classification model trained in Embodiment 1 of the present invention to classify images, and may include the following steps:

[0150] The system acquires the image to be classified in the image classification task, inputs the image to be classified into the trained supervised image classification model, outputs the class probability estimation vector of the image to be classified, and determines the image class of the image to be classified based on the class probability estimation vector of the image to be classified.

[0151] The image categories can be set according to the actual image classification task. For example, in scenarios such as product audience analysis and population aging analysis, the image classification task is to classify the facial attributes of the images to be classified, and the image categories can be set to various facial attribute categories such as male, female, child, youth, middle-aged, and elderly. In scenarios such as gait analysis, video surveillance, and sports science, the image classification task is to classify the human posture of the images to be classified, and the image categories can be set to various human posture categories such as still, walking, running, squatting, and jumping. In scenarios such as teaching evaluation, product sales, and personnel interviews, the image classification task is to classify the emotions of the people in the images to be classified, and the image categories can be set to various emotions such as happy, nervous, sad, disgusted, and bored. In scenarios such as renting, decorating, and buying a house, the image classification task is to classify the room style of the images to be classified, and the image categories can be set to various room styles such as pastoral, minimalist, classical, new Chinese, Mediterranean, and Southeast Asian.

[0152] This embodiment takes the classification of human poses in images as an example. After obtaining the human pose image to be classified, the human pose image is input into a trained supervised image classification model for feature extraction and analysis. The model outputs a category probability estimation vector for the human pose image. This category probability estimation vector represents the probability that the corresponding human pose image belongs to each image category. The image category corresponding to the highest probability value in the category probability estimation vector can then be determined as the image category of the human pose image, thus completing the human pose classification task.

[0153] It is understood that in the specific embodiments of this application, data related to facial images, human body images, room images, etc. are involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0154] This embodiment uses the supervised image classification model trained in Embodiment 1 of the present invention to perform feature extraction and feature analysis on the image to be classified, outputs the category probability estimation vector of the image to be classified, determines the image category of the image to be classified, and improves the classification accuracy of the image to be classified.

[0155] Corresponding to the training method of the image classification model in Example 1 above, Figure 3 A structural block diagram of the training device for the image classification model provided in Embodiment 3 of the present invention is given. For ease of explanation, only the parts related to the embodiments of the present invention are shown.

[0156] See Figure 3 The training apparatus for this image classification model includes:

[0157] The unsupervised reconstruction module 31 is used to obtain the images to be classified and their category labels in the training set. Each image to be classified is input into the unsupervised image reconstruction model to obtain the autoencoded feature vector and the reconstructed image feature vector of each image to be classified.

[0158] The matrix determination module 32 is used to cluster the autoencoded feature vectors of all images to be classified to obtain K feature vector clusters. Based on the images to be classified and their category labels corresponding to each autoencoded feature vector in each feature vector cluster, the noise probability transition matrix used to characterize the noise confusion information between image categories is determined, where K is a positive integer.

[0159] The supervised classification module 33 is used to input each image to be classified into the supervised image classification model to obtain the class probability estimation vector of each image to be classified.

[0160] The loss determination module 34 is used to determine the unsupervised model loss of the unsupervised image reconstruction model based on each image to be classified and the reconstructed image feature vector of each image to be classified, and to determine the supervised model loss of the supervised image classification model based on the noise probability transition matrix, the class probability estimation vector of each image to be classified and the class label.

[0161] The model training module 35 is used to train the unsupervised image reconstruction model based on the unsupervised model loss and the supervised image classification model based on the supervised model loss until the unsupervised model loss and the supervised model loss converge, thereby obtaining the trained unsupervised image reconstruction model and the trained supervised image classification model. The trained supervised image classification model is then used for image classification.

[0162] Optionally, the matrix determination module 32 mentioned above includes:

[0163] The condition judgment submodule is used to determine whether the training of the unsupervised image reconstruction model meets the preset training conditions;

[0164] The first clustering submodule is used to continue training the unsupervised image reconstruction model if the training of the unsupervised image reconstruction model does not meet the preset training conditions, until the unsupervised image reconstruction model meets the preset training conditions, and then cluster the autoencoded feature vectors of all images to be classified to obtain K feature vector clusters.

[0165] The second clustering submodule is used to directly cluster the autoencoded feature vectors of all images to be classified if the training of the unsupervised image reconstruction model meets the preset training conditions, and obtain K feature vector clusters.

[0166] Optionally, the matrix determination module 32 mentioned above includes:

[0167] The image quantity determination submodule is used to determine the number of images for each category label for any feature vector cluster, based on the images to be classified and their category labels corresponding to each autoencoded feature vector.

[0168] The cluster category determination submodule is used to determine the category of feature vector clusters based on the number of images for each category label;

[0169] The row dimension determination submodule is used to determine the row dimension of the feature vector cluster in the noise probability transition matrix based on the category of the feature vector cluster.

[0170] The column value determination submodule is used to determine the column value of the row dimension of the feature vector cluster in the noise probability transition matrix based on the row dimension of the feature vector cluster and the number of images of each category label.

[0171] The matrix determination submodule is used to determine the noise probability transition matrix, which represents the noise confusion information between each image category, based on the column values ​​of the row dimension corresponding to all feature vector clusters in the noise probability transition matrix.

[0172] Optionally, before the unsupervised image reconstruction model meets the preset training conditions, the loss determination module 34 includes:

[0173] The matrix preset submodule is used to obtain the preset noise probability transition identity matrix;

[0174] The first model loss calculation submodule is used to determine the supervised model loss of the supervised image classification model based on the preset noise probability transition unit matrix, the class probability estimation vector of each image to be classified, and the class label.

[0175] Optionally, after the unsupervised image reconstruction model meets the preset training conditions, the loss determination module 34 includes:

[0176] The first matrix determination submodule is used to determine the noise probability transition matrix in the t-th iteration of training.

[0177] The second model loss calculation submodule is used to determine the supervised model loss of the supervised image classification model in the training from the tth iteration to the t+Tth iteration, based on the noise probability transition matrix in the training of the tth iteration, the class probability estimation vector of each image to be classified, and the class label, where t is the preset number of iterations, T is the preset interval between iterations, and t and T are positive integers.

[0178] The second matrix determination submodule is used to determine the noise probability transition matrix in the (t+T+1)th iteration of training.

[0179] The third model loss calculation submodule is used to determine the supervised model loss of the supervised image classification model based on the noise probability transition matrix, the class probability estimation vector and class label of each image to be classified during the training from the t+T+1th to the t+2Tth iteration.

[0180] The iterative submodule is used to continue in this manner until the supervised model loss determined in a certain iteration of training converges.

[0181] Optionally, the loss determination module 34 mentioned above includes:

[0182] The unsupervised sub-loss calculation submodule is used to determine the feature vector of each image to be classified, and to determine the unsupervised sub-loss of each image to be classified based on the feature vector of each image to be classified and the feature vector of the reconstructed image.

[0183] The unsupervised model loss calculation submodule is used to determine the unsupervised model loss of the unsupervised image reconstruction model based on the unsupervised sub-losses of all images to be classified.

[0184] Optionally, the loss determination module 34 mentioned above includes:

[0185] The supervised sub-loss calculation submodule is used to calculate the supervised sub-loss for each image to be classified based on the noise probability transition matrix, the class probability estimation vector for each image to be classified, and the class label.

[0186] The supervised model loss calculation submodule is used to determine the supervised model loss of the supervised image classification model based on the supervised sub-losses of all images to be classified.

[0187] Figure 4 This is a schematic diagram of the structure of a computer device provided in Embodiment 4 of the present invention. Figure 4 As shown, the computer device of this embodiment includes: at least one processor ( Figure 4 Only one is shown in the diagram), a memory, and a computer program stored in the memory and capable of running on at least one processor, which, when executing the computer program, implements the steps in any of the above-described model training method embodiments.

[0188] This computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 4 The examples of computer devices are merely examples and do not constitute a limitation on computer devices. Computer devices may include more or fewer components than shown in the illustration, or combinations of certain components, or different components, such as network interfaces, displays, and input devices.

[0189] The processor referred to can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0190] Memory includes readable storage media, internal memory, etc., wherein internal memory can be the RAM of a computer device, providing an environment for the operation of the operating system and computer-readable instructions stored in the readable storage media. The readable storage media can be the hard drive of a computer device, or in other embodiments, it can be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, memory can include both internal storage units and external storage devices of a computer device. Memory is used to store the operating system, applications, bootloader, data, and other programs, such as program code for computer programs. Memory can also be used to temporarily store data that has been output or will be output.

[0191] Those skilled in the art will understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the functions described above can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the processes in the methods of the above embodiments by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0192] The present invention can implement all or part of the processes in the methods of the above embodiments, or it can be accomplished by a computer program product. When the computer program product is run on a computer device, the computer device executes the steps in the above method embodiments.

[0193] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0194] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0195] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0196] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0197] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for training an image classification model, the method comprising: The training method of the image classification model comprises: obtaining a to-be-classified image and a class label thereof in a training set, inputting each to-be-classified image into an unsupervised image reconstruction model to obtain an autoencoding feature vector and a reconstructed image feature vector of each to-be-classified image; performing clustering on the autoencoding feature vectors of all to-be-classified images to obtain K feature vector clustering clusters, and determining a noise probability transition matrix for representing noise confusion information between image classes according to the to-be-classified images corresponding to each autoencoding feature vector in each feature vector clustering cluster and the class labels of the to-be-classified images, wherein K is a positive integer; inputting each to-be-classified image into a supervised image classification model to obtain a class probability estimation vector of each to-be-classified image; determining an unsupervised model loss of the unsupervised image reconstruction model according to each to-be-classified image and the reconstructed image feature vector of each to-be-classified image, and determining a supervised model loss of the supervised image classification model according to the noise probability transition matrix, the class probability estimation vector of each to-be-classified image, and the class label; training the unsupervised image reconstruction model according to the unsupervised model loss and training the supervised image classification model according to the supervised model loss until the unsupervised model loss and the supervised model loss converge, obtaining a trained unsupervised image reconstruction model and a trained supervised image classification model, and performing image classification by using the trained supervised image classification model.

2. The method of claim 1, wherein, In the training process of the unsupervised image reconstruction model, further comprising: determining whether the training of the unsupervised image reconstruction model meets a preset training condition; if the training of the unsupervised image reconstruction model does not meet the preset training condition, continuing to train the unsupervised image reconstruction model until the unsupervised image reconstruction model meets the preset training condition, and then performing clustering on the autoencoding feature vectors of all to-be-classified images to obtain K feature vector clustering clusters; if the training of the unsupervised image reconstruction model meets the preset training condition, directly performing clustering on the autoencoding feature vectors of all to-be-classified images to obtain K feature vector clustering clusters.

3. The method of claim 2, wherein, Before the unsupervised image reconstruction model meets the preset training condition, the determination of the supervised model loss of the supervised image classification model according to the noise probability transition matrix, the class probability estimation vector of each to-be-classified image, and the class label comprises: obtaining a preset noise probability transition unit matrix; determining the supervised model loss of the supervised image classification model according to the preset noise probability transition unit matrix, the class probability estimation vector of each to-be-classified image, and the class label.

4. The method of claim 2 or 3, wherein, After the unsupervised image reconstruction model meets the preset training condition, the determination of the supervised model loss of the supervised image classification model according to the noise probability transition matrix, the class probability estimation vector of each to-be-classified image, and the class label comprises: determine a noise probability transition matrix in the tth iteration training; in the tth iteration training to the t+Tth iteration training, determine a supervised model loss of the supervised image classification model according to the noise probability transition matrix in the tth iteration training, the category probability estimation vector of each of the images to be classified, and the category label, wherein t is a preset iteration number, T is a preset iteration interval, and t and T are positive integers; determine a noise probability transition matrix in the t+T+1th iteration training; in the t+T+1th iteration training to the t+2Tth iteration training, determine a supervised model loss of the supervised image classification model according to the noise probability transition matrix in the t+T+1th iteration training, the category probability estimation vector of each of the images to be classified, and the category label; by analogy, until the supervised model loss determined in a certain iteration training converges. 5.The method of claim 1, wherein, determining, according to each of the self-encoding feature vectors corresponding to the images to be classified and the category labels of the images to be classified in each of the feature vector clustering clusters, a noise probability transition matrix for representing noise confusion information between image categories; for any feature vector clustering cluster, determining the number of images of each of the category labels according to the images to be classified corresponding to each of the self-encoding feature vectors and the category labels of the images to be classified; determining the category of the feature vector clustering cluster according to the number of images of each of the category labels; determining the row dimension corresponding to the feature vector clustering cluster in the noise probability transition matrix according to the category of the feature vector clustering cluster; determining the column value of the row dimension corresponding to the feature vector clustering cluster in the noise probability transition matrix according to the row dimension corresponding to the feature vector clustering cluster in the noise probability transition matrix and the number of images of each of the category labels; determining the noise probability transition matrix for representing noise confusion information between image categories according to the column values of the row dimensions corresponding to all feature vector clustering clusters in the noise probability transition matrix. 6.The method of claim 1, wherein, determining, according to each of the self-encoding feature vectors corresponding to the images to be classified and the category labels of the images to be classified in each of the feature vector clustering clusters, a noise probability transition matrix for representing noise confusion information between image categories; determining the number of images of each of the category labels according to the images to be classified corresponding to each of the self-encoding feature vectors and the category labels of the images to be classified; determining the category of the feature vector clustering cluster according to the number of images of each of the category labels; 7. The method of claim 1, wherein, determining the row dimension corresponding to the feature vector clustering cluster in the noise probability transition matrix according to the category of the feature vector clustering cluster; determining the column value of the row dimension corresponding to the feature vector clustering cluster in the noise probability transition matrix according to the row dimension corresponding to the feature vector clustering cluster in the noise probability transition matrix and the number of images of each of the category labels; determining the noise probability transition matrix for representing noise confusion information between image categories according to the column values of the row dimensions corresponding to all feature vector clustering clusters in the noise probability transition matrix.

8. A training device for an image classification model, characterized in that, determining, according to each of the self-encoding feature vectors corresponding to the images to be classified and the category labels of the images to be classified in each of the feature vector clustering clusters, a noise probability transition matrix for representing noise confusion information between image categories; determining the number of images of each of the category labels according to the images to be classified corresponding to each of the self-encoding feature vectors and the category labels of the images to be classified; determining the category of the feature vector clustering cluster according to the number of images of each of the category labels; determining the row dimension corresponding to the feature vector clustering cluster in the noise probability transition matrix according to the category of the feature vector clustering cluster; determining the column value of the row dimension corresponding to the feature vector clustering cluster in the noise probability transition matrix according to the row dimension corresponding to the feature vector clustering cluster in the noise probability transition matrix and the number of images of each of the category labels; determining the noise probability transition matrix for representing noise confusion information between image categories according to the column values of the row dimensions corresponding to all feature vector clustering clusters in the noise probability transition matrix. the training device of the image classification model comprises: The unsupervised reconstruction module is configured to obtain a to-be-classified image in a training set, input each to-be-classified image into an unsupervised image reconstruction model, and obtain an auto-encoding feature vector and a reconstructed image feature vector of each to-be-classified image; The matrix determination module is configured to cluster the auto-encoding feature vectors of all to-be-classified images to obtain K feature vector clustering clusters, and determine a noise probability transition matrix for representing noise confusion information between image categories according to the to-be-classified images corresponding to each auto-encoding feature vector in each feature vector clustering cluster and the category labels of the to-be-classified images, where K is a positive integer. The supervised classification module is configured to input each to-be-classified image into a supervised image classification model to obtain a category probability estimation vector of each to-be-classified image. The loss determination module is configured to determine an unsupervised model loss of the unsupervised image reconstruction model according to each to-be-classified image and the reconstructed image feature vector of each to-be-classified image, and determine a supervised model loss of the supervised image classification model according to the noise probability transition matrix, the category probability estimation vector of each to-be-classified image, and the category label. The model training module is configured to train the unsupervised image reconstruction model according to the unsupervised model loss, train the supervised image classification model according to the supervised model loss, until the unsupervised model loss and the supervised model loss converge, obtain a trained unsupervised image reconstruction model and a trained supervised image classification model, and perform image classification by using the trained supervised image classification model.

9. A computer device, comprising: The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor implements the image classification model training method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the image classification model training method according to any one of claims 1 to 7.

Images