A multi-label image classification method and system for non-precise labels

Abstract

The application discloses a multi-label image classification method and system for non-precise labels, and relates to the technical fields of computer vision and machine learning, and comprises the following steps: obtaining sample images to be trained, constructing an image feature matrix and a candidate label matrix; obtaining positive class prototypes and negative class prototypes of the sample images based on the candidate label matrix; obtaining a confidence parameter of the sample images based on the number of candidate labels in the sample images; constructing a confidence matrix according to the positive class prototypes, the negative class prototypes and the confidence parameter; optimizing a pseudo label matrix based on joint optimization of clustering structure, label consistency and confidence guidance according to the image feature matrix, the candidate label matrix and the confidence matrix; training a multi-label image classification model based on the optimized pseudo label matrix; and classifying and predicting new images based on the trained multi-label image classification model. The application provides an effective solution for the multi-label image classification problem involving noise labels.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and machine learning technology, and in particular relates to a method and system for classifying multi-labeled images with imprecise labels. Background Technology

[0002] Multi-label image classification is one of the core tasks in computer vision, with wide applications in fields such as intelligent surveillance, medical diagnosis, content retrieval, and e-commerce. Unlike traditional single-label classification, in multi-label classification, each image may belong to multiple categories simultaneously.

[0003] Traditional multi-label learning methods are based on an idealized assumption: the training data has complete and accurate label information. However, in practical applications, this assumption is often not met, mainly due to the following reasons: (1) the subjectivity of manual annotation, different annotators have different understandings and judgments of the same image content, resulting in inconsistent annotation results; (2) the problem of annotation omission, due to the complexity of image content or the time limit for annotation, annotators are prone to omit relevant but insignificant labels; (3) the problem of mislabeling, annotators may mistakenly label irrelevant labels as relevant, introducing false positive noise; (4) the high cost of annotation, obtaining large-scale, high-quality multi-label annotation data requires a lot of manpower and time; (5) the professional requirements of the domain, the annotation of certain professional fields (such as medical imaging) requires expert knowledge, which further increases the difficulty and cost of annotation.

[0004] To address the aforementioned issues, researchers have proposed the Partial Multi-Label Learning (PML) paradigm, aiming to compensate for inaccurate labeling through technical means. This is a weakly supervised learning approach that learns to accurately predict multi-label classifiers in imperfect environments. Current PML methods are mainly divided into two categories: (1) Label correction methods, which directly identify and remove noisy labels from the candidate label set. These methods mainly include matrix factorization, label propagation, and feature learning. However, these methods often rely on strong assumptions and have limited performance in complex noisy environments.

[0005] (2) Probabilistic modeling methods. These methods address label uncertainty by estimating label confidence levels. They mainly include confidence-weighted, expectation-maximization, and neural network-based modeling methods. However, these methods primarily focus on label-level modeling and give less consideration to the intrinsic structural information of the data.

[0006] Existing multi-label learning methods generally ignore the clustering structure information of the data, fail to fully utilize the similarity between samples, and lack a unified framework for simultaneously handling clustering and label denoising. Furthermore, traditional clustering methods are fundamentally incompatible with multi-label learning. Most traditional clustering is based on single categories, and the membership values ​​generated by clustering sum to 1 for each row (probability distribution constraint), while multi-label scenarios require binary label values ​​that can be arbitrarily summed. This incompatibility prevents the direct application of clustering techniques to multi-label scenarios, limiting the effective utilization of structural information in PML.

[0007] To address the aforementioned problems in existing technologies, there is an urgent need to propose a multi-label image classification method and system for non-precise labeling. Summary of the Invention

[0008] To address the aforementioned technical problems, this invention proposes a multi-label image classification method and system for non-precise labeling, thereby resolving the issues present in the prior art.

[0009] To achieve the above objectives, this invention provides a multi-label image classification method for non-precisely labeled images, comprising the following steps:

[0010] Obtain the sample images to be trained, and construct the image feature matrix and candidate label matrix;

[0011] Based on the candidate label matrix, the positive and negative class prototypes of each label category in the sample image are obtained using weighted clustering.

[0012] The confidence parameter of the sample image is obtained based on the number of candidate labels in the sample image;

[0013] Construct a confidence matrix based on the positive class prototype, negative class prototype, and confidence parameters;

[0014] Based on the image feature matrix, candidate label matrix, and confidence matrix, the pseudo-label matrix is ​​optimized by jointly optimizing the clustering structure, label consistency, and confidence guidance.

[0015] A multi-label image classification model is constructed, and the multi-label image classification model is trained based on the optimized pseudo-label matrix;

[0016] The new image is predicted using a trained multi-label image classification model.

[0017] Optionally, the process of acquiring sample images to be trained and constructing the image feature matrix and candidate label matrix includes:

[0018] Feature extraction is performed on the sample images to be trained to obtain the corresponding feature vectors, thereby constructing the image feature matrix;

[0019] The sample images to be trained are manually or automatically labeled to obtain candidate labels, thereby constructing a candidate label matrix, wherein the candidate labels include true labels and false labels.

[0020] Optionally, the process of obtaining the positive and negative prototypes of each label category in the sample image based on the candidate label matrix and weighted clustering includes:

[0021] For each label category, the sample images are divided into a positive class set and a negative class set based on the candidate label vectors. The feature vectors of the sample images in the positive class set are weighted and averaged to obtain the positive class prototype, and the weights are proportional to the candidate label values ​​of the sample images under the positive class prototype. The feature vectors of the sample images in the negative class set are weighted and averaged to obtain the negative class prototype, and the weights are proportional to the unlabeled values ​​of the sample images under the negative class prototype.

[0022] Optionally, the process of obtaining the confidence parameter of a sample image based on the number of candidate labels in the sample image includes:

[0023] Calculate the total number of candidate labels for a single sample image; using the total number of categories as a benchmark, take the complement of the ratio of the total number of candidate labels to the total number of categories to obtain the confidence parameter of the corresponding sample image.

[0024] Optionally, the process of constructing the confidence matrix based on the positive class prototype, the negative class prototype, and the confidence parameter includes:

[0025] For each sample image, calculate the distance from the feature vector to the positive class prototype and the distance to the negative class prototype. If the sample image is unlabeled, the confidence level is set to the highest value. If the sample image is labeled and the distance to the positive class prototype is less than the distance to the negative class prototype, the confidence level is also set to the highest value. If the sample image is labeled and the distance to the positive class prototype is greater than or equal to the distance to the negative class prototype, the confidence level is taken as the confidence parameter of the corresponding sample image.

[0026] Optionally, the process of optimizing the pseudo-label matrix based on the image feature matrix, candidate label matrix, and confidence matrix by jointly optimizing the clustering structure, label consistency, and confidence guidance includes:

[0027] A loss function is constructed, comprising a clustering structure term, a label consistency term, and a confidence guidance term. The clustering structure term uses the distance from the feature vector of the sample image to the class center as the weighted pseudo-label value. The label consistency term uses the difference between the pseudo-label value and the candidate label value as the weighted class weight matrix. The confidence guidance term uses the confidence matrix to weakly supervise the pseudo-label matrix. The loss function is used to alternately optimize the pseudo-label matrix until convergence, resulting in the optimized pseudo-label matrix.

[0028] Optionally, the process of alternately optimizing the pseudo-label matrix using a loss function until convergence, to obtain the optimized pseudo-label matrix, includes:

[0029] With the pseudo-label matrix and class weight matrix fixed, update each class center using a weighted average. With the class centers and class weight matrix fixed, solve the quadratic equation and project it onto the range of values ​​of the candidate label matrix, updating each element of the pseudo-label matrix. With the pseudo-label matrix and class centers fixed, update each element of the class weight matrix using a normalization method, so that the sum of the membership values ​​of a single sample to all classes is 1.

[0030] Optionally, the process of constructing a multi-label image classification model and training the multi-label image classification model based on the optimized pseudo-label matrix includes:

[0031] A multi-label image classification model is constructed based on a neural network. The optimized pseudo-label matrix is ​​used as the supervision information, and a hybrid loss function is used to train the multi-label image classification model. Positive sample images are targeted by pseudo-label values, and negative sample images are targeted by candidate label values.

[0032] The present invention also provides a multi-label image classification system for imprecise labeling, for implementing the method, comprising:

[0033] The feature extraction module is used to acquire sample images to be trained and construct image feature matrices and candidate label matrices;

[0034] The weighted clustering module is used to obtain the positive and negative class prototypes of each label category in the sample image based on the candidate label matrix and weighted clustering.

[0035] The credibility acquisition module is used to obtain the credibility parameter of a sample image based on the number of candidate labels in the sample image;

[0036] The confidence building module is used to construct a confidence matrix based on the positive class prototype, negative class prototype, and confidence parameters.

[0037] The matrix optimization module is used to optimize the pseudo-label matrix based on the image feature matrix, candidate label matrix, and confidence matrix by jointly optimizing the clustering structure, label consistency, and confidence guidance.

[0038] The model training module is used to construct a multi-label image classification model and train the multi-label image classification model based on the optimized pseudo-label matrix.

[0039] The image prediction module is used to perform multi-label classification prediction on new images based on the trained multi-label image classification model.

[0040] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method.

[0041] Compared with the prior art, the present invention has the following advantages and technical effects:

[0042] This invention constructs a sample representation space by extracting image feature matrices and simultaneously establishes a noisy candidate label matrix as weak supervision information, forming a dual-channel data foundation of "feature-label" and solving the noise sensitivity problem caused by traditional methods relying on only a single information source.

[0043] This invention calculates positive and negative prototypes for each marker category in the feature space, and establishes geometric constraints by comparing the distance from the sample to the two prototypes, thereby achieving explicit quantitative identification of noise markers and breaking through the limitations of traditional methods that rely solely on statistical assumptions.

[0044] This invention calculates the confidence parameter by dynamically relating the number of candidate labels in a sample to the total number of categories, automatically downweights high-frequency labeled samples, solves the systematic bias caused by "over-labeling" in manual annotation, and improves the fine granularity of noise detection.

[0045] This invention integrates prototype comparison with confidence parameters to construct a confidence matrix, which serves as a soft constraint in the optimization process, enabling gradual correction of noise labels and avoiding information loss caused by hard threshold decisions.

[0046] This invention designs a loss function that includes three components: cluster structure preservation, label consistency constraint, and confidence guidance. By solving the loss function through an alternating optimization algorithm, a denoised pseudo-label matrix is ​​generated that both preserves the multi-label characteristics and conforms to the data clustering structure, thus solving the fundamental compatibility problem between traditional clustering and multi-label learning.

[0047] This invention employs a hybrid loss function that differentiates between positive sample pseudo-label values ​​and negative sample candidate label values ​​to train a classifier. This maximizes the use of the original label information while maintaining noise robustness, thus achieving end-to-end noise-tolerant learning.

[0048] Ultimately, this invention provides a unified and effective solution for the problem of classifying multi-labeled images with noisy labels through a complete technical chain of "feature extraction - prototype construction - confidence evaluation - joint optimization - classification training", achieving significant performance improvements in multiple application fields such as natural scene images, e-commerce product images, and medical images. Attached Figure Description

[0049] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0050] Figure 1 This is a schematic diagram of the method flow according to an embodiment of the present invention. Detailed Implementation

[0051] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0052] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here. Example 1

[0053] like Figure 1 As shown, this embodiment provides a multi-label image classification method for non-precisely labeled images, including the following steps:

[0054] Obtain the sample images to be trained, and construct the image feature matrix and candidate label matrix;

[0055] Based on the candidate label matrix, the positive and negative class prototypes of each label category in the sample image are obtained using weighted clustering.

[0056] The confidence parameter of the sample image is obtained based on the number of candidate labels in the sample image;

[0057] Construct a confidence matrix based on the positive class prototype, negative class prototype, and confidence parameters;

[0058] Based on the image feature matrix, candidate label matrix, and confidence matrix, the pseudo-label matrix is ​​optimized by jointly optimizing the clustering structure, label consistency, and confidence guidance.

[0059] A multi-label image classification model is constructed, and the multi-label image classification model is trained based on the optimized pseudo-label matrix;

[0060] The new image is predicted using a trained multi-label image classification model.

[0061] The feasible process of acquiring sample images to be trained and constructing image feature matrices and candidate label matrices includes:

[0062] Feature extraction is performed on the training sample images to obtain corresponding feature vectors, thereby constructing an image feature matrix; manual or automatic annotation is performed on the training sample images to obtain candidate labels, thereby constructing a candidate label matrix, wherein the candidate labels include true labels and false labels.

[0063] As a specific implementation method, training image data is acquired, and for the input n images, an instance feature matrix is ​​obtained using traditional or deep feature extraction methods. ,in is the feature vector of the i-th sample image, n is the number of instances, and d is the feature dimension.

[0064] Constructing a candidate label matrix ,in, Let represent the candidate label vector of the i-th sample image, and further, let the j-th class candidate label value of the i-th sample be . , This indicates that the i-th sample has been labeled with a candidate label of the j-th label category. This indicates that the i-th sample is not labeled, and q is the number of labeled categories. Y contains both true labels and inaccurate labels.

[0065] An feasible process for obtaining the positive and negative prototypes of each label category in a sample image based on the candidate label matrix and weighted clustering includes:

[0066] For each label category, the sample images are divided into a positive class set and a negative class set based on the candidate label vectors. The feature vectors of the sample images in the positive class set are weighted and averaged to obtain the positive class prototype, and the weights are proportional to the candidate label values ​​of the sample images under the positive class prototype. The feature vectors of the sample images in the negative class set are weighted and averaged to obtain the negative class prototype, and the weights are proportional to the unlabeled values ​​of the sample images under the negative class prototype.

[0067] As a specific implementation method, based on the clustering hypothesis, weighted clustering is performed using the candidate label matrix Y to calculate the positive class sample centers for each label category, i.e., the positive class prototype matrix. ,in Let the j-th class positive prototype vector be calculated using the following formula: ,in, It represents the membership weight of the i-th sample to the j-th labeled category. Simultaneously, a negative class prototype matrix is ​​constructed. ,in The negative prototype vector of class j is calculated using the following formula: ,in It is the non-membership weight of the i-th sample with respect to the j-th label category.

[0068] An feasible process for obtaining the confidence parameter of a sample image based on the number of candidate labels in the sample image includes:

[0069] Calculate the total number of candidate labels for a single sample image; using the total number of categories as a benchmark, take the complement of the ratio of the total number of candidate labels to the total number of categories to obtain the confidence parameter of the corresponding sample image.

[0070] As a specific implementation method, a relatively accurate labeling confidence parameter θ is constructed for each sample. Although each sample image can be labeled with multiple categories, for samples belonging to too many categories, it is worth noting that the probability of this sample being mislabeled is higher. For the confidence parameter of the i-th sample... .

[0071] The feasible process of constructing a confidence matrix based on the positive class prototype, negative class prototype, and confidence parameters includes:

[0072] For each sample image, calculate the distance from the feature vector to the positive class prototype and the distance to the negative class prototype. If the sample image is unlabeled, the confidence level is set to the highest value. If the sample image is labeled and the distance to the positive class prototype is less than the distance to the negative class prototype, the confidence level is also set to the highest value. If the sample image is labeled and the distance to the positive class prototype is greater than or equal to the distance to the negative class prototype, the confidence level is taken as the confidence parameter of the corresponding sample image.

[0073] As a specific implementation method, the distance between each sample feature vector and its positive and negative prototypes is compared, and a confidence matrix is ​​constructed by combining the obtained relatively accurate label confidence parameter θ. For the i-th sample If a sample is not labeled by the j-th category, or is labeled and is closer to the prototype of the j-th category, then the sample is trusted and receives a high confidence value of 1. Otherwise, its confidence is estimated using a confidence parameter, calculated using the following formula:

[0074] ;

[0075] The feasible process of optimizing the pseudo-label matrix based on the image feature matrix, candidate label matrix, and confidence matrix by jointly optimizing the clustering structure, label consistency, and confidence guidance includes:

[0076] A loss function is constructed, comprising a clustering structure term, a label consistency term, and a confidence guidance term. The clustering structure term uses the distance from the feature vector of the sample image to the class center as the weighted pseudo-label value. The label consistency term uses the difference between the pseudo-label value and the candidate label value as the weighted class weight matrix. The confidence guidance term uses the confidence matrix to weakly supervise the pseudo-label matrix. The loss function is used to alternately optimize the pseudo-label matrix until convergence, resulting in the optimized pseudo-label matrix.

[0077] Furthermore, the process of alternately optimizing the pseudo-label matrix using a loss function until convergence, to obtain the optimized pseudo-label matrix, includes:

[0078] With the pseudo-label matrix and class weight matrix fixed, update each class center using a weighted average. With the class centers and class weight matrix fixed, solve the quadratic equation and project it onto the range of values ​​of the candidate label matrix, updating each element of the pseudo-label matrix. With the pseudo-label matrix and class centers fixed, update each element of the class weight matrix using a normalization method, so that the sum of the membership values ​​of a single sample to all classes is 1.

[0079] As a specific implementation method, in order to obtain a more accurate label matrix than the candidate label matrix Y for supervised classification model training, a loss function is constructed to optimize the pseudo-label matrix F. . This is a pseudo-label matrix. This is a learnable class weight matrix. This is a learnable class prototype matrix. The loss function mainly consists of three parts, the first part... , used to learn the clustering structure of data, where for The j-th column vector represents the class center of the j-th class, and the class weight of the i-th sample for the j-th class. The pseudo-label value of the i-th sample for the j-th class The product of the samples is For the membership value of the j-th class, the membership values ​​follow a probability distribution, and the sum of the membership values ​​of each sample for all classes is 1. The second part balances the relationship between the pseudo-label value and the class weights. The further the pseudo-label value is from the candidate label value, the higher the probability that it is a noise label, and the smaller the corresponding class weight; otherwise, the opposite is true. The third part uses the confidence matrix to weakly supervise the pseudo-label matrix F. The higher the confidence, the higher the pseudo-label value... The more reliable the value, the closer it tends to a value of 1. This is a copy of F.

[0080] The formula for the loss function is shown below:

[0081] ;

[0082] The feasible process of constructing a multi-label image classification model and training the multi-label image classification model based on the optimized pseudo-label matrix includes:

[0083] A multi-label image classification model is constructed based on a neural network. The optimized pseudo-label matrix is ​​used as the supervision information, and a hybrid loss function is used to train the multi-label image classification model. Positive sample images are targeted by pseudo-label values, and negative sample images are targeted by candidate label values.

[0084] As a specific implementation method, classifier training and prediction use an optimized pseudo-label matrix F to train a neural network classifier with a hybrid loss function H(f,y,p)=fln(p)+(1−y)ln(1−p), where p is the classifier output: positive samples use the pseudo-label f, and negative samples use the original candidate label y to perform multi-label classification prediction on the test image. The total training loss function is... :

[0085] ;

[0086] Where t(.) is the tanh activation function and s(.) is the sigmoid loss function. and For the total model network parameters, and Let be the network parameters for the k-th labeled category. Based on the trained network parameters, the new example... Substitute into the formula Obtain the tag vector . Example 2

[0087] The method in Example 1 is applied to natural scene image classification. The method includes the following steps:

[0088] Step 110: Obtain training image data:

[0089] This embodiment uses a dataset containing 5000 natural scene images, each with a resolution of 256×256 pixels. These images may simultaneously contain multiple semantic tags such as "sky," "ocean," "buildings," "vegetation," "roads," "vehicles," and "people." Scale-Invariant Feature Transform (SIFT) is used to extract local features, and then a Bag of Words model is used to generate a 1000-dimensional feature vector, forming a feature matrix X∈R^(1000×5000).

[0090] Alternatively, deep learning feature extraction methods can be used, such as using a pre-trained ResNet-50 network to extract 2048-dimensional deep feature vectors.

[0091] Step 120: Construct the candidate label matrix:

[0092] The corresponding candidate label matrix Y∈{0,1}^(5000×7) is obtained, which contains 7 label categories. Due to the inaccuracy of manual annotation, the candidate label set contains an average of 3.2 labels, of which about 35% are noise labels. For example, the candidate labels for a beach image may include ["sky", "ocean", "building" (noise)], where the "building" label is mislabeled.

[0093] Step 130: Calculate the prototypes of the positive and negative classes:

[0094] For each labeled category j, calculate the positive class prototype. and negative class prototype Prototype calculation for positive classes: The weight Taking the "sky" class as an example, the positive prototype of the "sky" class is obtained by weighted averaging of image features containing "sky" in all candidate labels. Negative class prototype calculation: Among them, weight .

[0095] Step 140: Calculate the confidence parameter θ:

[0096] Although each sample image can be labeled with multiple categories, for samples belonging to too many categories, it is more likely that the sample has been mislabeled. The confidence parameter for the i-th sample... For example, if a sample is labeled with 5 labels (out of a total of 7 categories), then θi = 1 - 5 / 7 ≈ 0.286, indicating that the sample has low confidence.

[0097] Step 150: Construct the confidence matrix:

[0098] Based on the distance comparison from the instance sample to the positive and negative prototypes and the confidence parameter, construct the confidence matrix B:

[0099] ;

[0100] Specifically, Euclidean distance is used to calculate the distance from the sample feature vector to the prototype.

[0101] Step 160: Optimize the pseudo-label matrix:

[0102] Construct a loss function for joint optimization:

[0103] ;

[0104] Alternating optimization algorithm is used:

[0105] Sub-step 161: Update the learnable class prototype matrix O:

[0106] ;

[0107] Sub-step 162: Update the pseudo-label matrix F:

[0108] For each element Solve the quadratic equation:

[0109] ;

[0110] Take the positive root and project it onto Interval.

[0111] Sub-step 163: Update the class weight matrix H:

[0112] ;

[0113] in ;

[0114] With parameters α=0.1 and β=1.0, the algorithm converged after 15 iterations.

[0115] Step 170: Classifier Training and Prediction:

[0116] A three-layer feedforward neural network was trained using the denoised pseudo-label matrix F. The input layer had 1000 neurons, the hidden layer had 512 neurons using the tanh activation function, and the output layer had 7 neurons using the sigmoid activation function.

[0117] Using a hybrid loss function:

[0118] ;

[0119] Where H(f,y,p)=f*ln(p)+(1-y)*ln(1-p).

[0120] When making predictions on the test image, the probability value of each label is output, and a threshold of 0.5 is set for binarization.

[0121] Experimental results:

[0122] On the test set, the average accuracy using this method reached 82.3%, which is an improvement of 13.8 percentage points compared to the baseline method trained with raw noise labels (68.5%). Example 3

[0123] The method in Example 1 is applied to e-commerce product image classification:

[0124] (1) Application scenario: For 50,000 product images on e-commerce platforms, each image may contain multi-dimensional tags such as "color", "material", "style", "target audience" and "occasion". Due to the inaccuracy of batch uploads by merchants or automatic annotation, about 30% of the candidate tags are noise tags.

[0125] (2) Technical implementation:

[0126] Feature extraction: 2048-dimensional deep features were extracted using a pre-trained ResNet-50 network; Labeling categories: 34 categories, including 12 colors, 8 styles, 6 materials, 5 occasions, and 3 groups of people; Parameter settings: α=0.05, β=2.0; Network structure: 2048-1024-512-34 multilayer perceptron.

[0127] (3) Optimization effect:

[0128] Tagging accuracy improved from 67.2% to 85.1%, user search matching accuracy improved by 19.3%, and the recommendation system click-through rate improved by 14.2%. Example 4

[0129] Based on the same general inventive concept, this invention also provides a multi-label image classification system for non-precise labeling. The system provided by this invention is described below, and the system described below can be referred to in conjunction with the method described above. The system includes:

[0130] The feature extraction module is used to acquire sample images to be trained and construct image feature matrices and candidate label matrices;

[0131] The weighted clustering module is used to obtain the positive and negative class prototypes of each label category in the sample image based on the candidate label matrix and weighted clustering.

[0132] The credibility acquisition module is used to obtain the credibility parameter of a sample image based on the number of candidate labels in the sample image;

[0133] The confidence building module is used to construct a confidence matrix based on the positive class prototype, negative class prototype, and confidence parameters.

[0134] The matrix optimization module is used to optimize the pseudo-label matrix based on the image feature matrix, candidate label matrix, and confidence matrix by jointly optimizing the clustering structure, label consistency, and confidence guidance.

[0135] The model training module is used to construct a multi-label image classification model and train the multi-label image classification model based on the optimized pseudo-label matrix.

[0136] The image prediction module is used to perform multi-label classification prediction on new images based on the trained multi-label image classification model. Example 5

[0137] This embodiment also discloses a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in Embodiment 1.

[0138] The computer device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device may include, but is not limited to, a processor and memory.

[0139] The processor can be a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC), etc.

[0140] The memory may include volatile memory or non-volatile memory, or both. Example 6

[0141] This embodiment also discloses a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method described in Embodiment 1.

[0142] The computer-readable storage medium includes both permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Example 7

[0143] This embodiment also discloses a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in Embodiment 1.

[0144] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A multi-label image classification method for imprecisely labeled images, characterized in that, Includes the following steps: Obtain the sample images to be trained, and construct the image feature matrix and candidate label matrix; Based on the candidate label matrix, the positive and negative class prototypes of each label category in the sample image are obtained by weighted clustering. The confidence parameter of the sample image is obtained based on the number of candidate labels in the sample image; Construct a confidence matrix based on the positive class prototype, negative class prototype, and confidence parameters; Based on the image feature matrix, candidate label matrix, and confidence matrix, the pseudo-label matrix is ​​optimized by jointly optimizing the clustering structure, label consistency, and confidence guidance. A multi-label image classification model is constructed, and the multi-label image classification model is trained based on the optimized pseudo-label matrix; Perform multi-label classification prediction on new images based on the trained multi-label image classification model; The process of optimizing the pseudo-label matrix based on the image feature matrix, candidate label matrix, and confidence matrix, through joint optimization of cluster structure, label consistency, and confidence guidance, includes: A loss function is constructed, comprising a clustering structure term, a label consistency term, and a confidence guidance term. The clustering structure term uses the distance from the feature vectors of the sample images to the class centers as the weighted pseudo-label values. The label consistency term uses the difference between the pseudo-label values ​​and the candidate label values ​​as the weighted class weight matrix. The confidence guidance term uses the confidence matrix to weakly supervise the pseudo-label matrix. The loss function is then used to alternately optimize the pseudo-label matrix until convergence, resulting in the optimized pseudo-label matrix. The process of constructing a multi-label image classification model and training the model based on the optimized pseudo-label matrix includes: A multi-label image classification model is constructed based on a neural network. The optimized pseudo-label matrix is ​​used as the supervision information, and a hybrid loss function is used to train the multi-label image classification model. Positive sample images are targeted by pseudo-label values, and negative sample images are targeted by candidate label values.

2. The method according to claim 1, characterized in that, The process of acquiring sample images to be trained and constructing the image feature matrix and candidate label matrix includes: Feature extraction is performed on the sample images to be trained to obtain the corresponding feature vectors, thereby constructing the image feature matrix; The sample images to be trained are manually or automatically labeled to obtain candidate labels, thereby constructing a candidate label matrix, wherein the candidate labels include true labels and false labels.

3. The method according to claim 2, characterized in that, Based on the candidate label matrix, the process of obtaining the positive and negative prototypes of each label category in the sample image using weighted clustering includes: For each label category, the sample images are divided into a positive class set and a negative class set based on the candidate label vectors. The feature vectors of the sample images in the positive class set are weighted and averaged to obtain the positive class prototype, and the weights are proportional to the candidate label values ​​of the sample images under the positive class prototype. The feature vectors of the sample images in the negative class set are weighted and averaged to obtain the negative class prototype, and the weights are proportional to the unlabeled values ​​of the sample images under the negative class prototype.

4. The method according to claim 1, characterized in that, The process of obtaining the confidence parameter of a sample image based on the number of candidate labels in the sample image includes: Calculate the total number of candidate labels for a single sample image; using the total number of categories as a benchmark, take the complement of the ratio of the total number of candidate labels to the total number of categories to obtain the confidence parameter of the corresponding sample image.

5. The method according to claim 1, characterized in that, The process of constructing the confidence matrix based on the positive class prototype, negative class prototype, and confidence parameters includes: For each sample image, calculate the distance from the feature vector to the positive class prototype and the distance to the negative class prototype. If the sample image is unlabeled, the confidence level is set to the highest value. If the sample image is labeled and the distance to the positive class prototype is less than the distance to the negative class prototype, the confidence level is also set to the highest value. If the sample image is labeled and the distance to the positive class prototype is greater than or equal to the distance to the negative class prototype, the confidence level is taken as the confidence parameter of the corresponding sample image.

6. The method according to claim 1, characterized in that, The process of alternately optimizing the pseudo-label matrix using a loss function until convergence, to obtain the optimized pseudo-label matrix, includes: With the pseudo-label matrix and class weight matrix fixed, update each class center using a weighted average. With the class centers and class weight matrix fixed, solve the quadratic equation and project it onto the range of values ​​of the candidate label matrix, updating each element of the pseudo-label matrix. With the pseudo-label matrix and class centers fixed, update each element of the class weight matrix using a normalization method, so that the sum of the membership values ​​of a single sample to all classes is 1.

7. A multi-label image classification system for imprecise labeling, characterized in that, For implementing the method of any one of claims 1-6, comprising: The feature extraction module is used to acquire sample images to be trained and construct image feature matrices and candidate label matrices; The weighted clustering module is used to obtain the positive and negative class prototypes of each label category in the sample image based on the candidate label matrix and weighted clustering. The credibility acquisition module is used to obtain the credibility parameter of a sample image based on the number of candidate labels in the sample image; The confidence building module is used to construct a confidence matrix based on the positive class prototype, negative class prototype, and confidence parameters. The matrix optimization module is used to optimize the pseudo-label matrix based on the image feature matrix, candidate label matrix, and confidence matrix by jointly optimizing the clustering structure, label consistency, and confidence guidance. The model training module is used to construct a multi-label image classification model and train the multi-label image classification model based on the optimized pseudo-label matrix. The image prediction module is used to perform multi-label classification prediction on new images based on the trained multi-label image classification model; The process of optimizing the pseudo-label matrix based on the image feature matrix, candidate label matrix, and confidence matrix, through joint optimization of cluster structure, label consistency, and confidence guidance, includes: A loss function is constructed, comprising a clustering structure term, a label consistency term, and a confidence guidance term. The clustering structure term uses the distance from the feature vectors of the sample images to the class centers as the weighted pseudo-label values. The label consistency term uses the difference between the pseudo-label values ​​and the candidate label values ​​as the weighted class weight matrix. The confidence guidance term uses the confidence matrix to weakly supervise the pseudo-label matrix. The loss function is then used to alternately optimize the pseudo-label matrix until convergence, resulting in the optimized pseudo-label matrix. The process of constructing a multi-label image classification model and training the model based on the optimized pseudo-label matrix includes: A multi-label image classification model is constructed based on a neural network. The optimized pseudo-label matrix is ​​used as the supervision information, and a hybrid loss function is used to train the multi-label image classification model. Positive sample images are targeted by pseudo-label values, and negative sample images are targeted by candidate label values.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-6.

Images