Multi-label learning method, device and equipment based on sample missing label enhancement

Abstract

The application relates to a multi-label learning method, device and equipment based on sample missing label enhancement, which comprises the following steps: acquiring a training data set of a missing label sample; pre-processing the training data set to obtain a processed training set with restored real labels; learning and aggregating the processed training set by using an algorithm adaptation strategy to obtain a multi-label learning classifier; taking the classifier as a label prediction model; and inputting a sample to be predicted into the label prediction model to obtain labels corresponding to the sample to be predicted. The method realizes label information enhancement by obtaining the processed training set with restored real labels; then the processed training set is induced by using the algorithm adaptation strategy to obtain a classifier considering the class imbalance problem in the processed training set; and the label prediction model is constructed based on the classifier to solve the multi-label class imbalance problem and improve the precision and accuracy of the predicted labels.

Description

Technical Field

[0001] This application relates to the field of labeling technology, and in particular to a multi-label learning method, apparatus and device based on sample missing label enhancement. Background Technology

[0002] Humans generate billions of bytes of data every day, leading to an increasing need for innovative work to effectively address the enormous challenges of multi-label learning brought about by big data. Multi-label learning, also known as multi-label classification, assigns multiple labels to each image instance simultaneously, and is crucial in various fields ranging from protein function classification and document classification to automatic image classification. However, for multi-label learning, collecting fully supervised data—that is, completely accurate and complete data—for supervised learning is usually difficult and expensive, often accompanied by high time costs. Therefore, how to model label dependencies and handle incomplete supervision under limited supervision to achieve efficient and accurate multi-label classification is a key bottleneck that urgently needs to be addressed in real-world classification tasks.

[0003] Currently, there are two main types of methods for multi-label learning with missing labels: those based on low-rank and those based on graph assumptions. The existence of label correlation usually implies that the output space is low-rank, and this assumption has been widely used to fill in missing items in label completion tasks. This is beneficial because it helps two key objectives in label-missing MLML: label correlation extraction and missing label completion. The multi-label classification problem with missing labels can be considered as a low-rank matrix completion problem with side information, i.e., features. The label matrix is ​​viewed as the output of a classifier matrix with side information, constrained by low rank, and the problem is generalized to a flexible empirical risk minimization framework. Typically, the classifier is decomposed into two low-rank matrices, and another optimization method is used to effectively handle large-scale problems. In this case, the existence of tail labels has a certain probability of breaking the low-rank property. Therefore, tail labels are considered outliers, and the output label matrix is ​​decomposed into two low-rank sparse label matrices, and then another corresponding optimization problem is solved to obtain the ideal classifier. Low-rank assumptions can be utilized in various ways. Based on low rank, using embedding methods to project labels into a low-dimensional space before training the classifier is also a widely used method. However, training sets for multi-label learning often suffer from class imbalance, which can lead to reduced efficiency and generalization performance in classifier training. Methods based on the low-rank hypothesis cannot currently handle class imbalance well.

[0004] Existing multi-label learning methods for missing labels can also employ graph-based models to address the missing multi-label problem. Let G = (V, E, W) represent a weighted graph, where V represents the vertex set, E represents the edge set, and W is a weight matrix. In a defined graph, the most typical strategy is to add a manifold regularization term within an empirical risk minimization framework. That is, construct a label-specific graph for each label from feature-induced similarity graphs. Then, add a manifold regularization term to each label distribution. The manifold regularization term generally needs to satisfy three assumptions: label consistency, sample-level smoothness, and label-level smoothness. Constraining these three conditions makes the results closer to the true label matrix. Graph information is mainly used to eliminate ambiguity in incomplete supervision and involves different techniques to utilize label correlations. Many graph-based methods focus only on the sample-level smoothness principle, while some works focus on label smoothness. Furthermore, deep learning models are used to eliminate missing label ambiguity by constructing a fully connected graph with labels as vertices and then training a graph neural network (GNN) to model label dependencies. The input to the GNN is the feature vector extracted from the image by a convolutional neural network, and the output is the predicted label. We utilize partial binary cross-entropy loss to reduce the normalization factor based on the label ratio. We employ a learning-like strategy similar to learning a self-progressing model to fill in the missing terms.

[0005] In summary, the two methods described above have dominated multi-label learning problems with missing labels to date. While some recent work has attempted to improve performance using deep models, these primarily involve trivial convolutional networks and autoencoders. Since machine learning datasets only provide logical labels, these methods train classifiers by directly utilizing the logical labels and feature information, then test the model's performance by classifying unknown examples. However, logical information often contains only the relevance of labels to examples, lacking information on the relative relevance of labels belonging to the same instance and the relative irrelevance of labels not belonging to the same instance. Summary of the Invention

[0006] This application provides a multi-label learning method, apparatus, and device based on sample missing label enhancement, which solves the technical problem that existing multi-label learning processes for missing labels suffer from class imbalance, resulting in inaccurate output predicted labels.

[0007] To achieve the above objectives, the embodiments of this application provide the following technical solutions:

[0008] On the one hand, a multi-label learning method based on missing sample labels is provided, including the following steps:

[0009] Obtain the training dataset for samples with missing labels;

[0010] The training dataset is preprocessed to obtain a processed training set that recovers the true labels;

[0011] Based on the training set, an algorithmic adaptation strategy is used to classify and aggregate the data to obtain a multi-label learning classifier.

[0012] The classifier is used as a label prediction model; the sample to be predicted is input into the label prediction model to obtain the label corresponding to the sample to be predicted.

[0013] Preferably, preprocessing the training dataset to obtain a processed training set for recovering the true labels includes:

[0014] The training dataset is optimized using a low-rank constrained tensor singular value decomposition method to obtain the correlation data of the missing label samples.

[0015] The training dataset is processed using a mapping model to obtain mapped data;

[0016] Based on the mapping data, the correlation data, and the training dataset, an optimization model is constructed.

[0017] The optimization model is iteratively updated and converged using the first-order gradient quasi-Newton method to obtain the optimal mapping parameters of the label distribution; based on the optimal mapping parameters, the mapping data is processed by thresholding to obtain the processed training set for recovering the true labels.

[0018] Preferably, the training dataset is optimized using a low-rank constrained tensor singular value decomposition method to obtain the relevance data of the missing label samples, including:

[0019] Based on the feature matrix, label matrix, and tensor nuclear norm of the training dataset, an augmented Lagrange equation is constructed.

[0020] The correlation data is obtained by iteratively updating the calculation based on the augmented Lagrange equation;

[0021] The augmented Lagrange equation is:

[0022]

[0023]

[0024] In the formula, To assist tensor variables, All are Lagrange multipliers, X is the feature matrix of the training dataset, and Y is the label matrix of the training dataset. , It is by , The third-order tensors constructed separately, Let F be the norm of the matrix. These all represent balance coefficients with different values. For correlation data, E represents the corruption of samples with missing labels.

[0025] Preferably, the optimization model is:

[0026]

[0027] In the formula, n is the total number of missing labeled samples in the training dataset, and y i Let be the q-dimensional logical binary label vector of the i-th missing label sample. , This is the weight matrix. For bias vectors, For relevance data, Y is the label matrix of the training dataset, I is the identity matrix, and T is the transpose of the matrix. , The Gaussian kernel function embeds the d-dimensional real-valued vector of the i-th missing-label sample into a high-dimensional space. Let F be the norm of the matrix. The label distribution matrix, These are the balancing parameters.

[0028] Preferably, the processing training set for recovering the true labels by thresholding the mapping data according to the optimal mapping parameters includes:

[0029] The mapping data is processed according to the optimal mapping parameters to obtain the mapping label distribution;

[0030] The mapping label distribution is normalized to obtain the label distribution and the label distribution matrix;

[0031] The labels in the label distribution are processed using a threshold processing formula to obtain the true labels;

[0032] The training set is constructed by combining the n missing label samples with the corresponding real labels.

[0033] The threshold processing formula is as follows:

[0034]

[0035]

[0036] In the formula, y l For the label of class l, Y represents the true label of column j. j Let j be the set of all labels in the j-th column of the label matrix. Let y be the label distribution of the l-th class in column j. l* The label with the highest label distribution value.

[0037] Preferably, the classification and aggregation processing based on the processed training set using an algorithm adaptation strategy to obtain a multi-label learning classifier includes:

[0038] The training set is classified according to positive missing label samples and negative missing label samples for each type of label to obtain a classification dataset.

[0039] From the classification dataset, select the true labels of class l and class k, and cross-couple the true labels of class k with those of class l to obtain the coupled label dataset. ;

[0040] K multi-class imbalanced learners are used to learn from all the coupled datasets and then coupled again to obtain a multi-label learned classifier.

[0041] Preferably, the multi-label learning method based on missing label enhancement includes: obtaining a threshold constant, and processing each class of real labels using the real-valued mapping function of the classifier to obtain the prediction confidence corresponding to each class of real labels; and distinguishing the type of missing label samples corresponding to each class of real labels based on whether the prediction confidence is greater than the threshold constant.

[0042] On the other hand, a multi-label learning device based on sample missing label enhancement is provided, including a data acquisition module, a preprocessing module, a learning aggregation module, and a prediction output module;

[0043] The data acquisition module is used to acquire the training dataset of samples with missing labels;

[0044] The preprocessing module is used to preprocess the training dataset to obtain a processed training set that recovers the true labels;

[0045] The learning aggregation module is used to classify and aggregate the training set using an algorithmic adaptation strategy to obtain a multi-label learning classifier.

[0046] The prediction output module is used to use the classifier as a label prediction model; the sample to be predicted is input into the label prediction model to obtain the label corresponding to the sample to be predicted.

[0047] Preferably, the preprocessing module is further configured to optimize the training dataset using a low-rank constrained tensor singular value decomposition method to obtain relevance data of missing label samples; process the training dataset using a mapping model to obtain mapping data; construct an optimization model based on the mapping data, the relevance data, and the training dataset; perform iterative update convergence calculation on the optimization model using a first-order gradient quasi-Newton method to obtain the optimal mapping parameters of the label distribution; and process the mapping data using thresholding based on the optimal mapping parameters to obtain a processed training set for recovering the true labels.

[0048] On the other hand, a terminal device is provided, including a processor and a memory;

[0049] The memory is used to store program code and transmit the program code to the processor;

[0050] The processor is configured to execute the aforementioned multi-label learning method based on sample missing label enhancement according to the instructions in the program code.

[0051] As can be seen from the above technical solutions, the embodiments of this application have the following advantages: The multi-label learning method, apparatus, and device based on sample missing label enhancement includes: acquiring a training dataset of samples with missing labels; preprocessing the training dataset to obtain a processed training set that recovers the true labels; using an algorithmic adaptation strategy to learn and aggregate the processed training set to obtain a multi-label learning classifier; using the classifier as a label prediction model; and inputting the sample to be predicted into the label prediction model to obtain the label corresponding to the sample to be predicted. This multi-label learning method based on sample missing label enhancement achieves label information enhancement by obtaining a processed training set that recovers the true labels; then, it uses an algorithmic adaptation strategy to induce the processed training set to obtain a classifier that considers the class imbalance problem in the processed training set; and it builds a label prediction model based on the classifier to solve the problem of multi-label class imbalance, and also improves the accuracy and precision of the predicted labels, thus solving the technical problem that existing multi-label learning processes for missing labels suffer from class imbalance, leading to inaccurate output predicted labels. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1This is a flowchart illustrating the steps of the multi-label learning method based on sample missing label enhancement as described in the embodiments of this application;

[0054] Figure 2 This is a schematic diagram of the label prediction model in the multi-label learning method based on sample missing label enhancement described in the embodiments of this application;

[0055] Figure 3 This is a framework diagram of a multi-label learning device based on sample missing label enhancement according to an embodiment of this application. Detailed Implementation

[0056] To make the inventive objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0057] In the description of the embodiments of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0058] In the embodiments of this application, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0059] This application provides a multi-label learning method, apparatus, and device based on missing label enhancement. It employs a two-stage approach, separating preprocessing and predictive classifier induction into separate steps, decomposing the problem to be solved and processing them sequentially, thus better refining the problem. By recovering the true labels from the obtained label distribution, the method offers greater interpretability and reliability. The predictive classifier's induction algorithmic adaptation strategy effectively utilizes label correlation while also addressing the class imbalance problem. This solves the technical problem of inaccurate output label predictions due to class imbalance in existing multi-label learning processes for missing labels.

[0060] Example 1:

[0061] Figure 1 This is a flowchart illustrating the steps of the multi-label learning method based on missing label enhancement described in the embodiments of this application. Figure 2 This is a schematic diagram of the label prediction model in the multi-label learning method based on missing label enhancement described in the embodiments of this application.

[0062] like Figure 1 As shown in the figure, this application provides a multi-label learning method based on sample missing label enhancement, including the following steps:

[0063] S1. Obtain the training dataset for samples with missing labels.

[0064] It should be noted that step S1 involves obtaining the training dataset. 'x' refers to the real-valued feature vector corresponding to the image obtained by feature extraction and vectorization of the missing-label sample (usually an image) through a neural network. It can be understood as all the features of the image. The label vector 'y' represents the relevant information of all existing class labels for the image; labels belonging to the image are set to 1, and those not belonging to the image are set to 0. Each missing-label sample (each image) corresponds to such a label vector, thus the label matrix reflects the multi-label classification of the missing-label sample. The missing-label sample x of the i-th instance... i Represented by a d-dimensional real-valued feature vector, this instance corresponds to a q-dimensional logical binary label vector. That is, there are a total of Missing label samples of instances, q-class labels. Theoretically, represented by... Feature matrix, and using Characteristic label matrix. Represents the label matrix of the first... A term in row j and column j . Then it can be considered that the first Each instance is associated with the j-th type of label. Then it can be considered that the first An instance is considered to have a missing label if it is irrelevant to the j-th label or its relevance is uncertain. This multi-label learning method, based on sample missing label enhancement, will generate a missing label matrix. Restore to the true label matrix The classifier is learned through real sample information and is used to predict the label matrix of unknown instances.

[0065] S2. Preprocess the training dataset to obtain the processed training set that recovers the true labels.

[0066] It should be noted that step S2 processes the training dataset obtained in step S1 to obtain a processed training set that restores the true label matrix, providing data for the subsequent classifier. This multi-label learning method based on missing sample label augmentation ensures that the label distribution closely resembles the true label information by obtaining the processed training set. After augmenting the label information, the credibility of the label to the instance sample can be obtained. This credibility representation can represent information that a single logical value cannot represent. It not only represents the credibility of the label to the instance, but also reflects the relative correlation between labels to the instance sample. With this credibility, the recovered labels can be obtained by threshold screening.

[0067] S3. Based on the processed training set, an algorithmic adaptation strategy is used for learning and aggregation to obtain a multi-label learning classifier.

[0068] It should be noted that in step S3, the data in the training set obtained in step S2 is processed to obtain a classifier suitable for multi-label learning. This allows the multi-label learning method based on missing sample label enhancement to adopt an algorithmic adaptation strategy to transform problems that arise during label prediction that are different from those in general. This strategy induces multiple class imbalance learners by randomly coupling with other labels. Each learner corresponds to a random pair of labels. The predictions of these learners for unseen samples are aggregated to determine whether the corresponding label is a classifier for related samples. Through the classifier, the problem of class imbalance can be well taken into account while utilizing label relevance.

[0069] S4. Use the classifier as the label prediction model; input the sample to be predicted into the label prediction model to obtain the label corresponding to the sample to be predicted.

[0070] It should be noted that in step S4, a label prediction model is constructed based on the allocator that can balance multiple labels obtained in step S3. This multi-label learning method based on missing label enhancement can use the label prediction model to predict the label of the sample to be predicted, and obtain the predicted label corresponding to the sample to be predicted.

[0071] This application provides a multi-label learning method based on missing label enhancement. The method includes: obtaining a training dataset of samples with missing labels; preprocessing the training dataset to obtain a processed training set that recovers the true labels; using an algorithmic adaptation strategy to learn and aggregate the processed training set to obtain a multi-label learning classifier; using the classifier as a label prediction model; and inputting the sample to be predicted into the label prediction model to obtain the label corresponding to the sample. This multi-label learning method based on missing label enhancement enhances label information by obtaining a processed training set that recovers the true labels; then, it uses an algorithmic adaptation strategy to induce the processed training set to obtain a classifier that considers the class imbalance problem in the processed training set; and finally, it constructs a label prediction model based on the classifier to solve the multi-label class imbalance problem and improve the accuracy and precision of the predicted labels. This solves the technical problem of inaccurate output labels caused by class imbalance in existing multi-label learning processes for missing labels.

[0072] In one embodiment of this application, preprocessing the training dataset to obtain a processed training set for recovering the true labels includes:

[0073] The training dataset is optimized using a low-rank constrained tensor singular value decomposition method to obtain the correlation data of samples with missing labels.

[0074] The training dataset is processed using a mapping model to obtain the mapping data;

[0075] Build an optimization model based on the mapping data, correlation data, and training dataset;

[0076] The optimization model is iteratively updated and converged using the first-order gradient quasi-Newton method to obtain the optimal mapping parameters of the label distribution; based on the optimal mapping parameters, the mapping data is processed by thresholding to obtain the processed training set for recovering the true labels.

[0077] The optimized model is:

[0078]

[0079] In the formula, n is the total number of missing labeled samples in the training dataset, and y i Let be the q-dimensional logical binary label vector of the i-th missing label sample. , This is the weight matrix. For bias vectors, For correlation data, Y is the label matrix of the training dataset, I is the identity matrix, and T is the transpose of the matrix. , The Gaussian kernel function embeds the d-dimensional real-valued vector of the i-th missing-label sample into a high-dimensional space. Let F be the norm of the matrix. The label distribution matrix, These are the balancing parameters.

[0080] It should be noted that this multi-label learning method based on missing label enhancement first preprocesses the training dataset using tensor singular value decomposition based on low-rank constraints, and then uses existing sample logistic label information and feature information to supervise the recovery of the label distribution, thus obtaining a processed training set with recovered true labels. The efficient first-order gradient quasi-Newton method, also known as the finite-memory BFGS method, is similar to the solution of low-rank representations and is also an existing method; therefore, the details of the first-order gradient quasi-Newton method will not be elaborated upon.

[0081] In this embodiment, the training dataset is optimized using a low-rank constrained tensor singular value decomposition method to obtain the relevance data of the missing label samples, including:

[0082] Based on the feature matrix, label matrix, and tensor nuclear norm of the training dataset, an augmented Lagrange equation is constructed.

[0083] The correlation data is obtained by iteratively updating the calculation based on the augmented Lagrange equation;

[0084] The augmented Lagrange equation is:

[0085]

[0086]

[0087] In the formula, To assist tensor variables, All are Lagrange multipliers, X is the feature matrix of the training dataset, and Y is the label matrix of the training dataset. , It is by , The third-order tensors constructed separately, Let F be the norm of the matrix. These all represent balance coefficients with different values. For correlation data, E represents the corruption of samples with missing labels.

[0088] It should be noted that this multi-label learning method based on missing label augmentation can utilize missing label samples x i As an example, we first use a mapping model to convert the training dataset into mapped data. , It is by For parameterized linear transformations, the mapped data can be represented by the first formula, which is:

[0089]

[0090] In the first formula, in order to obtain the optimal solution... The first formula can be optimized using an objective function, which is:

[0091]

[0092] In the formula, For loss function, To uncover the underlying information about the correlation between samples, For the balance parameter. and The detailed description includes: It is the loss function between the logical label and the label distribution. The recovered label distribution information should be close to the existing logical label information. For example, the logical label of a sample lacking a label is... To restore the label distribution as Therefore, there is reason to infer... and Both are close to 0. It is close to 1. Since the prior information about the true label distribution is implicit, therefore... The loss function can be transformed into the least squares loss (LS) function formula as follows:

[0093] .

[0094] and This is a crucial part of the label distribution recovery process. Specifically, if we use the second formula to represent the global sample correlation, then the second formula is:

[0095] .

[0096] Since unlabeled samples can be represented by a linear combination of relevant samples, global sample correlations in the feature space can be explored by minimizing the low-rank representation LRR. Sample correlations are obtained by applying a low-rank representation to the feature space, aiming to find the LRR of the feature matrix X to mine the global structure in the feature space. Therefore, assuming This leads to the rank minimization problem, which is expressed using the third formula:

[0097]

[0098] In the formula, E represents the damage of the sample with missing labels. It is the imbalance coefficient that balances the influence of the two parts on the lower rank. (Using...) If we use norms to handle the corruption of missing-label samples, then:

[0099] .

[0100] Since the third formula is a non-convex function, for ease of optimization, it is replaced by a kernel function, thus transforming the third formula into the fourth formula:

[0101]

[0102] In the formula, Let be the nuclear norm of matrix C. It is C's A singular value.

[0103] In this embodiment, during the construction of the augmented Lagrange equation, in addition to mining the underlying information of sample features, it is also necessary to utilize existing logical label information to improve the performance and accuracy of the label enhancement process. Therefore, this multi-label learning method based on sample missing label enhancement fully mines the implicit information of data samples based on tensor singular value decomposition with low-rank tensor constraints. Thus, the fourth formula is transformed into the fifth formula, which is:

[0104]

[0105] In the formula, and It is by , The third-order tensors constructed separately, This represents the singular value decomposition of tensors based on the tensor nuclear norm. It can be represented as:

[0106]

[0107] In the formula, Indicates along Third-dimensional Fast Fourier Expansion express The p-th diagonal element, where f is the subscript symbol for the singular value decomposition of the tensor. It can be calculated using the sixth formula, which is:

[0108]

[0109] In the formula, U, V and V are the singular value decompositions of the tensor, respectively.

[0110] Based on the unitary invariance of the matrix nuclear norm, we can Further transformed into:

[0111]

[0112] in Will Expanding into a block diagonal form is as follows:

[0113]

[0114] Considering the Fourier transform, a block-based circulant matrix can be directly block-diagonalized, therefore Represented as:

[0115] .

[0116] Therefore, this multi-label learning method based on missing sample labels utilizes tensor singular value decomposition based on low-rank tensor constraints to integrate the underlying information of existing logical labels into the formation process of sample correlation, and constructs an augmented Lagrange equation.

[0117] In the embodiments of this application, the processing training set for recovering the true labels is obtained by thresholding the mapping data according to the optimal mapping parameters.

[0118] The mapping data is processed according to the optimal mapping parameters to obtain the mapping label distribution;

[0119] The mapping label distribution is normalized to obtain the label distribution and the label distribution matrix;

[0120] A threshold processing formula is used to process the labels in the label distribution to obtain the true labels;

[0121] The training set is constructed by combining the n missing label samples with the corresponding real labels.

[0122] The threshold processing formula is:

[0123]

[0124]

[0125] In the formula, y l For the label of class l, Y represents the true label of column j. j Let j be the set of all labels in the j-th column of the label matrix. Let y be the label distribution of the l-th class in column j. l* The label with the highest label distribution value.

[0126] It should be noted that the optimal mapping parameters in the mapping data are obtained through iterative optimization using the optimization model. To pass unknown samples through Mapping yields the corresponding label distribution Because the label distribution needs to meet constraints, it is necessary to... Normalization is performed using softmax normalization. The resulting label distribution is then... and label distribution matrix The constraints are:

[0127] .

[0128] In this embodiment, the recovered label matrix is ​​thresholded to obtain a value greater than a certain threshold. The labels are the real labels. To prevent the output from being an empty set, a thresholding formula is used to output the label corresponding to the maximum label distribution value. The training set is then processed as follows: , Let be the true labels of all classes in the i-th missing label sample.

[0129] In one embodiment of this application, a multi-label classifier is obtained by learning and aggregating the training set using an algorithmic adaptation strategy, including:

[0130] The training set is classified according to positive missing label samples and negative missing label samples for each type of label to obtain a classification dataset.

[0131] Select the true labels of class l and class k from the classification dataset, and cross-couple the true labels of class k with the true labels of class l to obtain the coupled label dataset. ;

[0132] By using K multi-class imbalanced learners to learn from all coupled datasets and then coupling them again, a multi-label learned classifier is obtained.

[0133] It should be noted that the classification dataset for:

[0134]

[0135] In the formula, y j This refers to the positive and negative values ​​of all missing label samples for the j-th label class. In this embodiment, the classification dataset... The dataset contains q classes of labels. In the classification dataset, one class's true label is selected as the true label for class l. Then, this class l true label is removed from the classification dataset, and the remaining true labels are selected as the true label for class k. Finally, the true labels for class k and class l are cross-coupled to obtain the coupled label dataset. ,Right now:

[0136]

[0137] In the formula, y k Let be the true label of the k-th class K. Before obtaining the coupled dataset, the classification dataset needs to be set to the given label. In this case, the classification dataset is generalized to a multi-class training set. Multiple training sets The expression is:

[0138] .

[0139] In the embodiments of this application, during the process of obtaining a multi-label learning classifier, K random multi-class imbalanced learners are used. By Applications in A multi-class classifier can be obtained. .

[0140] In one embodiment of this application, a multi-label learning method based on missing label enhancement includes: obtaining a threshold constant, and processing each class of real labels using the real-valued mapping function of the classifier to obtain the prediction confidence corresponding to each class of real labels; and distinguishing the type of missing label sample corresponding to each class of real labels based on whether the prediction confidence is greater than the threshold constant.

[0141] It should be noted that the real-valued mapping function is:

[0142]

[0143] In the formula, For the missing label template x relative to the j-th type label The prediction confidence of positive samples (without considering x relative to the given information) (Whether it is a positive or negative sample).

[0144] In this embodiment, k types of tags are randomly selected for each type of tag. Cross-coupled with it, then real-valued mapping function Aggregate the prediction confidence scores of these K imbalanced multi-class classifiers. To obtain the predicted logistic labels, set the threshold function to a constant function. In other words, for a certain type of label The prediction confidence is greater than the threshold constant. The samples with positive missing labels can be considered as positive missing label samples, and vice versa.

[0145] In this embodiment of the application, for the threshold constant To evaluate the performance of this multi-label learning method based on missing label augmentation, the F1 score is used. The F1 score is commonly used to evaluate the performance of binary classifiers, especially in cases of class skewness. Indicates that the application will be used The F1 score achieved on the binary classification training set is:

[0146]

[0147]

[0148] Threshold constant This is determined by maximizing the corresponding F1 value, i.e.:

[0149] .

[0150] Example 2:

[0151] Figure 3 This is a flowchart illustrating the framework of the multi-label learning device based on sample missing label enhancement as described in the embodiments of this application.

[0152] like Figure 3 As shown, this application provides a multi-label learning device based on sample missing label enhancement, including a data acquisition module 10, a preprocessing module 20, a learning aggregation module 30, and a prediction output module 40;

[0153] Data acquisition module 10 is used to acquire the training dataset of samples with missing labels;

[0154] Preprocessing module 20 is used to preprocess the training dataset to obtain a processed training set that recovers the true labels;

[0155] The learning aggregation module 30 is used to learn and aggregate based on the training set using an algorithm adaptation strategy to obtain a multi-label learning classifier;

[0156] The prediction output module 40 is used to use the classifier as a label prediction model; the sample to be predicted is input into the label prediction model to obtain the label corresponding to the sample to be predicted.

[0157] In this embodiment, the preprocessing module 20 is further configured to optimize the training dataset using a low-rank constrained tensor singular value decomposition method to obtain the relevance data of the missing label samples; process the training dataset using a mapping model to obtain mapping data; construct an optimization model based on the mapping data, relevance data, and training dataset; perform iterative update convergence calculation on the optimization model using the first-order gradient quasi-Newton method to obtain the optimal mapping parameters of the label distribution; and process the mapping data using thresholding based on the optimal mapping parameters to obtain a processed training set for recovering the true labels.

[0158] It should be noted that the modules in the device of Embodiment 2 correspond to the steps in the method of Embodiment 1. The content of the multi-label learning method based on sample missing label enhancement has been described in detail in Embodiment 1, and the content of the modules in the device will not be described in detail in this Embodiment 2.

[0159] Example 3:

[0160] This application provides a terminal device, including a processor and a memory;

[0161] Memory is used to store program code and transfer the program code to the processor;

[0162] The processor is used to execute the aforementioned multi-label learning method based on missing label enhancement according to the instructions in the program code.

[0163] It should be noted that the processor is used to execute the steps in the above-described embodiment of a multi-label learning method based on sample missing label enhancement, according to the instructions in the program code. Alternatively, when the processor executes the computer program, it implements the functions of each module / unit in the above-described system / device embodiments.

[0164] For example, a computer program can be divided into one or more modules / units, one or more of which are stored in memory and executed by a processor to complete this application. One or more modules / units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in a terminal device.

[0165] Terminal devices can be computing devices such as desktop computers, laptops, handheld computers, and cloud servers. Terminal devices may include, but are not limited to, processors and memory. Those skilled in the art will understand that this does not constitute a limitation on the terminal device, which may include more or fewer components than illustrated, or combinations of certain components, or different components. For example, a terminal device may also include input / output devices, network access devices, buses, etc.

[0166] The processor referred to can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0167] Memory can be an internal storage unit of a terminal device, such as a hard drive or RAM. Memory can also be an external storage device, such as a plug-in hard drive, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, memory can include both internal and external storage units. Memory is used to store computer programs and other programs and data required by the terminal device. Memory can also be used for temporary storage of data that has been output or will be output.

[0168] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0169] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0170] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0171] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0172] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0173] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A multi-label learning method based on missing sample label augmentation, characterized in that, Includes the following steps: Obtain the training dataset of missing label samples, wherein the missing label samples are images; The training dataset is preprocessed to obtain a processed training set that recovers the true labels; Based on the training set, an algorithmic adaptation strategy is used to classify and aggregate the data to obtain a multi-label learning classifier. The classifier is used as a label prediction model; the sample to be predicted is input into the label prediction model to obtain the label corresponding to the sample to be predicted; The training dataset is preprocessed to obtain a processed training set for recovering the true labels, including: The training dataset is optimized using a low-rank constrained tensor singular value decomposition method to obtain the correlation data of the missing label samples. The training dataset is processed using a mapping model to obtain mapped data; Based on the mapping data, the correlation data, and the training dataset, an optimization model is constructed. The optimization model is iteratively updated and converged using the first-order gradient quasi-Newton method to obtain the optimal mapping parameters of the label distribution; based on the optimal mapping parameters, the mapping data is processed by thresholding to obtain the processing training set for recovering the true labels; Based on the training set, an algorithmic adaptation strategy is used for classification and aggregation to obtain a multi-label learning classifier, including: The training set is classified according to positive missing label samples and negative missing label samples for each type of label to obtain a classification dataset. From the classification dataset, select the true labels of class l and class k, and cross-couple the true labels of class k with those of class l to obtain the coupled label dataset. ; K multi-class imbalanced learners are used to learn all the coupled datasets and then coupled again to obtain a multi-label learned classifier; The optimization model is as follows: In the formula, n is the total number of missing labeled samples in the training dataset, and y i Let be the q-dimensional logical binary label vector of the i-th missing label sample. , This is the weight matrix. For bias vectors, For relevance data, Y is the label matrix of the training dataset, I is the identity matrix, and T is the transpose of the matrix. , The Gaussian kernel function embeds the d-dimensional real-valued vector of the i-th missing-label sample into a high-dimensional space, || || F Let F be the norm of the matrix. The label distribution matrix, For balancing parameters.

2. The multi-label learning method based on sample missing label enhancement according to claim 1, characterized in that, The training dataset is optimized using a low-rank constrained tensor singular value decomposition method to obtain the relevance data of the missing label samples, including: Based on the feature matrix, label matrix, and tensor nuclear norm of the training dataset, an augmented Lagrange equation is constructed. The correlation data is obtained by iteratively updating the calculation based on the augmented Lagrange equation; The augmented Lagrange equation is: In the formula, To assist tensor variables, All are Lagrange multipliers, X is the feature matrix of the training dataset, and Y is the label matrix of the training dataset. , It is by , The third-order tensors constructed respectively, || || F Let F be the norm of the matrix. These all represent balance coefficients with different values. For correlation data, E represents the corruption of samples with missing labels.

3. The multi-label learning method based on sample missing label enhancement according to claim 1, characterized in that, The mapping data is processed using thresholding based on the optimal mapping parameters to obtain a processing training set for recovering the true labels, including: The mapping data is processed according to the optimal mapping parameters to obtain the mapping label distribution; The mapping label distribution is normalized to obtain the label distribution and the label distribution matrix; The labels in the label distribution are processed using a threshold processing formula to obtain the true labels; The training set is constructed by combining the n missing label samples with the corresponding real labels. The threshold processing formula is as follows: In the formula, y l For the label of class l, Y represents the true label of column j. j Let j be the set of all labels in the j-th column of the label matrix. Let y be the label distribution of the l-th class in column j. l* The label with the highest label distribution value.

4. The multi-label learning method based on sample missing label enhancement according to claim 1, characterized in that, include: Obtain the threshold constant and process each class of real labels using the real-valued mapping function of the classifier to obtain the prediction confidence corresponding to each class of real labels; The type of missing label sample corresponding to each class of real labels is distinguished based on whether the predicted confidence level is greater than a threshold constant.

5. A multi-label learning device based on sample missing label augmentation, characterized in that, It includes a data acquisition module, a preprocessing module, a learning aggregation module, and a prediction output module; The data acquisition module is used to acquire the training dataset of missing label samples, wherein the missing label samples are images; The preprocessing module is used to preprocess the training dataset to obtain a processed training set that recovers the true labels; The learning aggregation module is used to classify and aggregate the training set using an algorithmic adaptation strategy to obtain a multi-label learning classifier. The prediction output module is used to use the classifier as a label prediction model; the sample to be predicted is input into the label prediction model to obtain the label corresponding to the sample to be predicted; The preprocessing module is also used to optimize the training dataset using a low-rank constrained tensor singular value decomposition method to obtain the correlation data of the missing label samples. The training dataset is processed using a mapping model to obtain mapped data; Based on the mapping data, the correlation data, and the training dataset, an optimization model is constructed. The optimization model is iteratively updated and converged using the first-order gradient quasi-Newton method to obtain the optimal mapping parameters of the label distribution; based on the optimal mapping parameters, the mapping data is processed by thresholding to obtain the processing training set for recovering the true labels; Based on the training set, an algorithmic adaptation strategy is used for classification and aggregation to obtain a multi-label learning classifier, including: The training set is classified according to positive missing label samples and negative missing label samples for each type of label to obtain a classification dataset. From the classification dataset, select the true labels of class l and class k, and cross-couple the true labels of class k with those of class l to obtain the coupled label dataset. ; K multi-class imbalanced learners are used to learn all the coupled datasets and then coupled again to obtain a multi-label learned classifier; The optimization model is as follows: In the formula, n is the total number of missing labeled samples in the training dataset, and y i Let be the q-dimensional logical binary label vector of the i-th missing label sample. , This is the weight matrix. For bias vectors, For relevance data, Y is the label matrix of the training dataset, I is the identity matrix, and T is the transpose of the matrix. , The Gaussian kernel function embeds the d-dimensional real-valued vector of the i-th missing-label sample into a high-dimensional space, || || F Let F be the norm of the matrix. The label distribution matrix, For balancing parameters.

6. A terminal device, characterized in that, Including the processor and memory; The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the multi-label learning method based on sample missing label enhancement as described in any one of claims 1-4 according to the instructions in the program code.

Images