A multi-instance multi-label learning method based on combined error correction coding strategy

Abstract

The application discloses the technical field of coding strategy and relates to a multi-instance multi-label learning method based on a combined error correction coding strategy. The multi-instance multi-label learning method based on the combined error correction coding strategy comprises the following steps: a feature embedding representation method of multi-instance bag level data structure information is designed based on a Fisher kernel, and an original multi-instance multi-label learning problem is converted into a multi-label learning problem; under the guidance of an error correction output coding idea, a coding strategy of randomly connecting multi-label sub-problems is proposed; a support vector machine model is constructed and trained, and hard labels representing the specific category performance of a predicted sample under each label are obtained based on a T criterion. The multi-instance multi-label learning method based on the combined error correction coding strategy effectively alleviates the class imbalance problem. In addition, the model divides different training data blocks based on coding random division to learn base classifiers, so that the similarity between the base classifiers is weakened, and the classification coding error correction capability is stronger.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of coding strategy, in particular, to a multi-instance multi-label learning method based on a combined error correction coding strategy. BACKGROUND

[0002] Multi-instance multi-label learning (MIML) is a kind of supervised learning method combining multi-instance learning (MIL) and multi-label learning (MLL). In the multi-instance learning framework, each data in the data set is regarded as a bag, and the input space is more flexible expressed through the association between the bag and the examples contained therein. Multi-label learning associates data with multiple labels, and expresses a more diverse output space through a set of binary classes. Taking image recognition classification as an example, a complete image is a bag, and the pixel blocks therein are examples, and each image has one or more class labels. Extracting pixel information to express the image and predicting the image class label based thereon is a typical MIML task. Nowadays, the MIML paradigm has been applied to computer vision, audio positioning mining, biomedical diagnosis and other practical problems.

[0003] In multi-instance multi-label learning, there are some problems that exist universally: the labels exist on the bag, and the examples constituting the bag have no label indication, so the information relied on by the supervised learning task such as classification needs to be obtained from the examples or the representation of the examples. If the examples are directly used for training, a reasonable method is needed to label the examples or identify the key examples triggering the labels, which requires a certain time consumption; if the example information is extracted for embedding to represent the bag, an effective way is needed to preserve the rich data structure feature information. In addition, since the samples do not belong to a certain specific label, there may be data imbalance problems at the bag level or the example level. In this unbalanced scenario, the model needs to consider how to better identify the minority class, and problems such as class overlap, data noise, and insufficient information will be magnified to increase the difficulty of learning, and at the same time, the shortcomings of the algorithm may also be covered up by the unbalanced problem and show false excellent performance.

[0004] Based on different focuses, many algorithms have been proposed: MIML degradation is a way to solve MIML tasks, that is, the original MIML problem is transformed into a MIL problem or a MLL problem, and then solved by means of the corresponding field method. Representative methods include MIMLBOOST and MIMLSVM. Specifically, the MIMLBOOST algorithm converts each MIML sample (B i , Y i ) into One MIL sample makes the original MIML task transform into a MIL task. Then, MIMLBOOST adopts MIBOOSTING, a MIL algorithm, to transform the multi-instance learning task into a traditional supervised learning task by assuming that each example in the bag contributes equally and independently to the bag label, and successfully resolves the MIML, a generalized multi-instance learning problem. While the MIMLSVM algorithm transforms each MIML sample (B i , Y i ) into a multi-label sample (τ(B i ), Y i ), where τ(·) is an embedding function that maps the bag B i into a single instance by means of structural clustering, so that the original multi-instance multi-label task is transformed into a multi-label task. In detail, the MIMLSVM algorithm performs k-medoids clustering on the training sample set at the bag level, and the distance between each bag and the class center is the component of B , and then uses the MLSVM algorithm to decompose the MLL task into several independent binary supervised learning problems. Subsequently, many algorithms have been improved on the basis of the above two classical algorithms. For example, the MIMLSVMmi algorithm replaces the multi-label learning algorithm MIBOOSTING adopted in the MIMLBOOST method with MISVM, a multi-instance algorithm; and the MIMLNN algorithm replaces the MLSVM adopted in the MIMLSVM with a two-layer neural network.

[0005] In addition, EnMIMLNN incorporates the idea of ensemble learning into the MIMLNN algorithm, replaces the BP neural network in the MIMLNN algorithm with an RBF neural network, and uses the ensemble Hausdorff distance to measure the distance between two samples. This allows the EnMIMLNN algorithm to consider the contribution of all domains and reflect the relationship between different domains, thereby solving the problem caused by the degradation of the MIML learning framework. At the same time, the EnMIMLNN algorithm is not very sensitive to outliers and has robust performance, but its prediction performance is poor when actual labeled examples are available. Therefore, Liu et al. designed the MK-EnMIMLNN algorithm based on the EnMIMLNN algorithm by learning a neural network that fuses multiple kernel functions. Specifically, the MK-EnMIMLNN algorithm improves the radial basis kernel function, an activation function used in the EnMIMLNN algorithm, to an artificially constructed kernel function that fuses the Gaussian kernel and the multivariate quadratic kernel. While taking advantage of the good local anti-interference ability of the Gaussian kernel function to the noise existing in the data, the multivariate quadratic kernel function makes up for the lack of global anti-interference ability caused by the limited range of parameters, so as to balance the learning ability and the generalization ability.

[0006] In terms of the relationship between examples, bags, and labels, Li et al. proposed the KISAR algorithm based on the consideration that highly correlated labels usually share some information, explored the reasonable relationship between input patterns and output labels, and identified key examples that triggered labels. Yang et al. believed that it was not rigorous to simply mark all unlabeled training samples as negative classes. In many practical applications, there is often a weak label problem in machine learning tasks, that is, users only mark a part of positive labels, and unlabeled labels are not necessarily negative. Therefore, under the assumption that highly correlated labels share some common examples, the MIMLWEL algorithm in the weak label scenario is proposed. In addition, with the development of deep learning, more and more deep learning frameworks are applied to explore the potential relationship between examples, bags, and labels. Yang et al. proposed the MIML-LLMC algorithm based on parallel attention and local label manifold correlation, which can discover example-label relationships and construct local label manifold correlation vectors based on label manifold generation and clustering; Su et al. proposed the CM2AL algorithm to combine bag-level classifiers and example-level classifiers, use uncertainty, label correlation, label space sparsity, and information obtained from queried bag instances to select the most informative bag-label pairs, thereby avoiding careful examination of all labels and improving algorithm efficiency.

[0007] As for the data imbalance problem, more and more methods and techniques have been proposed to adjust data distribution and enhance the model's ability to identify minority classes, thereby improving classification performance and model robustness. Common methods to solve the data imbalance problem include:

[0008] ① Resampling: Resampling methods aim to balance the class distribution by modifying the training set (i.e., oversampling / undersampling) or filter noise (i.e., cleaning resampling); ② Reweighting: Reweighting methods assign different weights to different samples to balance the attention of the classifier to different number of populations; ③ Ensemble: Ensemble methods improve the recognition ability of minority classes by combining multiple specific models. Liu et al. proposed the SPE algorithm to combine the resampling method and the ensemble learning framework, on the one hand, the ensemble method effectively reduces the variance introduced by resampling, on the other hand, the ensemble classifier trained using the training dataset generated by undersampling self-paced coordination data hardness (i.e., the difficulty of data classification under the corresponding model) makes the algorithm obtain robust performance even under highly overlapping classes and extremely skewed distribution; ④ Meta-learning method: With the development of meta-learning, more and more researchers apply it to solve the problem of data imbalance. For example, Learning to Teach learns a dynamic loss function, L2RW / Meta-Weight-Net learns an implicit / explicit data weighting function. In 2020, Liu et al. proposed the MESA algorithm to learn a meta-sampler to decouple model training and meta-training, which directly learns a sampling strategy from data, adaptively iteratively resamples the training set, obtains multiple classifiers, and forms a cascade ensemble model. However, so far, almost all algorithms have not considered the data imbalance problem that exists universally in multi-instance multi-label problems, and directly learning a classifier on the training data will result in a single learning classifier, ignoring the information contained in the minority class, and reducing the classification performance of the method. In addition, when the number of labels is large, the dimension of the class space is higher, and if the label space is sparse, the degree of data imbalance will increase. SUMMARY

[0009] The purpose of the present application is to provide a multi-instance multi-label learning method based on combined error correction coding strategy, to solve the problem of data imbalance in multi-instance multi-label classification task, and the technical problem that the traditional method often directly learns a classifier on the training data, resulting in a single model and easily ignoring the information contained in the minority class samples, thereby reducing the classification performance of the model. The original multi-instance multi-label learning problem is converted into a multi-label learning problem; then, in view of the problem of insufficient diversity of the base classifier of the multi-label classification model, the multi-label problem is converted into multiple sub-problems, and the complete classification model is trained based on the random selection and combination of the error correction coding corresponding to the sub-problems, to enhance the diversity of the multi-label classifier and effectively alleviate the class imbalance problem. In addition, the model randomly divides different training data blocks based on coding to learn the base classifier, so that the similarity between the base classifiers is weakened, and the error correction ability of the classification coding is stronger.

[0010] In order to achieve the above purpose, the technical scheme of the present application is as follows:

[0011] The application provides a multi-instance multi-label learning method based on a combined error correction coding strategy, and comprises the following steps:

[0012] S1. A feature embedding representation method of multi-instance bag level data structure information is designed based on a Fisher kernel, and the original multi-instance multi-label learning problem is converted into a multi-label learning problem;

[0013] S2. Under the guidance of an error correction output coding idea, a coding strategy of randomly connecting multi-label sub-problems is proposed;

[0014] S3. A support vector machine model is constructed and trained, and a hard label representing the specific category performance of a predicted sample under each label is obtained based on a T criterion.

[0015] As an aspect of the multi-instance multi-label learning method based on the combined error correction coding strategy, wherein S1 comprises the following steps:

[0016] S11. A vector expression of a bag based on instance information;

[0017] S12. A bag embedding method based on a Fisher kernel.

[0018] As an aspect of the multi-instance multi-label learning method based on the combined error correction coding strategy, wherein S11 comprises the following steps:

[0019] S111. A sample is a bag composed of n i instances. According to a classical hypothesis in multi-instance multi-label learning, instances i in the bag B are independently and identically distributed, a probability density function p of generating the instance is described by using a parameter θ, and the bag B i can be represented by a gradient vector based on a probability generation model:

[0020]

[0021] S112. Based on the gradient vector, a Fisher kernel function is used to measure the similarity between two samples, namely:

[0022]

[0023] Wherein, F θ is a Fisher information matrix of the probability density function p, namely:

[0024]

[0025] S113. Since F θ is a symmetric positive definite matrix, Cholesky decomposition is performed on the matrix Thus FK is written in the form of inner product, i.e.:

[0026]

[0027] wherein, Normalized gradient vector The package B is completed based on the example information i Vector expression.

[0028] As an aspect of the multi-instance multi-label learning method based on the combined error correction coding strategy of the present application, S2 comprises the following steps:

[0029] S21. Based on the idea of ECOC technology, a multi-label problem is decomposed into several single-label binary classification sub-problems;

[0030] S22. Randomly select a combined coding strategy.

[0031] As an aspect of the multi-instance multi-label learning method based on the combined error correction coding strategy of the present application, S21 comprises the following steps:

[0032] S21 comprises the following steps:

[0033] S211. ECOC technology decomposes a multi-classification problem into several binary classification problems, and completes three stages of coding, training of a classifier, and decoding to realize a multi-classification task;

[0034] S212. A multi-label problem obtained by degenerating a multi-instance multi-label problem is disassembled into several single-label binary classification sub-problems, a binary classifier under each label is trained respectively, and finally the outputs of multiple labels can be obtained.

[0035] As an aspect of the multi-instance multi-label learning method based on the combined error correction coding strategy of the present application, S22 comprises the following steps:

[0036] S221. For the task scene of the present application, several multi-label sub-problems converted from an original multi-label problem are randomly selected, an encoding matrix corresponding to the multi-label sub-problems is concatenated into a complete encoding matrix by using an OVA strategy, and then a training and decoding stage is completed;

[0037] S222. The probabilities that a sample belongs to each label are predicted, and a soft label set is obtained.

[0038] As an aspect of the multi-instance multi-label learning method based on the combined error correction coding strategy of the present application, S221 comprises the following steps:

[0039] S2211. Encoding stage: Let the original multi-instance multi-label problem Q have q labels, and the strategy considers k (k≤q) sub-labels of the same size. Let the ith sub-problem be w i The set Q k consists of sub-problems, and the number of elements in Q d is the size of the sub-problems to be studied, which is less than or equal to The strategy randomly selects d sub-problems from the set Q k without replacement, and for each sub-problem, the OVA encoding strategy is used to obtain the corresponding encoding matrix Z (i) ∈{-1, 0, +1} q×k , 1≤i≤d, and all Z (i) are connected to obtain the complete encoding matrix Z∈{-1, 0, +1} q×kd ;

[0040] S2212. Training stage: Based on each row of the encoding matrix Z, a binary classifier h j (·) is constructed and trained. When the label is encoded as +1 in the jth row, the samples related to the label in the training dataset of h j (·) are regarded as positive samples; when the label is encoded as -1 in the jth row, the samples related to the label in the training but unrelated to any +1 sample are regarded as negative samples; when the label is encoded as 0 in the jth row, only the samples related to the label in the dataset do not participate in the training of the classifier h j (·);

[0041] S2213. Decoding stage: For the test sample B new , first embed the instance information of B into the vector , and then send to the encoding matrix Z to complete the classification with all classifiers to obtain the corresponding code . Decoding means predicting the label vector from the code . This process relies on measuring the distance between different codes, and the Hamming distance is usually used to measure the distance. However, since the strategy randomly selects part of the sub-problems to construct the classifiers, the code z l in the lth column of Z may contain 0 codes unrelated to the current classifier. Therefore, the effective Hamming distance is defined between the effective code bits, and these meaningless code bits are excluded to obtain more accurate and reliable results, i.e.:

[0042]

[0043] As an aspect of the multi-instance multi-label learning method based on the combined error correction encoding strategy, in S222, if the code HD between the code corresponding to the first label and the code corresponding to the second label valid If the HD is less than a certain threshold t, the sample B is determined to be the sample to be tested new Mark this label, which is also one of the label set of the sample to be tested As for the determination of the threshold t, since the number of base classifiers involved in the training of each label in the model is different, a threshold t(l) related to the number of classifiers involved in the training of the first label is used to calculate the probability that a test sample belongs to the first label, so as to obtain a set of soft labels The t(l) is defined as:

[0044]

[0045] In fact, 1-t(l) represents the probability that the test sample belongs to the first label, and the soft label set of the sample to be tested B new

[0046]

[0047] As an aspect of the multi-instance multi-label learning method based on the combined error correction coding strategy, wherein S3 comprises the following steps:

[0048] S31. Based on the idea of the MLSVM algorithm, a support vector machine (SVM) classifier is established for each label (c=1, 2, …, q), and a soft label set is used as input data for training. Then, the T criterion is used for final class determination, that is:

[0049]

[0050] S32. If the sample to be labeled is classified as a positive class by the SVM, the sample is labeled as this class.

[0051] S33. If all SVMs classify the sample to be labeled as a negative class, the sample is labeled with the label of the class with the least negative value.

[0052] By using the above technical solution, the present application has the following advantages:

[0053] ​1. The application provides a multi-instance multi-label learning method based on a combined error correction coding strategy. The application focuses on the data imbalance scene and solves the imbalance problem based on the coding strategy. The advantage of the coding strategy in solving the imbalance problem is obvious. By converting the multi-label problem into multiple single-label binary classification problems, the coding strategy assigns a unique binary code to each class, maintains the balance of the binary classification sub-problems, and thus reduces the problems caused by the imbalanced data set. At the same time, different classifiers and parameter settings are used for each binary classification sub-problem to adapt to the characteristics of each sub-problem. For imbalanced sub-problems, a classifier more suitable for the minority class can be selected to increase the effective recognition ability of the minority class. In addition, the strategy adjusts the threshold to more flexibly control the degree of preference of the classifier for the minority class. These methods can effectively deal with the imbalance problem and improve the classification performance on the imbalanced data set.

[0054] 2. The application introduces the random idea into the error correction coding strategy, randomly selects and combines the error correction codes corresponding to the sub-problems, and trains the classification model based on the coding random division of different training data blocks, thereby enhancing the diversity of the base classifier and the error correction ability of the model, and improving the classification performance of the model. The CMIML algorithm proposed by the application randomly selects and combines the error correction codes corresponding to the multi-label sub-problems, so that the difference between the classifiers in the final model obtained by training is increased, thereby improving the diversity of the base classifier of the multi-label classification model and helping to alleviate the class imbalance problem. At the same time, the data blocks are randomly divided based on the sub-problem coding for training, which reduces the similarity between the base classifiers and enhances the error correction ability of the model.

[0055] 3. The application designs a fast and efficient feature embedding algorithm to express the multi-instance bag as a vector containing more data structure feature information. Before training the classifier by using the random coding strategy, the application designs a feature embedding representation method based on the Fisher kernel, which can extract more multi-instance bag level data structure information, thereby converting the original multi-instance multi-label learning problem into a multi-label learning problem. This efficient precursor step lays a good foundation for the execution efficiency of the integrated classifier training for the imbalance problem and the entire algorithm. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 The flowchart of the multi-instance multi-label learning method based on the combined error correction coding strategy of the application;

[0057] Figure 2 The error correction output coding schematic diagram of the application;

[0058] Figure 3 The optimization process diagram of the CMIML algorithm proposed by the application. DETAILED DESCRIPTION

[0059] The technical solution of the present invention will be specifically described below with reference to the accompanying drawings. It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0060] Figure 1 This paper presents a flowchart of a multi-instance multi-label learning method based on a combined error-correcting coding strategy, according to the present invention. To fill the research gap in multi-instance multi-label learning under imbalanced data scenarios, this invention proposes a coding multi-instance multi-label learning method (CMIML) based on a combined error-correcting coding strategy. After degenerating the MIML task into an MLL task by embedding representation packets based on instance information, the MLL problem is decomposed into several single-label binary classification sub-problems under the guidance of the ECOC ensemble idea. CMIML combines and trains base classifiers in a random manner, obtaining the probability of a sample under each label, and further predicting the specific class performance of a sample under each label. The flowchart of a multi-instance multi-label learning method based on a combined error-correcting coding strategy is shown below. Figure 1 As shown, in Figure 1 In the diagram, (a) represents the feature embedding representation: more multi-instance package-level data structure information is extracted based on the Fisher kernel, thus transforming the original multi-instance multi-label learning problem into a multi-label learning problem; (b) represents the training of the random combination encoding table: in the presence of From the set of d subproblems, d subproblems are randomly selected without replacement, and an OVA encoding strategy is applied to them to obtain the encoding matrix Z. (i) Connect all Z (i) Obtain the complete encoding matrix Z to be trained; (c) Transform the probability label into a hard label: input the soft label set to train the SVM classifier under each label.

[0061] A multi-instance, multi-label learning method based on combinatorial error correction coding strategy, such as Figure 1 As shown, it includes the following steps:

[0062] S1. A feature embedding representation method based on the Fisher kernel was designed to extract more multi-instance package-level data structure information, thereby transforming the original multi-instance multi-label learning problem into a multi-label learning problem:

[0063] FV algorithm was originally used in computer vision, which encodes the pixel blocks in an image as high-dimensional vectors using the Fisher kernel framework, and aggregates these vectors to represent the target image. Applying FV algorithm to MIML problem, the bags can be encoded and represented by the mapping The Fisher kernel and the bag embedding method based on Fisher kernel are described in detail below.

[0064] S1 includes the following specific steps:

[0065] S11. Vector representation of the bag based on the example information;

[0066] S11 includes the following specific steps:

[0067] S111. Sample is a bag composed of n i example sets. According to the classic assumption in multi-instance multi-label learning: the examples i in the bag B are independently and identically distributed, the probability density function p describing the generation of examples is used, and the bag B i can be represented by a gradient vector based on the probability generation model:

[0068]

[0069] Equation (1) intuitively represents the relationship between the parameters θ in the model p and the bag B i . In fact, the dimension of the gradient vector is only related to the number of parameters θ in the probability density function p, that is, although the number of examples contained in multiple bags is different, they can all be converted into gradient vectors

[0070] Since the Gaussian mixture model can approximate any continuous distribution with arbitrary precision, the GMM is selected as the probability density function p, and the parameters of the gth GMM are represented by θ = {α g , μ g , ∑ g}, where α g , μ g and ∑ g are Gaussian mixture weights, mean vectors and covariance matrices, respectively.

[0071] S112. Gradient vector based on the Fisher kernel function (Fisher Kernel, FK) to measure the similarity between two bags, that is:

[0072]

[0073] Among them, F θ It is the Fisher information matrix of the probability density function p, that is:

[0074]

[0075] S113. Due to F θ It is a symmetric positive definite matrix, and its Cholesky decomposition yields: Therefore, FK can be written in inner product form, that is:

[0076]

[0077] in, Normalized gradient vector Package B is then completed based on the example information. i vector representation

[0078] In fact, Dimension and gradient vector The dimensions are equal, with A nonlinear kernel learner with a core of is equivalent to a nonlinear kernel learner with a core of . The kernel learner is a linear kernel for the feature vectors. A clear advantage of this formula is that the model can learn a linear classifier very effectively. Through this efficient expression, multi-example multi-label data is transformed into multi-label data. While data imbalance still exists, this efficient preliminary step lays a solid foundation for subsequent training of the ensemble classifier to address imbalance issues and for the overall efficiency of the algorithm.

[0079] S12. Packet embedding method based on Fisher kernel.

[0080] S2. Guided by the idea of ​​Error Correcting Output Coding (ECOC), a coding strategy of random concatenation of multi-label sub-problems is proposed. This process is a key step in mitigating the problem of imbalanced data: the coding strategy transforms the multi-label problem into multiple single-label binary classification problems, assigning a unique binary code to each class, thus maintaining the balance of the binary classification sub-problems and alleviating the problems caused by imbalanced datasets. While ECOC is an effective means of solving multi-class classification problems, its concept inspires us to design algorithms for implementing multi-label classification tasks.

[0081] Figure 2 A schematic diagram of the error correction output encoding of the present invention is shown.

[0082] S2 includes the following specific steps:

[0083] S21. Based on the idea of ​​ECOC technology, a multi-label problem is decomposed into several single-label binary classification sub-problems;

[0084] S21 includes the following specific steps:

[0085] S211. The ECOC technology decomposes a multi-class problem into several binary classification sub-problems through three stages of encoding, training a classifier, and decoding, and specifically as shown in the formula: Figure 2 h j is a binary classifier, and the target class is determined by the classification results of each classifier. The encoding stage: each class is mapped to a row of code numbers, which contains the results of all binary classification problems decomposed from this class; the training stage: based on the original data, a set of binary classifiers is trained according to each column of the encoding matrix; the decoding stage: the trained binary classifiers are used to predict the class of a new sample, and a set of predicted code numbers is obtained, and the differences between the code numbers of each class are compared, and the sample is assigned to the class with the smallest difference.

[0086] S212. Common encoding matrices are binary or ternary encoding matrices. Binary encoding uses -1 and +1 to represent different classes, such as the OVA strategy. Ternary encoding uses -1, 0, and +1 to represent negative classes, irrelevant classes, and positive classes, respectively. The so-called irrelevant class means that the data corresponding to this class does not participate in the training of the corresponding base classifier, such as the OVO strategy using the ternary encoding rule.

[0087] S22. Randomly select and combine the encoding strategy.

[0088] S22 includes the following specific steps:

[0089] S221. Under the guidance of the ECOC integration idea, the multi-label problem is decomposed into several single-label binary classification sub-problems, and binary classifiers under each label are trained respectively, and finally the outputs of multiple labels can be obtained. Therefore, the present application performs multi-label learning based on the ECOC strategy, expects its performance to be consistent with the actual problem, and can effectively handle unbalanced data.

[0090] If the data imbalance problem is to be better alleviated, the similarity between the base classifiers needs to be weakened, the encoding error correction ability of the model needs to be enhanced, and randomness needs to be introduced, so that the combination of base classifiers in the integrated model is random, and the training data of the trained model is random, which can achieve this goal well. The present application randomly selects several multi-label sub-problems converted from the original multi-label problem, and uses the OVA strategy to concatenate the corresponding encoding matrix into a complete encoding matrix, which better enhances the diversity of the base classifiers and alleviates the class imbalance problem. Then the training and decoding stages are completed;

[0091] S221 includes the following specific steps:

[0092] S2211. Encoding stage: assuming that the original multi-instance multi-label problem Q has q labels, the strategy considers k(k≤q) sub-labels of the same size. Let the i-th sub-problem be iThen the set Q consisting of subproblems k The number of elements in the middle is d represents the size of the subproblem to be studied, which is less than or equal to 1. The strategy will be derived from set Q k D subproblems are randomly selected without replacement. For each subproblem, an OVA encoding strategy is used to obtain the corresponding encoding matrix Z. (i) ∈{-1, 0, +1} q×k 1≤i≤d, then connect all Z (i) If i = 1, 2, ..., d, then the complete encoding matrix Z ∈ {-1, 0, +1} to be trained can be obtained. q×kd .

[0093] The introduction of randomness increases the diversity of minority class samples. The strategy generates new synthetic samples by randomly transforming or augmenting minority class samples, thus providing more training data. This helps the model better learn the features of the minority class, improving the class distribution of imbalanced datasets. Simultaneously, the random combination of error-correcting codes increases the differences between classifiers in the final model, thereby enhancing the diversity of base classifiers in multi-label classification models and also helping to alleviate the class imbalance problem.

[0094] S2212. Training Phase: Construct a binary classifier h based on each row of the encoding matrix Z. j (·) and complete the training. When the label is encoded with +1 in the j-th row, it will be in h j In the training dataset (·), samples related to the label are considered positive samples; when the label is encoded with -1 in the j-th row, samples related to the label but unrelated to any +1 samples during training are considered negative samples; when the label is encoded with 0 in the j-th row, samples in the dataset only related to the label do not participate in classifier h. j Training (·).

[0095] S2213. Decoding Phase: For the sample B to be tested new First, its example information Embedded into vector In the middle, then The data is sent to the encoding matrix Z, where it is classified using all classifiers to obtain the corresponding code. Decoding, in this context, refers to decoding from the code. The process involves predicting label vectors within Z. This relies on measuring the distance between different codes, typically using Hamming distance. However, because the policy randomly selects some sub-problems to construct the classifier, the code z in the l-th column of Z... lThe codes in the model can contain 0 codes that are not related to the current classifier. Therefore, these meaningless positions should be removed when measuring the distance to obtain a more accurate and reliable result, thereby defining the effective Hamming distance between the effective code bits, i.e.:

[0096]

[0097] S222. Predict the probability of the sample belonging to each label to obtain a soft label set.

[0098] In S222, if the code of the sample B valid is smaller than a threshold t, the sample B new is marked with the label, which is also one of the elements in the label set of the sample B . As for the determination of the threshold t, since the number of base classifiers involved in the training of each label in the model is different, a threshold t(l) related to the number of classifiers involved in the training of the lth label is used to calculate the probability of a test sample belonging to the lth label, thereby obtaining a set of soft labels t(l) is defined as:

[0099]

[0100] In fact, 1-t(l) represents the probability of the test sample belonging to the lth label. The soft label set of the sample B new

[0101]

[0102] S3. Construct and train a support vector machine model to obtain a hard label representing the specific category performance of the predicted sample under each label based on the T criterion:

[0103] S3 includes the following specific steps:

[0104] S31. Based on the idea of the MLSVM algorithm, a support vector machine SVM classifier is established for each label (c = 1, 2,..., q) according to formula (7), and the soft label set is used as input data for training, and then the T criterion is used for final category determination;

[0105]

[0106] S32. If the sample to be labeled is classified as a positive class by the SVM, the sample is labeled as this class.

[0107] S33. If all SVMs classify the sample to be labeled as a negative class, the sample is labeled with the class label with the least negative value.

[0108] ​Thus, the transformation of the soft label set is completed, and the multi-label learning task is ended.

[0109] Figure 3 The specific process of the optimization of the CMIML algorithm proposed in the application is shown, and the above process is specifically as shown in Figure 3

[0110] Finally, it should be pointed out that, although the present application has been described with reference to the current specific embodiments, those skilled in the art should realize that the above embodiments are only used to illustrate the present application, and are not used as a limitation on the present application, and various equivalent changes or replacements can be made without departing from the concept of the present application, therefore, any changes or modifications of the above embodiments within the scope of the spirit of the present application will fall within the scope of the claims of the present application.​

Claims

1. A multi-instance multi-label learning method based on a combined error correction coding strategy, characterized in that, The method comprises the following steps: S1. A feature embedding representation method of multi-instance bag level data structure information is designed based on a Fisher kernel, and an original multi-instance multi-label learning problem is converted into a multi-label learning problem; S2. An encoding strategy of randomly connecting multi-label sub-problems is proposed under the guidance of an error correction output coding idea; The S2 comprises the following steps: S21. Based on the idea of the ECOC technology, a multi-label problem is decomposed into several single-label binary classification sub-problems; S22. A random selection and combination encoding strategy is selected; The S22 comprises the following steps: S221. For a task scenario, several multi-label sub-problems converted from an original multi-label problem are randomly selected, an OVA strategy is used to connect the corresponding encoding matrixes into a complete encoding matrix, and then a training and decoding stage is completed; The S221 comprises the following steps: S2211. Encoding stage: Let the original multi-instance multi-label problem Q have q labels, and the strategy consider k (k ≤ q) sub-labels of the same size, denoted as the ith sub-problem w i The set Q k consists of sub-problems, and the number of elements in the set is d is the size of the sub-problems to be studied, which is less than or equal to The strategy randomly selects d sub-problems from the set Q k without replacement, and for each sub-problem, the OVA encoding strategy is used to obtain the corresponding encoding matrix Z (i) ∈{-1,0,+1} q×k , 1 ≤ i ≤ d, and all Z (i) are connected, i.e., i = 1, 2,..., d, to obtain the complete encoding matrix Z ∈{-1,0,+1} q×kd to be trained; S2212. Training phase: construct binary classifier h based on each row of encoding matrix Z j (·) and complete training, when the label is encoded with +1 in the jth row, the sample related to this label in the training data set is regarded as a positive sample; when the label is encoded with -1 in the jth row, the sample related to this label in the training but unrelated to any +1 sample is regarded as a negative sample; when the label is encoded with 0 in the jth row, only the sample related to this label in the data set does not participate in the training of the classifier h j (·). j (·). S2213. Decoding phase: for the sample B under test new , firstly, its example information is embedded into a vector , then is sent to the encoding matrix Z, and the classification is completed with all classifiers to obtain the corresponding code So-called decoding is to predict the label vector from the code , and the process relies on measuring the distance between different codes, using Hamming distance to measure the distance. However, since the strategy randomly selects part of the sub-problems to construct the classifier, the code z l in the lth column of Z may contain 0 codes irrelevant to the current classifier, therefore, the effective Hamming distance is defined between the effective code bits, and these meaningless code bits are excluded to obtain more accurate and reliable results, that is: S222. Probabilities that a sample belongs to each label are predicted, and a soft label set is obtained; S3. A support vector machine model is constructed and trained, and a hard label representing a specific class performance of a predicted sample under each label is obtained based on a T criterion. 2.The multi-instance multi-label learning method based on a combined error correction coding strategy according to claim 1, wherein, The S1 comprises the following steps: S11. A vector expression of a bag based on example information is obtained; S12. A bag embedding method based on a Fisher kernel is obtained. 3.The multi-instance multi-label learning method based on a combined error correction coding strategy according to claim 2, wherein, The S11 comprises the following steps: S111. Sample is a bag of n i instances, according to the classic assumption in multi-instance multi-label learning: instances i in a bag B (j = 1, 2,..., n i ) are independently and identically distributed, the probability density function p describing the generation of instances is parameterized by θ, and a bag B i can be represented by a gradient vector based on the probabilistic generative model: S112. Based on a gradient vector, a Fisher kernel function is used to measure the similarity between two samples, namely: where F θ is the Fisher information matrix of the probability density function p, i.e.: S113. Since F θ is a symmetric positive definite matrix, Cholesky decomposition is applied to it Thus FKis written in the inner product form, i.e.: wherein Normalized gradient vector The package B is completed based on the example information i of the vector representation. 4.The multi-instance multi-label learning method based on a combined error correction coding strategy according to claim 1, wherein, The S21 comprises the following steps: S211. The ECOC technology decomposes a multi-classification problem into several binary classification problems, and completes three stages of encoding, training a classifier and decoding to realize a multi-classification task; S212. A multi-label problem obtained from a degenerate multi-instance multi-label problem is decomposed into several single-label binary classification sub-problems, a binary classifier under each label is trained, and finally the outputs of multiple labels can be obtained.

5. The multi-instance multi-label learning method based on a combined error correction coding strategy according to claim 1, characterized in that, If the code in S222 HD between the code corresponding to the first label and the code corresponding to the second label valid If the distance is less than a certain threshold t, then the sample B is determined to be the second label new Mark this label, which is also one of the label set of the sample to be tested, as the second label As the number of base classifiers involved in the training of each label in the model is different, a threshold t(l) related to the number of classifiers involved in the training of the first label is used to calculate the probability that a test sample belongs to the first label, so as to obtain a set of soft labels Definition of t(l): In fact, 1-t(l) represents the probability that the test sample belongs to the l-th label, and the soft label set of the test sample B new ​ 6.The multi-instance multi-label learning method based on a combined error correction coding strategy according to claim 1, wherein, The S3 comprises the following steps: S31. Based on the idea of the MLSVM algorithm, a support vector machine SVM classifier is established for each label (c=1, 2, …, q), a soft label set is used as input data for training, and then a T criterion is used for final class determination, namely: S32. If a sample to be labeled is classified as a positive class by the SVM, the sample is labeled as the class; S33. If the sample to be labeled is classified as a negative class by all SVMs, the sample is labeled with a class label with the least negative value.

Images