Method, device, equipment and medium for selecting partial multi-label features

Abstract

The present invention provides a method, device, equipment and medium for selecting features with too many labels, and relates to the technical field of data preprocessing. The method comprises S1, obtaining an original training data set with too many labels, and initializing a label confidence matrix and constructing a distribution area of ​​labels based on the original training data set. S2, updating the label confidence matrix based on the distribution of neighbors, disambiguating the distribution area of ​​untrustworthy labels, and iteratively updating the distribution area of ​​uncertain labels until the maximum number of iterations is reached, and obtaining the disambiguated labels. S3, based on the disambiguated labels, combining granular computing and graph theory, aggregate the labels from fine granularity to coarse granularity, and construct virtual coarse granular labels. S4, based on the virtual coarse granular labels, combined with multiple features obtained from sample data, construct a neighborhood rough set model suitable for over-labeled data, and use a forward greedy search algorithm to select the most important features.

Description

Technical Field

[0001] The present invention relates to the technical field of data preprocessing, and in particular to a method, device, equipment and medium for selecting multi-label features. Background Art

[0002] Partial multi-label learning (PML) is a learning framework for processing limited supervised information. The data used has a unique characteristic: each instance is assigned a set of candidate labels, but only some of the labels are true. The acquisition of such data has become more convenient and cost-effective, which has promoted the implementation of PML in multiple practical applications. However, existing data processing methods have some shortcomings when processing PML data.

[0003] In terms of feature selection, traditional feature selection methods often fail to fully adapt to the characteristics of PML data. They usually ignore the uncertainty and incompleteness of labels, resulting in the inability to accurately identify the feature subsets that are most relevant to the true label. In addition, these methods may not be able to effectively handle the interdependencies between features when dealing with high-dimensional data, and thus fail to fully utilize these relationships to improve the performance of the model. Computationally, traditional methods can be very expensive, especially when all possible feature subsets need to be evaluated, which is particularly problematic in large-scale PML datasets.

[0004] Existing technologies also face challenges in label disambiguation. They often lack accuracy in estimating label confidence, especially in the presence of a large number of false positive labels. This may cause the model to learn incorrect labels, thereby limiting its generalization ability. In addition, existing label disambiguation methods often fail to provide sufficient transparency and explainability when dealing with uncertainties in iterative or stage processes, making the label disambiguation process less transparent and difficult to provide credible decision support for end users. Finally, when dealing with incomplete disambiguation problems, these methods often fail to effectively reduce the interference caused by uncertain labels, resulting in degraded model performance. Therefore, developing new methods that can overcome these shortcomings is crucial to improving the accuracy and generalization ability of PML models. Summary of the invention

[0005] The present invention provides a method, device, equipment and medium for selecting multi-label features to improve at least one of the above-mentioned technical problems.

[0006] In a first aspect, the present invention provides a method for selecting multi-label features, which comprises steps S1 to S4.

[0007] S1. Obtain an original training data set with multiple labels, and initialize the label confidence matrix based on the original training data set, and construct the distribution areas of credible labels, untrustworthy labels, and uncertain labels.

[0008] S2. Update the label confidence matrix according to the distribution of the nearest neighbors, disambiguate the distribution area of ​​the untrustworthy labels, and iteratively update the distribution area of ​​the uncertain labels until the maximum number of iterations is reached to obtain the disambiguated labels.

[0009] S3. Based on the disambiguated labels, granular computing and graph theory are combined to aggregate the labels from fine-grained to coarse-grained, and construct virtual coarse-grained labels.

[0010] S4. Based on the virtual coarse-grained labels and in combination with a plurality of features obtained from the sample data, a neighborhood rough set model suitable for multi-labeled data is constructed, and a forward greedy search algorithm is used to select the most important feature.

[0011] In an optional implementation, step S1 specifically includes steps S11 to S16.

[0012] S11. Obtain the original training data set with multiple labels.

[0013] S12: Based on the original training data set, construct a label confidence matrix evenly distributed on the candidate label set. .

[0014] ;

[0015] ;

[0016] In the formula, Indicates Sample data No. Tags The uniformly distributed label confidence, is the number of sample data, is the number of labels, For sample data The candidate label set.

[0017] S13. Use the bucket KD-Tree algorithm to effectively search sample data of A close neighbor.

[0018] ;

[0019] In the formula, Represents sample data of Neighbors, Represents the neighbors of the sample data, Represents sample data No. Neighbors, represents the sample space.

[0020] S14. According to Nearest neighbors, get the probability matrix .

[0021] ;

[0022] ;

[0023] In the formula, For the Sample data No. Tags The probability of Represents sample data of Neighbors, represents the positive example set on the original noise label, Indicates the number of elements in a set, is the smoothing parameter to avoid zero probability cases, Indicates the number of classes corresponding to each label.

[0024] S15. Initialize a label confidence matrix according to the label confidence matrix and the probability matrix.

[0025] S16, according to the initialized label confidence matrix, based on the " The "criterion" divides the candidate labels into three non-overlapping regions to obtain the disambiguation label confidence matrix. The three non-overlapping regions are the credible label region, the uncredible label region, and the uncertain label region.

[0026] Preferably, the initialization model of the label confidence matrix is:

[0027] ;

[0028] In the formula, is an element in the matrix, indicating the Sample data No. Tags is the label confidence of the true label, is the number of sample data, is the number of labels, represents the label confidence matrix uniformly distributed on the candidate label set, is a probability matrix constructed from neighborhood information, is a balance parameter.

[0029] Preferably, the disambiguation label confidence matrix for:

[0030] ;

[0031] ;

[0032] In the formula, is the element in the current disambiguation label matrix, represents the mean of each sample in the current disambiguation label matrix, Represents the standard deviation of each sample in the current disambiguation label matrix.

[0033] In an optional implementation, step S2 specifically includes steps S21 to S24.

[0034] S21. Iterations, sample data of The neighbors are updated to . in, For sample data No. A close neighbor.

[0035] S22, the probability matrix can be updated as :

[0036] ;

[0037] ;

[0038] In the formula, For the The iteration Sample data No. Tags The probability of For the Sample data for iterations of Neighbors, Indicates passing The positive example set of disambiguation labels generated after iterations, Indicates the number of elements in a set, is the smoothing parameter to avoid zero probability cases, Indicates the number of classes corresponding to each label;

[0039] S23, in Propagate the current label confidence matrix based on , the label confidence matrix is ​​updated as : . is a balance parameter.

[0040] S24, according to Update the distribution areas of trusted labels, untrusted labels, and uncertain labels to obtain the The current disambiguation label confidence matrix of the iteration is updated as :

[0041] ;

[0042] ;

[0043] In the formula, For the The elements in the current disambiguation label confidence matrix of the iteration, represents the mean of each sample in the current disambiguation label matrix, Represents the standard deviation of each sample in the current disambiguation label matrix.

[0044] In an optional implementation, step S3 specifically includes steps S31 to S34.

[0045] S31. Take the label according to the disambiguated label and As vertices, create a completely undirected graph .in, , is a set of vertices, is the edge set. Define the label and The weight of the connecting edge is the label dissimilarity

[0046] ;

[0047] In the formula, It is the disambiguation label generated after the credible label induction phase is completed. The positive example set of It is the label after completing the iteration in the trusted label induction phase. The set of positive examples of ;

[0048] S32, based on the completely undirected graph, first use the famous Prim algorithm to construct a minimum spanning tree , and then remove some edges to divide the minimum spanning tree into multiple subtrees. The threshold for removing edges for:

[0049] ;

[0050] In the formula, is the number of labels.

[0051] S33, each subtree is regarded as a coarse-grained label aggregated from multiple fine-grained labels, and the values ​​of the fine-grained labels are aggregated into a new coarse-grained label according to the co-occurrence frequency of the labels in the coarse-grained label to obtain a virtual coarse-grained label .

[0052] S34, virtual coarse-grained label Down The values ​​are:

[0053] ;

[0054] In the formula, For sample data No. Tags, is a coarse-grained label consisting of one or more labels. express The number of tags in Representation Tags Download Sample The value of ξ is a parameter used to control whether the instance belongs to a virtual coarse-grained label. It means that if the condition is met, it returns 1, otherwise it returns 0.

[0055] In an optional implementation, step S4 specifically includes steps S41 to S46.

[0056] S41: Based on the virtual coarse-grained label and multiple features obtained from the sample data, a local multi-label decision system is constructed. , where: For the sample data set, is the feature set, is a set of virtual coarse-grained labels.

[0057] S42, based on , according to the bucket KD-Tree algorithm, search all sample data In feature subset on neighbors, get the neighborhood.

[0058] ;

[0059] In the formula, Represents sample data In feature subset on Neighbors, Represents the neighbors of the sample data, Represents sample data No. Neighbors, is a subset of the feature set.

[0060] S43. Definition is a virtual coarse-grained label The decision is The sample set is the coarse-grained label set About Feature Subsets The upper and lower approximations are:

[0061] ;

[0062] In the formula, is a set of virtual coarse-grained labels about the feature subset The lower approximation of is the number of virtual coarse-grained labels, is the sequence number of the virtual coarse-grained label, is a virtual coarse-grained label about a feature subset The lower approximation of is a virtual coarse-grained label The number of sample sets for decision making, Indicates the first samples, Indicates Sample data, Represents sample data In feature subset on Neighbors, is a set of virtual coarse-grained labels about the feature subset The upper approximation of is a virtual coarse-grained label about a feature subset The upper approximation of Represents the empty set.

[0063] S44, defining a virtual coarse-grained label according to the upper and lower approximation For feature subsets The degree of dependence is:

[0064] ;

[0065] In the formula, is a virtual coarse-grained label For feature subsets The degree of dependence is a virtual coarse-grained label About Feature Subsets The lower approximation of is a sample data set.

[0066] S45. Based on the degree of dependency, define features Relative to The importance of is:

[0067] ;

[0068] In the formula, Indicates importance, Represents a feature set The features that do not belong to feature subset A, represents a virtual coarse-grained label, Indicates the degree of dependence.

[0069] S46, based on the importance, using a forward greedy search algorithm to select the most important feature .

[0070] ;

[0071] In the formula, Indicates that multiple importance values ​​are greater than 0 of and is the index of the feature.

[0072] In a second aspect, the present invention provides a device for selecting multi-label features, which includes a region division module, a disambiguation module, an aggregation module and a feature selection module.

[0073] The region partitioning module is used to obtain the original training data set with multiple labels, initialize the label confidence matrix based on the original training data set, and construct the distribution areas of credible labels, untrustworthy labels and uncertain labels.

[0074] The disambiguation module is used to update the label confidence matrix according to the distribution of the nearest neighbors, disambiguate the distribution area of ​​the untrustworthy labels, and iteratively update the distribution area of ​​the uncertain labels until the maximum number of iterations is reached to obtain the disambiguated labels.

[0075] The aggregation module is used to aggregate the labels from fine-grained to coarse-grained based on the disambiguated labels, combining granular computing with graph theory, and constructing virtual coarse-grained labels.

[0076] The feature selection module is used to construct a neighborhood rough set model suitable for multi-labeled data based on the virtual coarse-grained labels and multiple features obtained from sample data, and adopt a forward greedy search algorithm to select the most important feature.

[0077] In a third aspect, the present invention provides a partial multi-label feature selection device, comprising a processor, a memory, and a computer program stored in the memory. The computer program can be executed by the processor to implement a partial multi-label feature selection method as described in any paragraph of the first aspect.

[0078] In a fourth aspect, the present invention provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute a multi-label feature selection method as described in any paragraph of the first aspect.

[0079] By adopting the above technical solution, the present invention can achieve the following technical effects:

[0080] In summary, the partial multi-label feature selection method based on credible label induction first estimates the label confidence of the candidate label of each PML training sample through iterative label propagation, and proposes a criterion to distinguish credible labels, uncredible labels and uncertain labels, and disambiguate the labels. At the same time, in order to reduce the interference of uncertain labels, granular computing and graph theory are used to aggregate fine-grained labels into coarse-grained labels. Finally, the coarse-grained labels are used to construct a neighborhood rough set model for feature selection.

[0081] The method for partial multi-label feature selection based on trusted label induction introduces a trusted label induction mechanism to make the label disambiguation process more transparent, while aggregating coarse-grained labels to reduce the incomplete disambiguation problem caused by uncertain labels, and integrating it into the construction of a neighborhood rough set model to establish an efficient partial multi-label feature selection method. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the specific implementation methods of the present invention. It should be understood that the following drawings only show certain specific implementation methods of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0083] Figure 1 It is a flowchart of a multi-label feature selection method. DETAILED DESCRIPTION

[0084] The technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0085] Example 1, please refer to Figure 1 The first embodiment of the present invention provides a method for selecting multi-label features based on trusted label induction, which can be executed by a multi-label feature selection device based on trusted label induction (hereinafter referred to as: feature selection device), and in particular, by one or more processors in the feature selection device to implement steps S1 to S4.

[0086] S1, obtain the original training data set with multiple labels, and initialize the label confidence matrix based on the original training data set, and construct the distribution areas of credible labels, untrustworthy labels and uncertain labels.

[0087] Preferably, the label confidence matrix is ​​initialized according to the label distribution of the original training data set to distinguish between credible labels, untrustworthy labels and uncertain labels. For a multi-label data set, is a label set with noisy labels. Indicates Sample data, is the number of sample data, Represents sample data No. Tags, is the number of labels.

[0088] When constructing the label confidence matrix, we must consider both the label information of the instance itself and its The label information of the neighbors. Therefore, the initialization model of the label confidence matrix is:

[0089] ;

[0090] In the formula, is an element in the matrix, indicating the Sample data No. Tags is the label confidence of the true label, is the number of sample data, is the number of labels, represents the label confidence matrix uniformly distributed on the candidate label set, is a probability matrix constructed from neighborhood information, is a balance parameter. In this embodiment, The default value is 0.5.

[0091] for :

[0092] ;

[0093] ;

[0094] In the formula, Indicates Sample data No. Tags The uniformly distributed label confidence, is the number of sample data, is the number of labels, For sample data The candidate label set.

[0095] Obviously, satisfy:

[0096] ;

[0097] ;

[0098] ;

[0099] In the formula, Representation Tags Sample data The value of .

[0100] Considering the amount of computation, the bucket KD-Tree algorithm is used to effectively search for sample data. of Neighbors . .

[0101] ;

[0102] In the formula, Represents sample data of Neighbors, Represents the neighbors of the sample data, Represents sample data No. Neighbors, Represents the sample space (i.e. the set of all sample data).

[0103] Then, the probability matrix The calculation formula is:

[0104] ;

[0105] ;

[0106] In the formula, For the Sample data No. Tags The probability of Represents sample data of Neighbors, represents the positive example set on the original noise label, Indicates the number of elements in a set, is the smoothing parameter to avoid zero probability cases, Indicates the number of classes corresponding to each label. The default is .

[0107] ;

[0108] In the formula, Indicates Sample data, Representation Tags Sample data The value of .

[0109] according to" "Criteria", assuming that the label confidence follows a normal distribution, the candidate labels are divided into three non-overlapping regions, namely the credible label region, the untrustworthy label region and the uncertain label region. Preferably, the credible label represents a true positive label that can be directly used for subsequent feature subset selection. The untrustworthy label represents a false positive label that needs to be disambiguated. The uncertain label represents a label that is uncertain whether it is a false positive.

[0110] Specifically, the 3σ criterion (three sigma criterion) is a method used in statistics to identify outliers in a data set. It is based on the property of a normal distribution, that is, approximately 99.73% of data values ​​are within three standard deviations of the mean. Therefore, any data point outside this range is considered an outlier. In this embodiment, the use of the 3σ criterion helps to reduce the number of uncertain labels that may be difficult to classify due to noise or other problems in the data. By identifying and processing these uncertain labels, the accuracy and robustness of the model can be improved.

[0111] Set the two thresholds for dividing the area respectively and ,in, Indicates that each sample is in the current disambiguation label matrix (excluding 0 and 1), Indicates that each sample is in the current disambiguation label matrix The standard deviation in (excluding 0 and 1). Since 0 and 1 represent the determined credible labels or untrustworthy labels, respectively, removing them can prevent interference with the subsequent determination of credible labels and untrustworthy labels, which helps to narrow the uncertainty area.

[0112] Then, the current disambiguation label confidence matrix for:

[0113] ;

[0114] In the formula, is the element in the current disambiguation label matrix, Represents the mean of each sample in the current disambiguation label matrix (excluding 0 and 1), Represents the standard deviation of each sample in the current disambiguation label matrix (excluding 0 and 1).

[0115] when When, label Located in the sample The uncertain area, where the label cannot be determined whether it is a sample The trusted labels need further iteration.

[0116] S2, according to the distribution of the nearest neighbors, updates the label confidence matrix, disambiguates the distribution area of ​​the untrustworthy label, and iteratively updates the distribution area of ​​the uncertain label until the maximum number of iterations is reached to obtain the disambiguated label.

[0117] Specifically, for the Iterations, sample data of The neighbors are updated as:

[0118] ;

[0119] In the formula, For sample data No. A close neighbor.

[0120] Preferably, , and are fixed at 10 and 5 respectively.

[0121] Therefore, the probability matrix can be updated as :

[0122] ;

[0123] ;

[0124] In the formula, For the The iteration Sample data No. Tags The probability of For the Sample data for iterations of Neighbors, Indicates passing The positive example set of disambiguation labels generated after iterations, Indicates the number of elements in a set, is the smoothing parameter to avoid zero probability cases, Indicates the number of classes corresponding to each label.

[0125] By Propagate the current label confidence based on , the label confidence matrix is ​​updated as:

[0126] ;

[0127] according to" Criteria", assuming that the label confidence follows a normal distribution, according to The candidate labels are divided into three disjoint regions, namely the credible label region, the uncredible label region and the uncertain label region.

[0128] No. The current disambiguation label confidence matrix of the iteration is updated as :

[0129] ;

[0130] ;

[0131] In the formula, For the The elements in the current disambiguation label confidence matrix of the iteration, represents the mean of each sample in the current disambiguation label matrix (excluding 0 and 1), Represents the standard deviation of each sample in the current disambiguation label matrix (excluding 0 and 1).

[0132] According to the threshold in the above formula, the proportions of credible labels, uncredible labels, and uncertain labels in the data are 15.87%, 68.26%, and 15.87%, respectively.

[0133] if ,Label is an instance The proportion of reliable labels is 15.87%, and the label The credibility risk of will be lower than 84.13% of other labels. In this case, for instance Keep Tags , and no disambiguation of labels is required.

[0134] On the contrary, if , then the label The risk of an untrustworthy label is lower than 84.13% of other labels. For example, tags Label disambiguation is required, that is, the label confidence matrix needs to be Set to 0.

[0135] when When, label Located in the sample The uncertain area, where the label cannot be determined whether it is a sample The credible labels need to be further iterated until the iteration reaches the preset maximum number of iterations, and the iterative process is terminated.

[0136] S3, based on the disambiguated labels, combines granular computing and graph theory to aggregate the labels from fine-grained to coarse-grained, and constructs virtual coarse-grained labels.

[0137] Although trusted label induction is used to purify the noisy label space in the first stage, this process cannot completely eliminate the ambiguity caused by noisy labels. Therefore, it is necessary to add a robust algorithm that can ignore the noise of difficult-to-identify labels. Granular computing is an important method to simulate human cognition to solve complex problems. Analyzing and solving problems at a coarse granularity can effectively reduce the impact of noise and improve the robustness and effectiveness of the system.

[0138] First, take the label and As vertices, create a completely undirected graph .in, , is a set of vertices, is a set of edges. This graph reflects the correlation between labels after the first stage of disambiguation.

[0139] Therefore, by the label and The edge weight of the connection is defined as the label dissimilarity :

[0140] ;

[0141] In the formula, It is the disambiguation label generated after the credible label induction phase is completed. The positive example set of It is the label after completing the iteration in the trusted label induction phase. The positive example set of .

[0142] , Smaller values ​​indicate greater label similarity.

[0143] Then, the well-known Prim’s algorithm is used to construct a minimum spanning tree (MST), which connects all vertices while minimizing the sum of edge weights to aggregate fine-grained labels into coarse-grained labels.

[0144] The key to Prim's algorithm is to select the edge with the smallest weight between the minimum spanning tree and the remaining vertex set in each iteration, so that the optimal edge is added to the minimum spanning tree at each step. The time complexity of the algorithm depends on the data structure used to select the minimum edge. When using a priority queue (such as a binary heap), the time complexity can reach O(E logV), where E is the number of edges and V is the number of vertices.

[0145] The specific steps to construct a minimum spanning tree (MST) are: 1. Select a starting point: Select any vertex from the graph as the starting point and regard it as part of the minimum spanning tree. 2. Find the minimum edge: Find the edge with the smallest weight between the selected vertices of the minimum spanning tree and the unselected vertices. 3. Expand the tree: Add the found minimum edge and the unselected vertices connected to it to the minimum spanning tree. 4. Repeat the search: Continue to search for the edge with the smallest weight between the vertices of the minimum spanning tree and the remaining unselected vertices. 5. Check the completion condition: Check whether all vertices are included in the minimum spanning tree. 6. End or continue: If all vertices are included, the algorithm ends. Otherwise, return to step 2 to continue. 7. Output MST: When all vertices are included, output the vertices and edges that constitute the minimum spanning tree.

[0146] Then, remove some edges The minimum spanning tree (MST) is divided into multiple subtrees to ensure that the fine-grained label division satisfies the requirement that labels within the same granularity are closely related and labels between different granularities are weakly related.

[0147] The threshold for removing edges is:

[0148] ;

[0149] For edge weights ,if , then the label and The relationship between is strongly uncorrelated, that is, the labels are weakly correlated. In this case, we will remove the connection and On the contrary, if , then the label and The relationship between is strongly correlated, and the connection in MST is retained and edge.

[0150] Each subtree is considered as a coarse-grained label aggregated from multiple fine-grained labels. .in, is a coarse-grained label consisting of one or more labels. is the sequence number of the virtual coarse-grained label, is the number of virtual coarse-grained labels.

[0151] Finally, according to the coarse-grained label The co-occurrence frequency of the inner tags, aggregate the values ​​of the fine-grained tags into a new coarse-grained tag, and obtain the virtual coarse-grained tag .

[0152] Therefore, the virtual coarse-grained label Down The values ​​are:

[0153] ;

[0154] In the formula, For sample data No. Tags, is a coarse-grained label consisting of one or more labels. express The number of tags in Representation Tags Download Sample The value of ξ is a parameter used to control whether the instance belongs to a virtual coarse-grained label. It means that if the condition is met, it returns 1, otherwise it returns 0.

[0155] Preferably, the value of ξ is preset to 0.7.

[0156] In this example, granular computing and graph theory are combined to aggregate labels from fine-grained to coarse-grained, and construct virtual coarse-grained labels. It is impossible to completely determine reliable labels and unreliable labels. Granular computing is an important method to simulate human cognition to solve complex problems. Analyzing and solving problems at a coarse-grained level can effectively reduce the impact of noise and improve the robustness and effectiveness of the system. Therefore, aggregating fine-grained labels into coarse-grained labels can reduce the negative impact of incomplete disambiguation, and integrate coarse-grained labels into the construction of the neighborhood rough set model (i.e., NRS model) to achieve feature selection.

[0157] S4, based on the virtual coarse-grained labels and in combination with a plurality of features obtained from the sample data, a neighborhood rough set model suitable for multi-labeled data is constructed, and a forward greedy search algorithm is used to select the most important feature.

[0158] The feature set is the set of all features associated with the sample set. These features are used to describe various attributes or characteristics of the sample data. They are very important components in machine learning tasks and are used to train models to complete specific tasks, such as classification, regression, etc. The following methods can be used to extract features that are useful for model training and prediction from sample data. 1. Manual feature extraction. 2. Automated feature extraction. 3. Deep learning feature extraction. 4. Data preprocessing and transformation. 5. Feature construction. 6. Use feature extraction algorithms. 7. Tree-based feature selection. 8. L1 regularization: Models that use penalty terms, such as Lasso regression, can reduce the coefficients of features by regularizing the penalty terms, thereby achieving feature selection.

[0159] The embodiment of the present invention invents a new local multi-label decision system based on the generated virtual coarse-grained labels. represents a local multi-label decision system, where is a sample data set, is the feature set, is a set of virtual coarse-grained labels. is the number of features.

[0160] According to the bucket KD-Tree algorithm, search all sample data In feature subset on Nearest neighbors, get the neighborhood : is a subset of the feature set.

[0161] ;

[0162] In the formula, Represents sample data In feature subset on Neighbors, Represents the neighbors of the sample data, Represents sample data No. A close neighbor.

[0163] In this embodiment, the Euclidean distance between sample data and features is used to search for the nearest neighbor. , is a distance measure calculated by Euclidean distance on a subset of features.

[0164] definition is a virtual coarse-grained label The decision is The sample set is the coarse-grained label set About Feature Subsets The upper and lower approximations are:

[0165] ;

[0166] In the formula, is a set of virtual coarse-grained labels about the feature subset The lower approximation of is the number of virtual coarse-grained labels, is the sequence number of the virtual coarse-grained label, is a virtual coarse-grained label about a feature subset The lower approximation of is a virtual coarse-grained label The number of sample sets for decision making, Indicates the first samples, Indicates Sample data, Represents sample data In feature subset on Neighbors, is a set of virtual coarse-grained labels about the feature subset The upper approximation of is a virtual coarse-grained label about a feature subset The upper approximation of Represents the empty set.

[0167] in, To estimate the subset The ability to represent the feature subset The neighborhood of the previous sample data always belongs to the same class.

[0168] According to the upper and lower approximations, define virtual coarse-grained labels For feature subsets The degree of dependence is:

[0169] ;

[0170] In the formula, is a virtual coarse-grained label For feature subsets The degree of dependence is a virtual coarse-grained label About Feature Subsets The lower approximation of is a sample data set.

[0171] Dependence is the feature subset Approximating Virtual Coarse-Grained Labels ability. , The larger the value, the distinguish The stronger the ability, Represents a virtual coarse-grained label Completely dependent on feature subset .

[0172] Based on the degree of dependency, define the features Relative to The importance of is:

[0173] ;

[0174] In the formula, Indicates importance, Represents a feature set The features that do not belong to feature subset A, represents a virtual coarse-grained label, Indicates the degree of dependence.

[0175] Based on the importance, a forward greedy search algorithm is used to select the most important feature .

[0176] ;

[0177] In the formula, Indicates that multiple importance values ​​are greater than 0 of and is the index of the feature.

[0178] In this embodiment, coarse-grained labels are aggregated and integrated into the construction of the NRS model, and a forward greedy search algorithm is used to select the most important features, and the features are reduced in dimension to remove irrelevant, noisy and redundant features.

[0179] In summary, the partial multi-label feature selection method based on credible label induction first estimates the label confidence of the candidate label of each PML training sample through iterative label propagation, and proposes a criterion to distinguish credible labels, uncredible labels and uncertain labels, and disambiguate the labels. At the same time, in order to reduce the interference of uncertain labels, granular computing and graph theory are used to aggregate fine-grained labels into coarse-grained labels. Finally, the coarse-grained labels are used to construct a neighborhood rough set model for feature selection.

[0180] The method for partial multi-label feature selection based on trusted label induction introduces a trusted label induction mechanism to make the label disambiguation process more transparent, while aggregating coarse-grained labels to reduce the incomplete disambiguation problem caused by uncertain labels, and integrating it into the construction of a neighborhood rough set model to establish an efficient partial multi-label feature selection method.

[0181] The embodiment of the present invention can be applied to the data preprocessing stage of partial multi-label learning (PML). It not only disambiguates the labels, but also performs feature selection to provide more accurate training data for partial multi-label learning (PML) training classification models. It helps: 1. Improve the accuracy and robustness of the model: Through label disambiguation, the impact of noise labels on model training can be reduced, thereby improving the accuracy of the model. At the same time, feature selection helps to remove irrelevant or redundant features, reduce the dimension of the feature space, and improve the robustness of the model. 2. Enhance the generalization ability of the model: Label disambiguation helps to identify and utilize real label information, while feature selection ensures that the model only focuses on the most useful features. The combination of the two can improve the generalization ability of the model for new and unseen data. 3. Reduce the risk of overfitting: Feature selection reduces the risk of model overfitting by reducing the dimension of the feature space, especially when the number of features is more than the training samples. 4. Improve learning efficiency: Feature selection can reduce the amount of calculation, because the model only needs to be trained on fewer features, which can speed up training and improve learning efficiency. 5. Improve the label ambiguity problem: The label disambiguation strategy can more accurately identify the true label and reduce the interference of false positive labels, thereby improving the problems caused by label ambiguity. 6. Improve the discriminability of representation: Combining contrastive learning and label disambiguation, it is possible to learn representations with greater discriminability, which is crucial to the classification performance of the model. 7. Reduce dependence on prior knowledge: Multi-label classification and label disambiguation can be processed simultaneously without the need for prior knowledge, which reduces the dependence on prior knowledge of label correlation. 8. Improve the ability to process large data sets: In modern society, a large amount of data is generated. Label disambiguation and feature selection can help models process these large data sets more effectively, especially in fields such as biology. The information in these fields is ambiguous, and machine learning is very suitable for the analysis of such ambiguous information.

[0182] Based on the above steps, partial multi-label learning (PML) also includes the following general steps.

[0183] S5. Training a partial multi-label classification model according to the sample data set, the disambiguated labels and the most important features.

[0184] S6. Obtain data to be classified.

[0185] The sample data and the data to be classified can be specific data in the following fields: 1. Biological information analysis: gene sequence, protein data, biomarkers. 2. Public network data sets: website traffic data, network connection records. 3. Industrial automation: production line sensor data, machine status information. 4. Transportation: traffic monitoring data, GPS trajectory. 5. Financial field: transaction records, credit scores, payment data. 5. Machine learning model deployment: standard data sets, such as the iris data set, for model demonstration.

[0186] S7. Obtain the most important feature of the data to be classified, and input it into the multi-label classification model to obtain the classification label of the data to be classified.

[0187] Taking the identification of the content in the picture as the classification target of partial multi-label learning (PML) as an example, the original training data set contains 1. Image data: The training data set needs to contain a large number of image files, which are the basis of model learning. Images can be color or grayscale. 2. Candidate label set: Each image will have a candidate label set, which includes all labels that may be related to the image content. These label sets will contain correct labels and some irrelevant noise labels. 3. True label information: In PML, the true labels need to be identified from the candidate label set. These true labels are the target of model training and are used to guide the model to learn which labels are truly related to the image content. 4. Feature description: The feature description or feature vector of the image is also part of the training data. These features can be pixel values, color histograms, texture information, shape descriptors, etc. They are used to represent the image content and are used for model learning. 5. Label correlation information: In some cases, there may be correlation between labels. This information can help the model better understand the relationship between labels and improve learning effects. 6. Image annotation: In crowdsourced image annotation, only a part of the candidate labels given by the annotator are valid labels. The training data needs to contain this annotation information so that the model can learn from it and identify valid labels. 7. Image preprocessing information: including information on preprocessing steps such as image resizing and normalization, which are necessary for model training and prediction.

[0188] The candidate label set is disambiguated through steps S1 and S2. Then, the most useful features are screened out from multiple feature descriptions through steps S3 and S4. Then, a multi-label classification model is trained based on the disambiguated labels and screening.

[0189] When the image to be identified is obtained, features are first extracted from the image, and then the features are input into the multi-label classification model to obtain the label of the image to be identified. The label can be the content contained in the image, such as mountains, clouds, houses, blue sky, people, etc.

[0190] It is understandable that the feature selection device may be an electronic device with computing capabilities, such as a portable notebook computer, a desktop computer, a server, a smart phone, or a tablet computer.

[0191] Embodiment 2: The present invention provides a device for selecting multi-label features based on trusted label induction, which comprises a region division module, a disambiguation module, an aggregation module and a feature selection module.

[0192] The region partitioning module is used to obtain the original training data set with multiple labels, initialize the label confidence matrix based on the original training data set, and construct the distribution areas of credible labels, untrustworthy labels and uncertain labels.

[0193] The disambiguation module is used to update the label confidence matrix according to the distribution of the nearest neighbors, disambiguate the distribution area of ​​the untrustworthy labels, and iteratively update the distribution area of ​​the uncertain labels until the maximum number of iterations is reached to obtain the disambiguated labels.

[0194] The aggregation module is used to aggregate the labels from fine-grained to coarse-grained based on the disambiguated labels, combining granular computing with graph theory, and constructing virtual coarse-grained labels.

[0195] The feature selection module is used to construct a neighborhood rough set model suitable for multi-labeled data based on the virtual coarse-grained labels and multiple features obtained from sample data, and adopt a forward greedy search algorithm to select the most important feature.

[0196] Embodiment 3, the present invention provides a device for selecting partial multi-label features based on credible label induction, comprising a processor, a memory, and a computer program stored in the memory. The computer program can be executed by the processor to implement a method for selecting partial multi-label features based on credible label induction as described in any paragraph of Embodiment 1.

[0197] Embodiment 4: The present invention provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute a method for selecting multi-label features based on trusted label induction as described in any paragraph of Embodiment 1.

[0198] In several embodiments provided in the embodiments of the present invention, it should be understood that the disclosed apparatus and method can also be implemented in other ways. The apparatus and method embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the apparatus, method and computer program product according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0199] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0200] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, an electronic device, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code. It should be noted that in this article, the term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such process, method, article or device. Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0201] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0202] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0203] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0204] The "first\second" mentioned in the embodiments is only to distinguish similar objects, and does not represent a specific order for the objects. It is understandable that the "first\second" can be interchanged with the specific order or sequence where permitted. It should be understood that the objects distinguished by "first\second" can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than those illustrated or described herein.

[0205] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for selecting multi-label features, characterized in that: Include: Obtain the original training data set with multiple labels, initialize the label confidence matrix based on the original training data set, and construct the distribution areas of credible labels, untrustworthy labels, and uncertain labels; According to the distribution of the nearest neighbors, the label confidence matrix is ​​updated, the distribution area of ​​the untrustworthy label is disambiguated, and the distribution area of ​​the uncertain label is iteratively updated until the maximum number of iterations is reached to obtain the disambiguated label; Based on the disambiguated labels, granular computing and graph theory are combined to aggregate the labels from fine-grained to coarse-grained, and construct virtual coarse-grained labels. According to the virtual coarse-grained labels, a neighborhood rough set model suitable for multi-labeled data is constructed in combination with a plurality of features obtained from the sample data, and a forward greedy search algorithm is used to select the most important feature; Obtain the original training data set with multiple labels, initialize the label confidence matrix based on the original training data set, and construct the distribution areas of credible labels, untrustworthy labels, and uncertain labels, including: Get the original training data set with multiple labels; Based on the original training data set, a label confidence matrix uniformly distributed on the candidate label set is constructed ; ； ； In the formula, Indicates Sample data No. Tags The uniformly distributed label confidence, is the number of sample data, is the number of labels, For sample data The candidate label set of Use the bucket KD-Tree algorithm to efficiently search sample data of a neighbor; ； In the formula, Represents sample data of Neighbors, Represents the neighbors of the sample data, Represents sample data No. Neighbors, represents the sample space; According to the Nearest neighbors, get the probability matrix ; ； ； In the formula, For the Sample data No. Tags The probability of Represents sample data of Neighbors, represents the positive example set on the original noise label, Indicates the number of elements in a set, is the smoothing parameter to avoid zero probability cases, Indicates the number of classes corresponding to each label; Initialize a label confidence matrix according to the label confidence matrix and the probability matrix; According to the initialized label confidence matrix, based on" The "criterion" divides the candidate labels into three non-overlapping regions to obtain the disambiguation label confidence matrix; the three non-overlapping regions are the credible label region, the uncredible label region, and the uncertain label region; Based on the disambiguated labels, granular computing and graph theory are combined to aggregate the labels from fine-grained to coarse-grained, and construct virtual coarse-grained labels, including: According to the disambiguated label, take the label and As vertices, create a completely undirected graph ;in, , is a set of vertices, is a set of edges; define labels and The weight of the connecting edge is the label dissimilarity ; ； In the formula, It is the disambiguation label generated after the credible label induction phase is completed. The positive example set of It is the label after completing the iteration in the trusted label induction phase. The set of positive examples of ; According to the completely undirected graph, the minimum spanning tree is first constructed using the Prim algorithm. , and then remove some edges to divide the minimum spanning tree into multiple subtrees; the threshold for removing edges for: ； In the formula, is the number of labels; Each subtree is considered as a coarse-grained label aggregated from multiple fine-grained labels. According to the co-occurrence frequency of labels in the coarse-grained label, the values ​​of the fine-grained labels are aggregated into a new coarse-grained label to obtain a virtual coarse-grained label. ; Virtual coarse-grained label Down The values ​​are: ； In the formula, For sample data No. Tags, is a coarse-grained label consisting of one or more labels. express The number of tags in Representation Tags Download Sample The value of ξ is a parameter used to control whether the instance belongs to a virtual coarse-grained label. It means that if the condition is met, it returns 1, otherwise it returns 0.

2. A method for selecting multi-label features according to claim 1, characterized in that: Initialization model of the label confidence matrix: ； In the formula, is an element in the matrix, indicating the Sample data No. Tags is the label confidence of the true label, is the number of sample data, is the number of labels, represents the label confidence matrix uniformly distributed on the candidate label set, is a probability matrix constructed from neighborhood information, is the equilibrium parameter; Disambiguation label confidence matrix for: ； ； In the formula, is the element in the current disambiguation label matrix, represents the mean of each sample in the current disambiguation label matrix, Represents the standard deviation of each sample in the current disambiguation label matrix.

3. A method for selecting multi-label features according to claim 1, characterized in that: According to the distribution of the nearest neighbors, the label confidence matrix is ​​updated, the distribution area of ​​the untrustworthy label is disambiguated, and the distribution area of ​​the uncertain label is iteratively updated until the maximum number of iterations is reached to obtain the disambiguated label, which includes: For Iterations, sample data of The neighbors are updated to ; in, For sample data No. a neighbor; The probability matrix can be updated as : ； ； In the formula, For the The iteration Sample data No. Tags The probability of For the Sample data for iterations of Neighbors, Indicates passing The positive example set of disambiguation labels generated after iterations, Indicates the number of elements in a set, is the smoothing parameter to avoid zero probability cases, Indicates the number of classes corresponding to each label; exist Propagate the current label confidence matrix based on , the label confidence matrix is ​​updated as : ; is the balance parameter; according to Update the distribution areas of trusted labels, untrusted labels, and uncertain labels to obtain the The current disambiguation label confidence matrix of the iteration is updated as : ； ； In the formula, For the The elements in the current disambiguation label confidence matrix of the iteration, represents the mean of each sample in the current disambiguation label matrix, Represents the standard deviation of each sample in the current disambiguation label matrix.

4. The method for selecting a multi-label feature according to claim 1, characterized in that: According to the virtual coarse-grained labels, combined with multiple features obtained from the sample data, a neighborhood rough set model suitable for multi-labeled data is constructed, and a forward greedy search algorithm is used to select the most important feature, specifically including: According to the virtual coarse-grained label, combined with multiple features obtained from the sample data, a local multi-label decision system is constructed , where: For the sample data set, is the feature set, is a set of virtual coarse-grained labels; based on , according to the bucket KD-Tree algorithm, search all sample data In feature subset on Nearest neighbors, get the neighborhood; ； In the formula, Represents sample data In feature subset on Neighbors, Represents the neighbors of the sample data, Represents sample data No. Neighbors, is a subset of the feature set; definition is a virtual coarse-grained label The decision is The sample set is the coarse-grained label set About Feature Subsets The upper and lower approximations are: ； In the formula, is a set of virtual coarse-grained labels about the feature subset The lower approximation of is the number of virtual coarse-grained labels, is the sequence number of the virtual coarse-grained label, is a virtual coarse-grained label about a feature subset The lower approximation of is a virtual coarse-grained label The number of sample sets for decision making, Indicates the first samples, Indicates Sample data, Represents sample data In feature subset on Neighbors, is a set of virtual coarse-grained labels about the feature subset The upper approximation of is a virtual coarse-grained label about a feature subset The upper approximation of represents the empty set; According to the upper and lower approximations, define virtual coarse-grained labels For feature subsets The degree of dependence is: ； In the formula, is a virtual coarse-grained label For feature subsets The degree of dependence is a virtual coarse-grained label About Feature Subsets The lower approximation of is a sample data set; Based on the degree of dependency, define the features Relative to The importance of is: ； In the formula, Indicates importance, Represents a feature set The features that do not belong to feature subset A, represents a virtual coarse-grained label, Indicates the degree of dependence; Based on the importance, a forward greedy search algorithm is used to select the most important feature ; ； In the formula, Indicates that multiple importance values ​​are greater than 0 of and is the index of the feature.

5. A device for selecting multi-label features, characterized in that: Include: The region partitioning module is used to obtain the original training data set with multiple labels, initialize the label confidence matrix based on the original training data set, and construct the distribution areas of credible labels, untrustworthy labels, and uncertain labels; The disambiguation module is used to update the label confidence matrix according to the distribution of the nearest neighbors, disambiguate the distribution area of ​​the untrustworthy labels, and iteratively update the distribution area of ​​the uncertain labels until the maximum number of iterations is reached to obtain the disambiguated labels; The aggregation module is used to aggregate the labels from fine-grained to coarse-grained based on the disambiguated labels, combining granular computing and graph theory, and constructing virtual coarse-grained labels; A feature selection module is used to construct a neighborhood rough set model suitable for partial multi-labeled data based on the virtual coarse-grained labels and multiple features obtained from the sample data, and to select the most important feature using a forward greedy search algorithm; The area division module is specifically used for: Get the original training data set with multiple labels; Based on the original training data set, a label confidence matrix uniformly distributed on the candidate label set is constructed ; ； ； In the formula, Indicates Sample data No. Tags The uniformly distributed label confidence, is the number of sample data, is the number of labels, For sample data The candidate label set of Use the bucket KD-Tree algorithm to efficiently search sample data of a neighbor; ； In the formula, Represents sample data of Neighbors, Represents the neighbors of the sample data, Represents sample data No. Neighbors, represents the sample space; According to the Nearest neighbors, get the probability matrix ; ； ； In the formula, For the Sample data No. Tags The probability of Represents sample data of Neighbors, represents the positive example set on the original noise label, Indicates the number of elements in a set, is the smoothing parameter to avoid zero probability cases, Indicates the number of classes corresponding to each label; Initialize a label confidence matrix according to the label confidence matrix and the probability matrix; According to the initialized label confidence matrix, based on" The "criterion" divides the candidate labels into three non-overlapping regions to obtain the disambiguation label confidence matrix; the three non-overlapping regions are the credible label region, the uncredible label region, and the uncertain label region; Aggregation module, specifically used for: According to the disambiguated label, take the label and As vertices, create a completely undirected graph ;in, , is a set of vertices, is a set of edges; define labels and The weight of the connecting edge is the label dissimilarity ; ； In the formula, It is the disambiguation label generated after the credible label induction phase is completed. The positive example set of It is the label after completing the iteration in the trusted label induction phase. The set of positive examples of ; According to the completely undirected graph, the minimum spanning tree is first constructed using the Prim algorithm. , and then remove some edges to divide the minimum spanning tree into multiple subtrees; the threshold for removing edges for: ； In the formula, is the number of labels; Each subtree is considered as a coarse-grained label aggregated from multiple fine-grained labels. According to the co-occurrence frequency of labels in the coarse-grained label, the values ​​of the fine-grained labels are aggregated into a new coarse-grained label to obtain a virtual coarse-grained label. ; Virtual coarse-grained label Down The values ​​are: ； In the formula, For sample data No. Tags, is a coarse-grained label consisting of one or more labels. express The number of tags in Representation Tags Download Sample The value of ξ is a parameter used to control whether the instance belongs to a virtual coarse-grained label. It means that if the condition is met, it returns 1, otherwise it returns 0.

6. A multi-label feature selection device, characterized in that: It comprises a processor, a memory, and a computer program stored in the memory; the computer program can be executed by the processor to implement a multi-label feature selection method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute a multi-label feature selection method as described in any one of claims 1 to 4.

Images