Partially multi-label feature selection method and device based on adaptive manifold and mutual information

Abstract

The present invention discloses a method and device for selecting partial multi-label features based on adaptive manifold and mutual information, which relates to the field of data mining. The method comprises the following steps: S1, constructing a feature matrix and a logical label matrix using a data set; S2, constructing a feature selection model; S3, constructing a manifold regularization term and a label similarity graph learning term of an objective function based on the feature matrix and the logical label matrix; S4, training the feature selection model using the objective function; S5, inputting the partial multi-label data set into the trained feature selection model to obtain a label confidence matrix and a weight matrix of the feature selection model, calculating the mutual information between the feature matrix and the label confidence matrix, calculating the feature importance matrix based on the mutual information, and screening the final feature selection results based on the importance ranking. The present invention eliminates the influence of noise labels on label correlation by dynamically learning the label similarity graph weight matrix, and realizes feature selection by combining the weight matrix with the mutual information of the label confidence.

Description

Technical Field

[0001] The present invention relates to the field of data mining, and in particular to a method and device for selecting partial multi-label features based on adaptive manifold and mutual information. Background Art

[0002] Multi-label learning assumes that each data instance should contain all possible labels associated with it, and these labels should be accurate. However, compared to single-label data, the number of candidate labels for multi-label data increases, and the task of labeling data is more difficult. The annotator may roughly assign a set of candidate labels to each sample, resulting in the sample's label set containing irrelevant noise labels. These noise labels seriously affect the performance of the multi-label classification model. In order to reduce the impact of noise labels on model performance, researchers have proposed a framework for model learning from noisy labels. The research paradigm for multi-label learning problems is mainly partial multi-label learning.

[0003] However, existing partial multi-label learning methods mainly focus on how to identify noise labels and remove them to reduce their impact on model performance, while ignoring the impact of features on the model. In order to describe complex and diverse semantic information, the features of multi-label data are usually high-dimensional. In the feature space, there are many cases where features lack correlation and are redundant, and only a few discriminative features can provide useful information when distinguishing different categories or predicting targets. Existing partial multi-label learning methods do not filter or eliminate original features during the modeling process, but instead use all information in the data space for modeling and prediction. Doing so not only carries the risk of overfitting, but also increases computational costs. How to reduce the feature dimensionality of multi-label data in the presence of noise labels has become one of the important challenges facing partial multi-label learning. Summary of the Invention

[0004] In response to the above problems, the present invention proposes a method and device for partial multi-label feature selection based on adaptive manifold and mutual information. The relationship between the label confidence matrix and the feature matrix of the input sample is constructed through the feature selection model. The feature similarity and semantic similarity of the training samples are used to construct the similarity graph weight matrix, and the label similarity graph weight matrix is ​​dynamically learned to eliminate the influence of noise labels on label correlation. Then, the label confidence matrix is ​​guided to learn based on the feature similarity graph weight matrix and the label similarity graph weight matrix, thereby completing partial multi-label disambiguation. Finally, the method is used to The norm constrains the model weights to obtain the correlation between features and label confidence, and combines the model weight matrix and the mutual information between features and label confidence to achieve feature selection; and uses adaptive manifold learning and non-convex sparse regularization technology to provide a computable data preprocessing method for high-dimensional data in multi-label environments.

[0005] On the one hand, the partial multi-label feature selection method based on adaptive manifold and mutual information has the following specific steps:

[0006] S1, a step of processing a multi-labeled dataset, obtaining a multi-labeled dataset and constructing a feature matrix; converting candidate tags of the multi-labeled dataset into logical tags and constructing a logical tag matrix;

[0007] S2, feature selection model construction step, constructs a feature selection model with feature matrix as input, weight matrix as model training parameter, and label confidence matrix as output;

[0008] S3, manifold regularization term construction step,

[0009] The similarity between features is calculated based on the feature matrix to obtain a feature similarity matrix, the semantic similarity matrix is ​​calculated based on the logical label matrix, and the feature graph weight matrix is ​​calculated based on the feature similarity matrix and the semantic similarity matrix;

[0010] Obtaining an initial label correlation matrix based on the logical label matrix, constructing a label similarity graph learning item of the objective function of the feature selection model based on the initial label correlation matrix, and learning the label graph weight matrix during the feature selection model training process;

[0011] The manifold regularization term of the objective function of the feature selection model is constructed based on the label map weight matrix and the feature map weight matrix;

[0012] S4, feature selection model training step, using the objective function to train the feature selection model to obtain a trained feature selection model;

[0013] S5, feature selection step, inputs the over-labeled data set into the trained feature selection model to obtain the label confidence matrix and the trained weight matrix; calculates the mutual information of the feature matrix and the label confidence matrix to obtain the mutual information matrix; calculates the feature importance matrix based on the mutual information matrix and the trained weight matrix, calculates the importance of each dimension of the feature matrix to all categories based on the feature importance matrix, and selects the features of the corresponding dimension according to the importance ranking to obtain the final feature selection result.

[0014] Preferably, the feature selection model is expressed as:

[0015] ;

[0016] in, Represents the i-th sample in the feature matrix The label confidence vector and label confidence matrix , represents the total number of samples, Indicates the number of categories; Represents the weight matrix of the feature selection model; Represents the bias vector of the feature selection model; represents the exponential function, Represents a summation function.

[0017] Preferably, the similarity between features is calculated based on the feature matrix to obtain a feature similarity matrix, the semantic similarity matrix is ​​calculated based on the logical label matrix, and the feature graph weight matrix is ​​calculated based on the feature similarity matrix and the semantic similarity matrix, as follows:

[0018] Calculate the Wasserstein distance between samples in the feature matrix to obtain the Wasserstein distance matrix; it is expressed as:

[0019] ;

[0020] in, Indicates the samples Hedi samples The Wasserstein distance between all Construct the Wasserstein distance matrix; express and The proportional function, express and The geodesic distance between represents the total sample size;

[0021] The feature similarity matrix between samples is calculated based on the Wasserstein distance matrix, which is expressed as:

[0022] ;

[0023] in, Indicates the samples Hedi samples The feature similarity of all Construct a feature similarity matrix; represents the exponential function, represents the Gaussian kernel size;

[0024] According to the logical labeling matrix, the semantic similarity matrix is ​​calculated using the Jaccard similarity coefficient, which is expressed as:

[0025] ;

[0026] in, Indicates the samples Hedi samples The corresponding semantic similarity, all Construct a semantic similarity matrix; Indicates the first samples the corresponding logical label vector; Indicates the samples the corresponding logical label vector; represents the intersection of sets, Represents the union of sets; Indicates the cardinality of a set;

[0027] The feature similarity graph weight matrix is ​​calculated based on the feature similarity matrix and the semantic similarity matrix, which is expressed as:

[0028] ;

[0029] in, Indicates the samples Hedi samples The similarity graph weights between all Construct a feature similarity graph weight matrix; represents the weight coefficient of the linear combination.

[0030] Preferably, the initial label correlation matrix is ​​obtained based on the logical label correlation matrix, and the label similarity graph learning item of the objective function of the feature selection model is constructed according to the initial label correlation matrix, which is specifically as follows:

[0031] The initial marker correlation matrix is ​​calculated using the logical marker matrix, which is expressed as:

[0032] ;

[0033] in, Indicates the Class and The initial label correlation of all classes Constructing an initial marker correlation matrix; Indicates the a logical vector of classes; Indicates the a logical vector of classes; represents the transpose of a vector, Represents a vector norm;

[0034] The label similarity graph learning term is constructed based on the initial label correlation matrix, which is expressed as:

[0035] ;

[0036] ;

[0037] in, represents the weight matrix of labeled similarity graph; express Middle Class and Similarity graph weights between classes; and Respectively Middle elements and elements; and They represent the first elements and elements; 、 、 and The diagonal elements of the degree matrix representing the initial marker correlation matrix; express The transpose of express The diagonal elements of Indicates constraints; Indicates the number of categories; Indicates taking the minimum value.

[0038] Preferably, the manifold regularization term of the objective function is expressed as:

[0039] ;

[0040] ;

[0041] in, Indicates taking the minimum value; and Respectively represent samples Hedi samples The label confidence vector of ; and Respectively represent Class and The label confidence vector of each class; represents the total sample size; Indicates the number of categories; Indicates the samples Hedi samples Similarity graph weight between ; Represents the first Class and Similarity graph weights between classes; Represents a vector norm; Represents matrix transpose; Represents the feature similarity graph weight matrix The Laplace matrix of Represents the weight matrix of the labeled similarity graph The Laplace matrix of Represents the rank of the matrix.

[0042] Preferably, the mutual information between the feature matrix and the label confidence matrix is ​​calculated to obtain a mutual information matrix; the feature importance matrix is ​​calculated based on the mutual information matrix and the trained weight matrix, and the importance of each dimension of the feature matrix to all categories is calculated based on the feature importance matrix. The features of the corresponding dimensions are screened out according to the importance ranking to obtain the final feature selection result, which is specifically as follows:

[0043] Calculate the mutual information of the feature matrix and the label confidence matrix to obtain the mutual information matrix, which is expressed as:

[0044] ;

[0045] in, Indicates the Dimensional features and The correlation of all classes Construct the mutual information matrix; Indicates the sample dimensional features; Indicates the The labeling confidence of each class; express and The joint probability of express and The conditional probability of express probability;

[0046] Normalize the mutual information matrix so that the sum of the correlations between each dimension feature and all labels is 1;

[0047] The feature importance matrix is ​​calculated based on the normalized mutual information matrix and the model weight matrix; it is expressed as:

[0048] ;

[0049] in, Indicates the Dimensional feature pair The importance of each class, Construct the feature importance matrix; Represents the trained weight matrix elements; represents the normalized mutual information matrix elements; is the weight coefficient of the linear combination;

[0050] According to the feature importance matrix, the importance of each dimension feature to all categories is calculated, which is expressed as;

[0051] ;

[0052] in, The first The row vector of the dimensional features, Represents a vector norm; Indicates the The importance of dimensional features; Indicates the number of feature dimensions; It indicates universal quantifier;

[0053] According to importance Sort in descending order and select the top Important features and their corresponding indexes, get the selected feature index set, where, Indicates the number of selected features; the final feature selection result is obtained based on the features corresponding to the selected feature index set.

[0054] Preferably, it is characterized in that the objective function also includes a sparse regularization term and a loss function, and the objective function is a weighted combination of the manifold regularization term, the sparse regularization term, the loss function and the labeled similarity graph learning term.

[0055] Preferably, the loss function is expressed as:

[0056] ;

[0057] in, Indicates taking the minimum value; Indicates the The label confidence vector of the sample No. elements; represents the label confidence matrix; represents the logical label matrix; and Represent the weight matrix No. and vectors; and Represent the bias vectors No. and elements; represents the logarithmic function with base 2; Represents the matrix norm; represents the total sample size; Indicates the number of categories.

[0058] Preferably, the sparse regularization term is expressed as:

[0059] ;

[0060] in, Represents the matrix norm, ; represents the number of feature dimensions, Indicates the number of categories; Representation matrix No. elements.

[0061] On the other hand, the partial multi-label feature selection device based on adaptive manifold and mutual information includes the following:

[0062] The multi-label dataset processing module is used to obtain the multi-label dataset and construct a feature matrix; convert the candidate labels of the multi-label dataset into logical labels and construct a logical label matrix;

[0063] Feature selection model construction module, used to build a feature selection model with feature matrix as input, weight matrix as model training parameter, and label confidence matrix as output;

[0064] A manifold regularization term construction module is used to calculate the similarity between features based on the feature matrix to obtain a feature similarity matrix, calculate the semantic similarity matrix based on the logical label matrix, and calculate the feature graph weight matrix based on the feature similarity matrix and the semantic similarity matrix; obtain the initial label correlation matrix based on the logical label matrix, construct the label similarity graph learning term of the objective function of the feature selection model based on the initial label correlation matrix, and learn the label graph weight matrix during the feature selection model training process; and construct the manifold regularization term of the objective function of the feature selection model based on the label graph weight matrix and the feature graph weight matrix.

[0065] The feature selection model training module is used to train the feature selection model using the objective function to obtain a trained feature selection model;

[0066] The feature selection module inputs the multi-labeled dataset into the trained feature selection model to obtain the label confidence matrix and the trained weight matrix; calculates the mutual information of the feature matrix and the label confidence matrix to obtain the mutual information matrix; calculates the feature importance matrix based on the mutual information matrix and the trained weight matrix, calculates the importance of each dimension of the feature matrix to all categories based on the feature importance matrix, and selects the features of the corresponding dimensions according to the importance ranking to obtain the final feature selection result.

[0067] Compared with the prior art, the present invention has the following beneficial effects:

[0068] (1) The present invention first constructs the relationship between the label confidence matrix and the sample feature matrix through a nonlinear model, constructs the similarity graph weight matrix of the training sample and its neighbors using the feature similarity and semantic similarity of the training sample, and dynamically learns the label similarity graph weight matrix to eliminate the influence of noise labels on label correlation; then guides the label confidence matrix learning based on the feature similarity graph weight matrix and the label similarity graph weight matrix, and then completes the partial multi-label disambiguation; finally, adopts The norm constrains the model weights to obtain the correlation between features and label confidence, and combines the model weight matrix and the mutual information between features and label confidence to achieve feature selection;

[0069] (2) The present invention uses adaptive manifold learning and non-convex sparse regularization technology to provide a computable data preprocessing method for high-dimensional data in a multi-label environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] The present invention will be described in further detail below with reference to the accompanying drawings;

[0071] Figure 1 Flowchart of a partial multi-label feature selection method based on adaptive manifold and mutual information according to an embodiment of the present invention;

[0072] Figure 2 Schematic diagram of the process of the partial multi-marker feature selection method based on adaptive manifold and mutual information according to an embodiment of the present invention;

[0073] Figure 3 This is a diagram illustrating a method for selecting multiple labels based on adaptive manifold and mutual information according to an embodiment of the present invention;

[0074] Figure 4 This is a structural block diagram of a device for selecting partial multi-label features based on adaptive manifold and mutual information according to an embodiment of the present invention. DETAILED DESCRIPTION

[0075] The present invention is further described below through specific embodiments.

[0076] With the development of internet technology, the amount of data has increased dramatically, and the amount of data that needs to be labeled is also growing rapidly. Labeling multi-label data is inherently time-consuming. As the data volume increases, annotators can only roughly label the data. As a result, the candidate label set for the data contains not only relevant labels but also a small number of irrelevant labels, known as noise labels. Using this type of imprecisely labeled data to build effective learning models is defined as partial multi-label learning.

[0077] The method presented in this paper is primarily applied to the field of partial multi-label learning and represents a novel multi-label learning framework. In this framework, each example is assigned multiple candidate labels, only some of which are correct. The focus of this paper is on how to select features from these multiple labels, even those containing noisy ones, to improve the efficiency and performance of subsequent classification learning.

[0078] like Figure 1 and Figure 2 As shown in Figure 2, the partial multi-label feature selection method based on adaptive manifold and mutual information has the following specific steps:

[0079] S1, processing steps for multi-label dataset.

[0080] Obtain a multi-label dataset and construct a feature matrix; convert the candidate tags of the multi-label dataset into logical tags and construct a logical tag matrix, as follows:

[0081] S11, obtain the multi-labeled dataset to be processed, denoted as ,in, Indicates the training instances, Indicates that the dimension in the range of real numbers is vector, express of dimensional features, is the characteristic dimension of the sample; express The candidate tag set, Is a The output space of possible class labels, is the total sample size, Indicates the number of categories.

[0082] In the partially multi-labeled dataset to be processed, each sample contains sample features and candidate labels. The candidate labels are divided into correct labels and irrelevant labels (i.e., noise labels). The partially multi-labeled dataset to be processed can be some labeled sample images, such as Figure 3 As shown in the figure, the image sample is labeled by the labeler as "lake", "tree", "duck", "swan", "cloud", "bird" and "house", which are the candidate tag sets for the image sample. However, "cloud", "bird" and "house" in these candidate tag sets are not elements contained in the image and are noise tags.

[0083] S12, for each over-labeled sample , constructed using the hard thresholding method Logical label vector of , for any ,if ,but ,otherwise ;in, For the Category tags, express No. The logical value of the candidate label. Then, the logical label matrix of all training samples is constructed and the feature matrix .in, represents the matrix transpose, represents the set of real numbers, is the total sample size, represents the dimension of the feature space, is the number of categories.

[0084] The construction formula of the logical label vector is:

[0085] ;

[0086] S2, feature selection model building step.

[0087] Construct a feature selection model with feature matrix as input, weight matrix as model training parameters, and label confidence matrix as output.

[0088] According to the feature matrix, obtain the label confidence matrix , expressed as:

[0089] ;

[0090] in, For samples The label confidence vector of Representation sample Belong to category probability; is the weight matrix of the model, is the bias vector of the model; represents the exponential function, Represents a summation function.

[0091] Specifically, each line in the multi-label dataset is set as a sample. is the logical label matrix of all samples, is the feature matrix of all samples. Assume that the instance sample (ie: feature matrix ) and the label confidence matrix There is a nonlinear mapping relationship between them, and then the label confidence matrix is ​​obtained through this nonlinear mapping relationship. .

[0092] S3, manifold regularization term construction step.

[0093] By mining the topological structure of the feature space, the similarity graph weights of the features are obtained; a correlation adaptive learning mechanism of the tag space is constructed to dynamically learn the similarity graph weights of the tags; and a manifold regularization term is constructed based on the similarity graph weights of the features and the similarity graph weights of the tags to guide the tag confidence learning. The method includes the following steps:

[0094] S31, using Wasserstein distance to measure the differences between instances and calculate the Wasserstein distance matrix , expressed as:

[0095] ;

[0096] ;

[0097] in, Indicates the samples Hedi samples Wasserstein distance between them; represents the proportional function, express and The geodesic distance between is the total sample size.

[0098] Calculate the feature similarity matrix between samples based on Wasserstein distance , expressed as:

[0099] ;

[0100] ;

[0101] in, represents the exponential function, Represents the Gaussian kernel size.

[0102] Specifically, considering that in high-dimensional space, the commonly used Euclidean distance often cannot well describe the true distance and difference between instances, nor can it reflect the true topological structure of manifold data, the Wasserstein distance is used. To measure the difference between each group of samples in the feature space.

[0103] S32, using the Jaccard similarity coefficient to construct the semantic similarity matrix between samples ; expressed as:

[0104] ;

[0105] ;

[0106] in, express and Corresponding semantic similarity; and Respectively represent Instances Hedi Instances the corresponding logical label vector; represents the intersection of sets, Represents the union of sets.

[0107] S33, calculating the feature similarity graph weight matrix of the training instance based on the feature similarity matrix and the semantic similarity matrix , expressed as:

[0108] ;

[0109] ;

[0110] in, For the samples Hedi samples The similarity graph weight between is the weight coefficient of the linear combination.

[0111] In this embodiment, if the overlap of candidate tag sets of two samples is small, the similarity of the two samples should be relatively low. Therefore, the feature similarity and semantic similarity are combined to construct the feature similarity weight matrix of the samples. .

[0112] S34, calculating the initial marker correlation matrix using the logical marker matrix .

[0113] Specifically, the tag confidence matrix is ​​learned based on the correlation between tags to complete tag disambiguation. In addition, in order to reduce the impact of noise tags on tag confidence learning, the tag similarity graph weight matrix is ​​dynamically learned during the model learning process. The initial tag correlation matrix is ​​calculated using the logical tag matrix. To measure the correlation between different classes. The initial label correlation matrix is ​​expressed as:

[0114] ;

[0115] ;

[0116] in, Indicates the Class and The initial label correlation of the classes, and Respectively represent Class and a logical vector of classes; represents the transpose of a vector, Represents a vector norm.

[0117] S35, based on the initial marker correlation matrix Perform dynamic learning to obtain the weight matrix of the labeled similarity graph , .

[0118] Specifically, the correlation matrix with the initial label is constructed through graph diffusion theory The loss function learns the weight matrix of labeled similar graphs . The learning formula is:

[0119] ;

[0120] ;

[0121] in, Indicates the Class and Similarity graph weights between classes; and Respectively No. elements and elements, and Respectively No. elements and elements; 、 、 for The diagonal elements of the degree matrix are calculated as follows: ; express The transpose of express diagonal elements of .

[0122] S36, feature similarity graph weight matrix based on training instances and label correlation graph weight matrix , construct the manifold regularization term to induce the label confidence matrix study.

[0123] The formula for the manifold regularization term is:

[0124] ;

[0125] ;

[0126] in, and Represents samples and samples The label confidence vector of and Respectively represent Class and The label confidence vector of each class; is the total sample size, is the number of categories; For the samples Hedi samples The similarity graph weight between For the Class and Similarity graph weights between classes; For vector norm, Represents matrix transpose; is a matrix The Laplace matrix of is a diagonal matrix whose diagonal elements are is a matrix The Laplace matrix of is a diagonal matrix whose diagonal elements are ; Represents the rank of the matrix.

[0127] In this embodiment, based on the smoothness assumption, the label confidence should maintain the relationship between samples in the domain in the sample feature space, and the correlation between label confidences should be consistent with the label correlation. Therefore, a manifold regularization term is constructed to induce label confidence matrix learning.

[0128] S4, feature selection model training step.

[0129] In order to obtain the weight matrix and bias of the model, the following loss function is constructed based on the feature matrix, logical label matrix and label confidence matrix:

[0130] ;

[0131] in, Represents a vector No. elements, and Respectively No. and vectors, and Respectively No. and elements; represents the logarithmic function with base 2, For the matrix norm; is the total sample size, is the number of categories.

[0132] use The norm constrains the model weight matrix and constructs a sparse regularization term to guide model weight learning.

[0133] The formula for the sparse regularization term is:

[0134] ;

[0135] in, Represents the matrix norm, ; is the number of feature dimensions, is the number of categories; is a matrix No. elements.

[0136] Specifically, in order to make the relevant features of the overly labeled data obtain greater weights and reduce the weights of redundant or less important features to zero, the sparse coding technology is used to Norm regularization imposes a sparsity constraint on the model weight matrix.

[0137] Therefore, the objective function of partial multi-label feature selection includes loss function, manifold regularization term, sparse regularization term and label similarity graph learning term. The formula of the objective function is expressed as:

[0138] ;

[0139] ;

[0140] in, 、 and represents the regularization parameter, is the weight matrix of the model, represents the bias vector, is the label confidence matrix, is the weight matrix of labeled similarity graphs, represents the initial label correlation matrix, 、 and for The diagonal elements of the degree matrix, is the feature similarity graph weight matrix, is a matrix The Laplace matrix of is a diagonal matrix, is a matrix The Laplace matrix of is a diagonal matrix; is the total sample size, Indicates the number of categories; represents the transpose of matrices and vectors, For the matrix norm, Represents the matrix norm, represents the rank of the matrix; express diagonal elements of .

[0141] S5, feature selection step.

[0142] The alternating iterative update strategy is used to optimize the objective function until convergence or the maximum number of iterations is reached, and the label disambiguation is completed to obtain the optimal model parameters of the partial multi-label feature selection model. and ; Calculate the label confidence matrix of the training instance according to the nonlinear model, expressed as:

[0143] ;

[0144] in, and is the optimal model parameter obtained by training, Indicates a dimensional all-1 vector, Represents vector transpose.

[0145] Specifically, a commonly used alternating iterative update strategy is used to update the variables in the objective function. Iterate and update until the objective function converges or the maximum number of iterations is reached. By optimizing this formula, the optimal model parameters can be obtained. and , and calculate the label confidence matrix of the training instance through the optimal model parameters .

[0146] S51, calculate the mutual information matrix of features and label confidence .

[0147] ;

[0148] ;

[0149] in, Indicates the sample dimensional features, Indicates the The label confidence of each class, express and The joint probability of express and The conditional probability of express The probability of Indicates the Dimensional features and Class correlation.

[0150] Normalize the mutual information matrix to obtain a normalized mutual information matrix , expressed as:

[0151] ;

[0152] ;

[0153] in, express The entropy of express In this embodiment, the information theory technology is used to obtain the correlation between each dimension feature and each label, and the mutual information matrix is ​​normalized so that the sum of the correlations between each dimension feature and all labels is 1.

[0154] S52, calculating the feature importance matrix according to the normalized mutual information matrix and the model weight matrix .

[0155] ;

[0156] ;

[0157] in, Indicates the Dimensional feature pair The importance of a class, is a matrix No. elements, is the weight coefficient of the linear combination.

[0158] According to the feature importance matrix, the importance of each dimension feature to all categories is calculated. The calculation formula is:

[0159] ;

[0160] in, express No. OK, For vector norm, For the The importance of dimensional features; Indicates a universal quantifier.

[0161] According to importance Sort in descending order and select the top Important features and their corresponding indexes, get the selected feature index set .in, Indicates the number of selected features. According to the selected feature index set, the final feature selection result is obtained. .

[0162] In this example, the final feature selection results are achieved by combining a filtering approach with an embedded approach. Specifically, a filtering approach (i.e., calculating the mutual information between features and label confidence) and an embedded approach (model weight matrix) are used to determine the importance of features to labels. These two importances are then combined through linear interpolation to obtain the final importance score. Based on this importance score, the optimal feature subset is selected to achieve feature dimensionality reduction for overly labeled data.

[0163] In summary, the partial multi-label feature selection method based on adaptive manifold and mutual information was the first to explore the impact of semantic relevance on the sample topological structure, and proposed a new learning framework for partial multi-label learning. In addition, this method invented a new label similarity graph weight matrix learning mechanism, which aims to eliminate the influence of noise labels on label correlation. Next, the present invention adopts sparse regularization technology to achieve feature selection. Finally, the partial multi-label feature selection method based on adaptive manifold and mutual information obtains the importance of features by integrating the mutual information of features and label confidence and the model weight matrix, so that the learning method can adapt to the partial multi-label environment of high-dimensional data, and improve the computability and effectiveness of partial multi-label learning.

[0164] The present invention applies manifold learning and information theory to the algorithm of the present invention, providing a computable data preprocessing method for multi-label environments with high-dimensional data. It has important practical application value and theoretical significance, and shows significant advantages in applications in the fields of text classification, ecological informatics, and multimedia content analysis.

[0165] like Figure 4 As shown, the present invention also discloses a partial multi-label feature selection device based on adaptive manifold and mutual information, comprising:

[0166] The multi-labeled dataset processing module 401 is used to obtain the multi-labeled dataset and construct a feature matrix; convert the candidate labels of the multi-labeled dataset into logical labels and construct a logical label matrix;

[0167] A feature selection model construction module 402 is used to construct a feature selection model with a feature matrix as input, a weight matrix as model training parameters, and a label confidence matrix as output;

[0168] Manifold regularization term construction module 403 is used to calculate the similarity between features based on the feature matrix to obtain a feature similarity matrix, calculate the semantic similarity matrix based on the logical label matrix, and calculate the feature graph weight matrix based on the feature similarity matrix and the semantic similarity matrix; obtain an initial label correlation matrix based on the logical label matrix, construct a label similarity graph learning term of the objective function of the feature selection model based on the initial label correlation matrix, learn the label graph weight matrix during the feature selection model training process; and construct a manifold regularization term of the objective function of the feature selection model based on the label graph weight matrix and the feature graph weight matrix;

[0169] The feature selection model training module 404 is used to train the feature selection model using the objective function to obtain a trained feature selection model;

[0170] The feature selection module 405 inputs the over-labeled dataset into the trained feature selection model to obtain a label confidence matrix and a trained weight matrix; calculates the mutual information between the feature matrix and the label confidence matrix to obtain a mutual information matrix; calculates the feature importance matrix based on the mutual information matrix and the trained weight matrix, calculates the importance of each dimension of the feature matrix to all categories based on the feature importance matrix, and selects the features of the corresponding dimensions according to the importance ranking to obtain the final feature selection result.

[0171] The specific implementation of the partial multi-label feature selection device based on adaptive manifold and mutual information is the same as the partial multi-label feature selection method based on adaptive manifold and mutual information, and will not be repeated in this embodiment.

[0172] The present invention provides a device for selecting partial multi-marker features based on adaptive manifolds and mutual information, comprising a processor, a memory, and a computer program stored in the memory. The computer program can be executed by the processor to implement the method for selecting partial multi-marker features based on adaptive manifolds and mutual information as described in any paragraph of Example 1.

[0173] The present invention provides a computer-readable storage medium, characterized in that the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute a partial multi-label feature selection method based on adaptive manifold and mutual information as described in any paragraph of Example 1.

[0174] It is understandable that the classification device may be an electronic device with computing capabilities, such as a portable notebook computer, a desktop computer, a server, a smart phone, or a tablet computer.

[0175] In the several embodiments provided in the embodiments of the present invention, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device and method embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of the devices, methods, and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or part of the code, which contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the boxes can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, as well as the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified functions or actions, or can be implemented using a combination of dedicated hardware and computer instructions.

[0176] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0177] If the functions are implemented in the form of software modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, electronic device, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks. It should be noted that, in this document, the terms "comprise," "include," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or device that includes a series of elements includes not only those elements but also other elements not explicitly listed, or also includes elements inherent to such process, method, article, or device. Without further constraints, an element defined by the phrase "comprises a..." does not preclude the existence of additional identical elements in the process, method, article or apparatus that includes the element.

[0178] The above is only a specific implementation of the present invention, but the design concept of the present invention is not limited to this. Any non-substantial changes to the present invention using this concept shall be deemed as an infringement of the protection scope of the present invention.

Claims

1. A partial multi-label feature selection method based on adaptive manifold and mutual information, characterized in that: The steps include: S1, the step of processing the multi-labeled dataset, obtaining the multi-labeled dataset and constructing the feature matrix; Convert candidate tags of multi-label dataset into logical tags and construct a logical tag matrix; S2, feature selection model construction step, constructs a feature selection model with feature matrix as input, weight matrix as model training parameter, and label confidence matrix as output; S3, manifold regularization term construction step, The similarity between features is calculated based on the feature matrix to obtain a feature similarity matrix, the semantic similarity matrix is ​​calculated based on the logical label matrix, and the feature graph weight matrix is ​​calculated based on the feature similarity matrix and the semantic similarity matrix; Obtaining an initial label correlation matrix based on the logical label matrix, constructing a label similarity graph learning item of the objective function of the feature selection model based on the initial label correlation matrix, and learning the label graph weight matrix during the feature selection model training process; The manifold regularization term of the objective function of the feature selection model is constructed based on the label map weight matrix and the feature map weight matrix; S4, feature selection model training step, using the objective function to train the feature selection model to obtain a trained feature selection model; S5, feature selection step, inputs the over-labeled dataset into the trained feature selection model to obtain the label confidence matrix and the trained weight matrix; Calculate the mutual information between the feature matrix and the label confidence matrix to obtain the mutual information matrix; The feature importance matrix is ​​calculated based on the mutual information matrix and the trained weight matrix. The importance of each dimension of the feature matrix to all categories is calculated based on the feature importance matrix. The features of the corresponding dimensions are screened out according to the importance ranking to obtain the final feature selection result.

2. The method for partial multi-label feature selection based on adaptive manifold and mutual information according to claim 1, characterized in that: The feature selection model is expressed as: ； in, Represents the i-th sample in the feature matrix The label confidence vector and label confidence matrix , represents the total number of samples, Indicates the number of categories; Represents the weight matrix of the feature selection model; Represents the bias vector of the feature selection model; represents the exponential function, Represents a summation function.

3. The method for partial multi-label feature selection based on adaptive manifold and mutual information according to claim 1, characterized in that: The similarity between features is calculated based on the feature matrix to obtain a feature similarity matrix, the semantic similarity matrix is ​​calculated based on the logical label matrix, and the feature graph weight matrix is ​​calculated based on the feature similarity matrix and the semantic similarity matrix, as follows: Calculate the Wasserstein distance between samples in the feature matrix to obtain the Wasserstein distance matrix; it is expressed as: ； in, Indicates the samples Hedi samples The Wasserstein distance between all Construct the Wasserstein distance matrix; express and The proportional function, express and The geodesic distance between represents the total sample size; The feature similarity matrix between samples is calculated based on the Wasserstein distance matrix, which is expressed as: ； in, Indicates the samples Hedi samples The feature similarity of all Construct a feature similarity matrix; represents the exponential function, represents the Gaussian kernel size; According to the logical labeling matrix, the semantic similarity matrix is ​​calculated using the Jaccard similarity coefficient, which is expressed as: ； in, Indicates the samples Hedi samples The corresponding semantic similarity, all Construct a semantic similarity matrix; Indicates the first samples the corresponding logical label vector; Indicates the samples the corresponding logical label vector; represents the intersection of sets, Represents the union of sets; Indicates the cardinality of a set; The feature similarity graph weight matrix is ​​calculated based on the feature similarity matrix and the semantic similarity matrix, which is expressed as: ； in, Indicates the samples Hedi samples The similarity graph weights between all Construct a feature similarity graph weight matrix; Represents the weight coefficient of the linear combination.

4. The method for partial multi-label feature selection based on adaptive manifold and mutual information according to claim 1, characterized in that: The initial label correlation matrix is ​​obtained based on the logical label matrix, and the label similarity graph learning item of the objective function of the feature selection model is constructed according to the initial label correlation matrix, which is specifically as follows: The initial marker correlation matrix is ​​calculated using the logical marker matrix, which is expressed as: ； in, Indicates the Class and The initial label correlation of all classes Constructing an initial marker correlation matrix; Indicates the a logical vector of classes; Indicates the a logical vector of classes; represents the transpose of a vector, Represents a vector norm; The label similarity graph learning term is constructed based on the initial label correlation matrix, which is expressed as: ； ； in, represents the weight matrix of labeled similarity graph; express Middle Class and Similarity graph weights between classes; and Respectively Middle elements and elements; and They represent the first elements and elements; 、 、 and The diagonal elements of the degree matrix representing the initial marker correlation matrix; express The transpose of express The diagonal elements of Indicates constraints; Indicates the number of categories; Indicates taking the minimum value.

5. The method for partial multi-label feature selection based on adaptive manifold and mutual information according to claim 1, characterized in that: The manifold regularization term of the objective function is expressed as: ； ； in, Indicates taking the minimum value; and Respectively represent samples Hedi samples The label confidence vector of ; and Respectively represent Class and The label confidence vector of each class; represents the total sample size; Indicates the number of categories; Indicates the samples Hedi samples Similarity graph weight between ; Represents the first Class and Similarity graph weights between classes; Represents a vector norm; Represents matrix transpose; Represents the feature similarity graph weight matrix The Laplace matrix of Represents the weight matrix of the labeled similarity graph The Laplace matrix of Represents the rank of the matrix.

6. The method for partial multi-label feature selection based on adaptive manifold and mutual information according to claim 1, characterized in that: The mutual information between the feature matrix and the label confidence matrix is ​​calculated to obtain a mutual information matrix; The feature importance matrix is ​​calculated based on the mutual information matrix and the trained weight matrix. The importance of each dimension of the feature matrix to all categories is calculated based on the feature importance matrix. The features of the corresponding dimensions are selected according to the importance ranking to obtain the final feature selection results, as follows: Calculate the mutual information of the feature matrix and the label confidence matrix to obtain the mutual information matrix, which is expressed as: ； in, Indicates the Dimensional features and The correlation of all classes Construct the mutual information matrix; Indicates the sample dimensional features; Indicates the The labeling confidence of each class; express and The joint probability of express and The conditional probability of express probability; Normalize the mutual information matrix so that the sum of the correlations between each dimension feature and all labels is 1; The feature importance matrix is ​​calculated based on the normalized mutual information matrix and the model weight matrix; it is expressed as: ； in, Indicates the Dimensional feature pair The importance of each class, Construct the feature importance matrix; Represents the trained weight matrix elements; represents the normalized mutual information matrix elements; is the weight coefficient of the linear combination; The importance of each dimension feature to all categories is calculated according to the feature importance matrix, which is expressed as: ； in, The first The row vector of the dimensional features, Represents a vector norm; Indicates the The importance of dimensional features; Indicates the number of feature dimensions; It indicates universal quantifier; According to importance Sort in descending order and select the top Important features and their corresponding indexes, get the selected feature index set, where, Indicates the number of selected features; the final feature selection result is obtained based on the features corresponding to the selected feature index set.

7. The method for partial multi-label feature selection based on adaptive manifold and mutual information according to claim 1, characterized in that: The objective function also includes a sparse regularization term and a loss function, and the objective function is a weighted combination of the manifold regularization term, the sparse regularization term, the loss function and the labeled similarity graph learning term.

8. The method for partial multi-label feature selection based on adaptive manifold and mutual information according to claim 7, characterized in that: The loss function is expressed as: ； in, Indicates taking the minimum value; Indicates the The label confidence vector of the sample No. elements; represents the label confidence matrix; represents the logical label matrix; and Represent the weight matrix No. and vectors; and Represent the bias vectors No. and elements; represents the logarithmic function with base 2; Represents the matrix norm; represents the total sample size; Indicates the number of categories.

9. The method for partial multi-label feature selection based on adaptive manifold and mutual information according to claim 7, characterized in that: The sparse regularization term is expressed as: ； in, Represents the matrix norm, ; represents the number of feature dimensions, Indicates the number of categories; Representation matrix No. elements.

10. A device for selecting partial multi-label features based on adaptive manifold and mutual information, comprising: The multi-label dataset processing module is used to obtain the multi-label dataset and construct the feature matrix; Convert candidate tags of multi-label dataset into logical tags and construct a logical tag matrix; Feature selection model construction module, used to build a feature selection model with feature matrix as input, weight matrix as model training parameter, and label confidence matrix as output; A manifold regularization term construction module is used to calculate the similarity between features based on the feature matrix to obtain a feature similarity matrix, calculate the semantic similarity matrix based on the logical label matrix, and calculate the feature graph weight matrix based on the feature similarity matrix and the semantic similarity matrix; Obtaining an initial label correlation matrix based on the logical label matrix, constructing a label similarity graph learning item of the objective function of the feature selection model based on the initial label correlation matrix, and learning the label graph weight matrix during the feature selection model training process; The manifold regularization term of the objective function of the feature selection model is constructed based on the label map weight matrix and the feature map weight matrix; The feature selection model training module is used to train the feature selection model using the objective function to obtain a trained feature selection model; The feature selection module inputs the multi-labeled dataset into the trained feature selection model to obtain the label confidence matrix and the trained weight matrix; Calculate the mutual information between the feature matrix and the label confidence matrix to obtain the mutual information matrix; The feature importance matrix is ​​calculated based on the mutual information matrix and the trained weight matrix. The importance of each dimension of the feature matrix to all categories is calculated based on the feature importance matrix. The features of the corresponding dimensions are screened out according to the importance ranking to obtain the final feature selection result.

Images