A dual-channel label-guided multi-label feature selection method and system

Abstract

The application discloses a kind of double-channel label guide multi-label feature selection method and system, belong to feature engineering technique.Method mainly includes: obtaining the feature matrix and positive label matrix of multi-label data set, by performing logical negation to positive label matrix, generate mirror negative label matrix, and construct graph Laplacian matrix based on feature matrix;Based on the data after pre-processing, a multi-label model is constructed, and the objective function of the multi-label model includes at least positive label regression loss term, negative label regression loss term, label alignment constraint term, graph regularization term and sparse constraint term;The constraint is processed by relaxation, and the optimization function is constructed by combining the Lagrange multiplier method, and then the objective function is iteratively solved according to the KKT condition, and after iterative convergence, the feature importance is evaluated based on the projection matrix used to associate features and positive labels;The application meets the demand of multi-label learning for accurate and efficient feature selection, can fully utilize label information, enhance anti-interference ability and consider efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of feature engineering and artificial intelligence, in particular to a double-channel label guided multi-label feature selection method and system, which can efficiently complete the core task in multi-label scenarios such as text classification and image annotation. BACKGROUND

[0002] In real-world scenarios, many objects often correspond to multiple labels at the same time (such as a web page can be labeled as sports, military, finance, health, food, etc.), such multi-label data widely exists in text classification, image annotation, type identification, music style division, etc. fields, and multi-label learning has thus become a core technology for processing such data. However, with the rapid development of science and technology and computers, multi-label data sets generally exhibit high-dimensional feature space characteristics, which contain a large number of redundant and irrelevant attributes, which not only greatly increases the model training cost, but also significantly reduces the classification accuracy of the learning model, so it is crucial to alleviate the negative effects of irrelevant features, discard redundant features by extracting key features to reduce the dimension and preserve the original feature semantics, in order to improve the performance of multi-label learning. As a key solution, feature selection, which is different from feature extraction that constructs new features, can preserve the interpretability of original attributes, and has become the primary concern for efficient management of multi-label data.

[0003] Current multi-label feature selection methods attempt to construct a mapping relationship between training samples and label sets to support new multi-label set prediction, and embedded methods can integrate feature selection into model optimization, but still have obvious defects, such as incomplete use of label information, insufficient anti-interference ability, not handling fuzzy label noise, and not fully preserving the local manifold structure of features, etc., which is difficult to meet the demand for accurate and efficient feature selection in multi-label learning. Therefore, there is an urgent need for a feature selection method that can fully utilize label information, enhance anti-interference ability, and consider efficiency. SUMMARY

[0004] The purpose of the present application is to solve the problems of high-dimensional feature redundancy, insufficient use of single-label information, and weak anti-interference ability in multi-label learning, and to provide a double-channel label guided multi-label feature selection method and system.

[0005] The purpose of the present application is achieved by the following technical solutions:

[0006] In a first aspect, a double-channel label guided multi-label feature selection method is provided, comprising the following steps:

[0007] S1. Data preprocessing: obtaining a feature matrix and a positive label matrix of a multi-label data set, generating a mirror negative label matrix by performing logical negation on the positive label matrix, and constructing a graph Laplacian matrix based on the feature matrix;

[0008] S2. Model construction: based on the pre-processed data, a multi-label model is constructed, and the objective function of the multi-label model includes at least:

[0009] Positive label regression loss term: used to constrain the error between the feature matrix after being mapped by the first projection matrix and the positive label matrix;

[0010] Negative label regression loss term: used to constrain the error between the feature matrix after being mapped by the second projection matrix and the mirrored negative label matrix;

[0011] Label alignment constraint term: used to constrain the approximation degree of the sum of the mapping results of the first projection matrix and the second projection matrix to the all-1 matrix;

[0012] Graph regularization term: the difference between the first projection matrix and the second projection matrix is used to alleviate the interference of ambiguous labels, and the local popular structure of the features is reserved by combining the graph Laplacian matrix;

[0013] Sparse constraint term: the first projection matrix and the second projection matrix are constrained;

[0014] S3. Model optimization solution: by relaxing the constraints and combining the Lagrange multiplier method to construct an optimization function, the objective function is iteratively solved according to the KKT condition until convergence to obtain the optimal first projection matrix and second projection matrix;

[0015] S4. Feature selection: the norm of each row of the optimal first projection matrix is calculated, all features are sorted according to the norm from large to small, and the feature subset composed of the features with non-zero norm is output.

[0016] In some embodiments, the graph Laplacian matrix is constructed based on the feature matrix, including:

[0017] The cosine similarity between the features is calculated based on the feature matrix, and the graph Laplacian matrix is constructed by combining the all-1 matrix.

[0018] In some embodiments, the objective function is specifically:

[0019]

[0020] wherein, the first projection matrix is represented by W, the second projection matrix is represented by H, the feature matrix is represented by X, the positive label matrix is represented by Y, the mirrored negative label matrix is represented by Y, the graph Laplacian matrix is represented by L, the Frobenius norm is represented by ||·||F, the matrix trace is represented by tr(·), the positive label regression loss term is represented by Lr, denotes a negative label regression loss term, denotes a label alignment constraint term, is a full 1 matrix of the same dimension as is a full 1 matrix of the same dimension as is a graph regularization term, is a sparsity constraint term, is a hyperparameter balancing each constraint term.

[0021] In some embodiments, step S3 specifically comprises:

[0022] relaxing to , where is a diagonal matrix;

[0023] non-negative constraints W ≥ 0 and ≥ 0 are incorporated into the objective function by Lagrange multipliers to construct an unconstrained Lagrangian function;

[0024] partial derivatives of the Lagrangian function with respect to W and are solved, and the multiplication update rules of W and are derived in combination with the KKT condition;

[0025] After initializing W and , iterative updates are performed according to the multiplication update rules, and the diagonal matrix D is updated after each iteration until the change in the value of the objective function is less than a preset threshold.

[0026] In some embodiments, the Top-k features with non-zero norms or the features with norms greater than a preset threshold are selected in step S4 to form the optimal feature subset, and the index of the optimal feature subset is output.

[0027] In a second aspect, a dual-channel label-guided multi-label feature selection system is provided, comprising:

[0028] A data preprocessing module is configured to obtain a feature matrix and a positive label matrix of a multi-label data set, generate a mirror negative label matrix by performing logical negation on the positive label matrix, and construct a graph Laplacian matrix based on the feature matrix.

[0029] A model construction module is configured to construct a multi-label model based on the preprocessed data, and the objective function of the multi-label model at least includes:

[0030] a positive label regression loss term: used to constrain the error between the feature matrix after being mapped by a first projection matrix and the positive label matrix;

[0031] Negative label regression loss term: used to constrain the error between the feature matrix and its mirror image negative label matrix after being mapped by the second projection matrix;

[0032] Label alignment constraint: Used to constrain how closely the sum of the mapping results of the first projection matrix and the second projection matrix approximates the all-one matrix;

[0033] Graph regularization term: The difference between the first projection matrix and the second projection matrix is ​​used to alleviate the interference of blurred labels, and the local popular structure of features is preserved by combining the graph Laplacian matrix;

[0034] Sparse constraint term: Apply constraints to the first projection matrix and the second projection matrix;

[0035] The model optimization and solution module is used to construct an optimization function by relaxing constraints and combining it with the Lagrange multiplier method, and then iteratively solve the objective function according to the KKT conditions until convergence is obtained to obtain the optimal first projection matrix and second projection matrix.

[0036] The feature selection module is used to calculate the norm of each row of the optimal first projection matrix, sort all features in descending order of norm, and select features with non-zero norms to form the optimal feature subset for output.

[0037] It should be further noted that the technical features corresponding to the above-mentioned embodiments can be combined or substituted with each other to form new technical solutions without conflict.

[0038] Compared with the prior art, the beneficial effects of the present invention are:

[0039] This invention generates a mirror negative label matrix by performing logical inversion on the positive label matrix, constructing a "positive-negative" dual-channel label input. This overcomes the limitation of traditional methods that rely solely on positive labels, comprehensively capturing the complex relationship between features and labels. Simultaneously, it integrates multiple constraint optimization objectives, including regression loss, label alignment, graph regularization, and sparse constraints, combined with an efficient iterative strategy to solve the projection matrix. Finally, based on the quantified feature importance of the projection matrix's row norm, a core feature subset is selected. This subset can be directly adapted to mainstream multi-label classification models such as SVM and MLKNN, significantly improving classification performance while reducing data dimensionality. It is particularly suitable for high-dimensional multi-label datasets and exhibits strong robustness to hyperparameters, stably handling the multi-label data processing needs of different fields. Attached Figure Description

[0040] Figure 1 This is a flowchart of a dual-channel label-guided multi-label feature selection method according to the present invention. Detailed Implementation

[0041] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] It should be noted that the defects in the solutions in the prior art are all the results of the inventors' practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed by the embodiments of this application in the following text should be the inventors' contributions to this application in the process of invention and creation, and should not be understood as technical content known to those skilled in the art.

[0043] Based on the above statements, and since this embodiment involves technologies related to label feature selection, the following explanations are provided to clarify the objectives, technical solutions, and advantages of this application embodiment:

[0044] 1. Multi-label learning is a supervised learning paradigm where a sample can belong to multiple sets of class labels simultaneously, rather than just a single class. Compared to traditional single-label learning, its fundamental challenge lies in modeling the dependencies and correlations between labels, thereby improving the ability to represent complex target spaces. Typical applications include image annotation, text classification, and gene function prediction.

[0045] 2. Feature Matrix () is the input representation of multi-labeled data, formally defined as ,in Let be the number of samples, and d be the feature dimension. Each row of the matrix corresponds to the feature vector of a sample, which is the core object for subsequent feature selection and mapping modeling.

[0046] 3. Label Matrix , The matrix is ​​divided into a positive label matrix and a mirror image of the negative label matrix. The positive label matrix... :in Indicates sample Belongs to the label Otherwise, it is 0. Mirror negative label matrix : Through the The elements are inverted one by one to obtain the supplementary information that the sample does not belong to a certain label, which is the key carrier of dual-channel information fusion.

[0047] 4. Projection Matrix This is used to implement a linear mapping from the feature space to the label space, in the form of: .in Describe the mapping relationship between features and positive labels. The mapping relationship between descriptive features and mirror negative labels, and its sparse structure directly determines the set of discriminative features selected.

[0048] 5. Graph Laplacian matrix ( ) is an important tool for manifold regularization, used to preserve the local geometric structure of samples in the feature space. Its construction method is as follows: ,in Let L be the adjacency matrix and S be the similarity matrix (usually calculated from the cosine similarity of the sample feature vectors). The regularization constraint of L can suppress label noise and preserve the local manifold information of the features.

[0049] 6. Norms, including the Frobenius Norm. ): Used to measure the overall error of a matrix, defined as the square root of the sum of the squares of all elements of the matrix, and is often used to measure the deviation between the predicted mapping and the true label.

[0050] 7. The Karush–Kuhn–Tucker Conditions are one of the necessary conditions for obtaining the optimal solution to a constrained optimization problem, and are widely used in non-smooth and convex optimization problems. The core idea is to transform the original optimization problem into a system of equations that satisfies optimality, feasibility, and complementary relaxation conditions by introducing Lagrange multipliers.

[0051] In view of the technical problems pointed out in the background art, the present invention provides the following embodiments:

[0052] In one exemplary embodiment, a dual-channel label-guided multi-label feature selection method is provided, such as... Figure 1 As shown, it includes the following steps:

[0053] S1. Data preprocessing: Obtain the feature matrix and positive label matrix of the multi-label dataset, generate a mirror negative label matrix by performing logical inversion on the positive label matrix, and construct a graph Laplacian matrix based on the feature matrix;

[0054] S2. Model Construction: A multi-label model is constructed based on the preprocessed data. The objective function of the multi-label model includes at least the following:

[0055] Positive label regression loss term: used to constrain the error between the feature matrix and the positive label matrix after mapping through the first projection matrix;

[0056] Negative label regression loss term: used to constrain the error between the feature matrix and its mirror image negative label matrix after being mapped by the second projection matrix;

[0057] Label alignment constraint: Used to constrain how closely the sum of the mapping results of the first projection matrix and the second projection matrix approximates the all-one matrix;

[0058] Graph regularization term: The difference between the first projection matrix and the second projection matrix is ​​used to alleviate the interference of blurred labels, and the local popular structure of features is preserved by combining the graph Laplacian matrix;

[0059] Sparse constraint term: Apply constraints to the first projection matrix and the second projection matrix;

[0060] S3. Model optimization solution: By relaxing the constraints and combining the Lagrange multiplier method to construct the optimization function, the objective function is then iteratively solved according to the KKT conditions until convergence is obtained to obtain the optimal first projection matrix and second projection matrix;

[0061] S4. Feature Selection: Calculate the norm of each row of the optimal first projection matrix, sort all features in descending order of norm, and select features with non-zero norms to form the optimal feature subset for output.

[0062] The principle of this invention is as follows: Addressing the problems of existing multi-label feature selection methods that rely on positive labels, ignore effective information from negative labels, have weak resistance to fuzzy label interference, and fail to preserve local feature structure, this invention constructs a mirrored negative label matrix. Integrating positive and negative label information, a design is implemented that includes a regression loss term, a label alignment constraint term, a graph regularization term, and... - The objective function of the norm sparse constraint term is solved by combining an iterative optimization strategy based on KKT conditions to obtain the projection matrix. and Ultimately based on The row vector norm is used to select the optimal feature subset, which can more comprehensively capture feature-label associations and improve feature discriminativeness. It can be effectively applied to multi-label learning scenarios such as text classification and image annotation.

[0063] First, in step S1, the basic data for the multi-label learning task is prepared. A feature matrix containing n samples and d features, and a positive label matrix containing l labels are obtained. By performing a logical inversion on the positive label matrix, a mirrored negative label matrix is ​​generated, forming a dual-channel input of positive and negative labels. Simultaneously, the cosine similarity between features is calculated based on the feature matrix, and a graph Laplacian matrix is ​​constructed by combining it with the all-one matrix, providing a foundation for the subsequent local manifold structure that preserves the features.

[0064] The data preprocessing stage constructs a dual-channel input and a foundation matrix. This stage is a prerequisite for technology implementation; its core is to transform the raw multi-label data into a structured input of "features + dual labels" and to calculate the key matrix used to preserve the feature structure, laying the data foundation for subsequent model construction. Specifically, the data preprocessing stage mainly consists of three parts: raw data input and format definition, construction of the mirror negative label matrix, and calculation of the graph Laplacian matrix L.

[0065] 1. Raw data input and format definition

[0066] First, obtain the core data for the multi-label learning task, clarify the data dimensions and physical meaning, and ensure that the data format meets the model calculation requirements:

[0067] Feature matrix : Input high-dimensional feature data of n samples and construct a feature matrix Where n represents the number of samples (e.g., the number of documents in text classification, the number of images in image annotation), and d represents the feature dimension of each sample (e.g., the number of bag-of-words features in text, the number of pixels or convolutional features in an image), the matrix... This represents the j-th feature value of the i-th sample.

[0068] Positive label matrix : Input the multi-label annotation results of the input samples and construct the positive label matrix. , where l represents the total number of tags (such as the number of tags like sports and military in text classification). This indicates that the i-th sample clearly belongs to the j-th label. This indicates that the i-th sample does not belong to the j-th label. Here... It is not noise, but contains accurate information that is unrelated to the sample and the label, providing a basis for subsequent use of negative labels.

[0069] 2. Mirrored negative label matrix Build: Activate dual-channel label information

[0070] Existing methods rely solely on positive labels Y Ignore non-candidate labels To address the deficiency in effective information, this technology uses logical inversion to convert negative labels into directly usable positive information, forming a dual-channel input of "positive label + negative label":

[0071] Positive label matrix Y Perform a logical inversion operation on each element to generate a mirrored negative label matrix. The specific calculation rules are as follows: For example, if (If sample i belongs to label j), then ;like (If sample i does not belong to label j), then .

[0072] The core value of this operation lies in transforming the negative information of "the sample does not belong to a certain label" into the positive information of "the sample belongs to a certain negative label," enabling the model to simultaneously transform the information from "belonging to" (…). ) and "not belonging to" It captures the relationship between features and labels from two dimensions, avoiding the one-sidedness of single-label spatial information.

[0073] 3. Graph Laplace Matrix Calculation: Preserving the characteristic local manifold structure

[0074] To mitigate noise interference from fuzzy labels and preserve the intrinsic structure of the feature space, it is necessary to base it on the feature matrix. Constructing the Graph Laplace Matrix The specific steps are as follows:

[0075] First, calculate the similarity matrix. , the feature matrix The feature is split into rows (each row represents all sample values ​​for one feature), and the cosine distance between any two feature row vectors is calculated to measure the similarity between features, thus constructing a similarity matrix. ,in This represents the similarity between the i-th feature and the j-th feature. The higher the similarity, the better. The larger the value.

[0076] For a matrix of all ones A , building and S A matrix of all ones of the same dimension ,Right now (for all) ).

[0077] Generating the Graph Laplacian Matrix L ,pass The Graph Laplacian matrix is ​​calculated. . L The core function is to preserve the local manifold structure of the feature space (i.e., to maintain consistency in the projection results of similar features) by constraining the feature projection matrix in subsequent optimization, thereby reducing the interference of fuzzy labels on feature selection.

[0078] In step S2, the parameters and variables for model optimization are initialized. Four regularization parameters are set to balance the negative label regression loss, adjust the label alignment constraint strength, determine the weight of the graph regularization term, and control the feature sparsity constraint strength (the optimal values ​​will be determined later through grid search). Two projection matrices are initialized, one to establish the mapping relationship between features and positive labels, and the other to establish the mapping relationship between features and negative labels. The iteration counter is initialized to 0 to prepare for subsequent iterations. A multi-objective collaborative optimization objective function is constructed. This function contains four core constraints: first, the regression loss constraint, ensuring that the two projection matrices can accurately associate features with positive and negative labels respectively; second, the label alignment constraint, making the sum of the results after the features are mapped by the two projection matrices approximate a matrix of all 1s, which not only conforms to the inversion logic of positive and negative labels but also denoises the original positive labels; third, the graph regularization constraint, using the difference between the two projection matrices to alleviate the interference of blurred labels, and combining with the previously constructed graph Laplacian matrix to preserve the local structure of features; and fourth, the feature sparsity constraint, ensuring the row sparsity of the projection matrix through a specific norm, laying the foundation for subsequent selection of core features. At the same time, non-negativity constraints are added to the two projection matrices to ensure that the optimization results have practical significance.

[0079] This step integrates four constraints—regression loss, label alignment, structure preservation, and feature sparsity—to construct a multi-objective optimization function. This ensures that the model, while learning the feature-label mapping, selects a subset of features with strong discriminative power and low redundancy. For example, this step specifically includes:

[0080] 1. Overall form of the objective function

[0081] The objective function of this technique is guided by "minimizing error + maximizing discriminability", and the specific formula is as follows (including non-negativity constraints):

[0082]

[0083] in, Let represent the first projection matrix, which is the projection matrix from the features to the positive labels. Measure the first The feature is related to the first (Discrimination weight of each positive label) The second projection matrix represents the projection matrix from the features to the mirror negative labels. (Measure the discriminative weight of the i-th feature for the j-th negative label) Represents the characteristic matrix, Represents the positive label matrix. Represents a mirrored negative label matrix. Represents the graph Laplace matrix. For the Frobenius norm, For the trace of the matrix, This represents the loss term in positive-label regression. This represents the negative-labeled regression loss term. This indicates label alignment constraints. To and A matrix of all ones of the same dimension For graph regularization terms, For sparse constraint terms, To balance the hyperparameters of each constraint term.

[0084] 2. Functions and significance of each constraint item

[0085] (1) Regression loss term: to ensure the accuracy of feature-label mapping

[0086] Positive label regression loss: Used to measure the feature matrix Through projection matrix The result mapped to the positive label space is the same as the true positive label matrix. The error is to make As close as possible This ensures that features can be accurately associated with positive labels.

[0087] Negative label regression loss: Consistent with the logic of positive label loss, it measures... pass The result of mapping to the negative label space and The error, Weights used to balance the losses of positive and negative labels (to prevent the loss of a certain label space from being ignored).

[0088] (2) Label alignment constraint: ensures the spatial logical consistency of the two labels.

[0089] This item is ,in To and A matrix of all ones of the same dimension, its core logic is based on Construction rules (Positive label mapping results) and The sum of (negative label mapping results) should approximate all 1.

[0090] Label alignment constraints serve a dual purpose: ensuring that the mapping results between the positive and negative label spaces conform to "inversion logic," thus avoiding contradictions. Simultaneously, through... For the original positive label Denoising (because) Approaching , Can be corrected (fuzzy annotations in the text).

[0091] (3) Graph regularization term: reducing noise and preserving feature structure

[0092] This item is Combined with the Graph Laplace matrix Difference from projection matrix It achieves two main functions: the first function is to alleviate the interference of fuzzy labels. If there is fuzzy labeling in a certain label (such as the labeling of sample i to label j is uncertain). and The difference will decrease, and this item will be constrained. and The association can magnify the clear label. The difference thus suppresses the influence of vague labels. The second function is to preserve the local structure of features. This contains feature similarity information, which is obtained through... Ensure similar features The values ​​should be similar to avoid damaging the feature structure.

[0093] (4) Sparse constraint terms: Filtering core features

[0094] This item is ,use -norm (for) Each line of the query After finding the norm, then calculate the norm for all rows. Norms, whose property is that they can induce The row sparsity allows unimportant features to correspond to... When the row vector norm approaches 0, the important features correspond to... The row vector norm is non-zero, thus enabling feature selection. It is used to balance sparsity and regression loss (to avoid excessive sparsity leading to increased mapping error).

[0095] (5) Nonnegativity constraint:

[0096] Based on the physical meaning of the non-negative (0 or 1) label values ​​and the projection weights, constraints are made. and All elements are non-negative, ensuring that the optimization results are interpretable.

[0097] In step S3, since the objective function contains non-smoothness... The norm and non-negativity constraints prevent direct solutions. This technique relaxes the constraints and uses the Lagrange multiplier method to construct the optimization function. The non-negativity constraint of the projection matrix is ​​incorporated into the objective function, creating an unconstrained Lagrange function. This transforms the constrained optimization problem into an unconstrained minimization problem. The solution is then iteratively applied based on the KKT conditions, progressively optimizing the solution. and The specific steps are as follows:

[0098] 1. Relaxation of non-smooth terms: Transform into a solvable smooth problem.

[0099] against - The nonsmoothness of the norm will relaxation ,in Given a diagonal matrix, its i-th diagonal element... ( (The denominator should be a very small positive number, avoiding a denominator of 0).

[0100] Relaxation principle: Through the dynamic update of D, Approximation in iterations At the same time, the objective function is transformed into a smooth function, which facilitates subsequent differentiation.

[0101] 2. Construct the Lagrangian function, incorporating nonnegativity constraints.

[0102] Non-negativity constraints W ≥0 and ≥0 is incorporated into the objective function using Lagrange multipliers to construct an unconstrained Lagrangian function:

[0103]

[0104] Among them Ψ and These are Lagrange multiplier matrices, respectively corresponding to and nonnegativity constraint, and To constrain penalty items.

[0105] 3. Partial derivative calculation and application of KKT conditions: Derivation of the update formula

[0106] Find the partial derivatives: with respect to... and Find the partial derivatives of the Lagrange function and set them to 0 (condition for optimal solution).

[0107] right Partial derivative:

[0108]

[0109] right Partial derivative:

[0110]

[0111] 4. Simplification of KKT conditions: Based on KKT conditions (the optimal solution satisfies...) , (i.e., constraints and variables are complementary), eliminating the Lagrange multipliers, and deriving the result. and Iterative update formula:

[0112] Updated formula:

[0113]

[0114] Updated formula:

[0115]

[0116] 5. Iterative Convergence Determination: After each iteration, the iteration counter is updated, and the difference in the objective function before and after the iteration is calculated. If the difference is less than a preset threshold, the model is considered to have converged, and the iteration stops. If it has not converged, the update steps of the projection matrix and the relaxation diagonal matrix will continue to be repeated.

[0117] For example, the iteration count is initialized to 0, and the initialization is random. and Iteratively update according to the above formula , The diagonal matrix D is updated synchronously. Then, the difference in the objective function before and after the iteration is calculated. If the difference is less than a preset threshold, the iteration stops. and The solution converges to the optimal solution; otherwise, the iteration continues.

[0118] Finally, feature selection and result output are completed. After iterative convergence, the importance of features is evaluated based on the first projection matrix W used to associate features. The norm of each row of the projection matrix is ​​calculated. The larger the norm, the stronger the comprehensive discrimination ability of the corresponding feature for positive and negative labels. All features are sorted from largest to smallest norm, and features with non-zero norms are selected to form the optimal feature subset. Finally, the feature index of this subset is output.

[0119] For example, in step S4, based on the converged first projection matrix W Features with strong discriminative power are selected to provide input for subsequent multi-label model training, specifically including:

[0120] 1. Feature importance ranking

[0121] because The row vectors correspond to the discriminative weights of a single feature for all labels. (where the weight vector is the i-th feature), calculate Each line norm ( The larger the norm, the stronger the feature's overall discriminative power across all labels. The features are sorted from largest to smallest to obtain a ranking of feature importance.

[0122] 2. Optimal Feature Subset Selection and Output

[0123] Based on actual needs, select the top-ranked features with non-zero norms to form the optimal feature subset (features with zero norms can be discarded directly because they have no discriminative contribution to the labels). Output the index of the optimal feature subset (e.g., the 3rd, 5th, 12th... features) and the corresponding feature matrix (only the columns of the selected features are retained). This subset can be directly used for training multi-label classifiers such as SVM and MLKNN.

[0124] In another exemplary embodiment, based on the same inventive concept as the method embodiment, a dual-channel label-guided multi-label feature selection system is provided, comprising:

[0125] The data preprocessing module is used to obtain the feature matrix and positive label matrix of the multi-label dataset, generate a mirror negative label matrix by performing logical inversion on the positive label matrix, and construct a graph Laplacian matrix based on the feature matrix.

[0126] The model building module is used to construct a multi-label model based on the preprocessed data. The objective function of the multi-label model includes at least the following:

[0127] Positive label regression loss term: used to constrain the error between the feature matrix and the positive label matrix after mapping through the first projection matrix;

[0128] Negative label regression loss term: used to constrain the error between the feature matrix and its mirror image negative label matrix after being mapped by the second projection matrix;

[0129] Label alignment constraint: Used to constrain how closely the sum of the mapping results of the first projection matrix and the second projection matrix approximates the all-one matrix;

[0130] Graph regularization term: The difference between the first projection matrix and the second projection matrix is ​​used to alleviate the interference of blurred labels, and the local popular structure of features is preserved by combining the graph Laplacian matrix;

[0131] Sparse constraint term: Apply constraints to the first projection matrix and the second projection matrix;

[0132] The model optimization and solution module is used to construct an optimization function by relaxing constraints and combining it with the Lagrange multiplier method, and then iteratively solve the objective function according to the KKT conditions until convergence is obtained to obtain the optimal first projection matrix and second projection matrix.

[0133] The feature selection module is used to calculate the norm of each row of the optimal first projection matrix, sort all features in descending order of norm, and select features with non-zero norms to form the optimal feature subset for output.

[0134] The above detailed embodiments are a description of the present invention. It should not be considered that the specific embodiments of the present invention are limited to these descriptions. For those skilled in the art, several simple deductions and substitutions can be made without departing from the concept of the present invention, and all of these should be considered to fall within the protection scope of the present invention.

Claims

1. A dual-channel label-guided multi-label feature selection method applied to text classification, characterized in that, Includes the following steps: S1. Data Preprocessing: Obtain the feature matrix and positive label matrix of the multi-label dataset in text classification. Generate a mirror negative label matrix by performing a logical inversion on the positive label matrix, and construct a graph Laplacian matrix based on the feature matrix. The feature matrix is ​​as follows: Where n represents the number of samples, including the number of documents in text classification, and d represents the feature dimension of each sample, including the number of bag-of-words features of the text in text classification; The positive label matrix is Where l represents the total number of tags, including the number of tags in the text category; S2. Model Construction: A multi-label model is constructed based on the preprocessed data. The objective function of the multi-label model includes at least the following: Positive label regression loss term: used to constrain the error between the feature matrix and the positive label matrix after mapping through the first projection matrix; Negative label regression loss term: used to constrain the error between the feature matrix and its mirror image negative label matrix after being mapped by the second projection matrix; Label alignment constraint: Used to constrain how closely the sum of the mapping results of the first projection matrix and the second projection matrix approximates the all-one matrix; Graph regularization term: The difference between the first projection matrix and the second projection matrix is ​​used to alleviate the interference of blurred labels, and the local popular structure of features is preserved by combining the graph Laplacian matrix; Sparse constraint term: Apply constraints to the first projection matrix and the second projection matrix; S3. Model optimization solution: By relaxing the constraints and combining the Lagrange multiplier method to construct the optimization function, the objective function is then iteratively solved according to the KKT conditions until convergence is obtained to obtain the optimal first projection matrix and second projection matrix; S4. Feature Selection: Calculate the norm of each row of the optimal first projection matrix, sort all features in descending order of norm, and select features with non-zero norms to form the optimal feature subset for output.

2. The dual-channel label-guided multi-label feature selection method according to claim 1, characterized in that, The construction of the graph Laplacian matrix based on the feature matrix includes: The cosine similarity between features is calculated based on the feature matrix, and a graph Laplacian matrix is ​​constructed by combining the all-one matrix.

3. The dual-channel label-guided multi-label feature selection method according to claim 1, characterized in that, The objective function is specifically: ， in, Denotes the first projection matrix. Represents the second projection matrix. Represents the characteristic matrix, Represents the positive label matrix. Represents a mirrored negative label matrix. Represents the graph Laplace matrix. For the Frobenius norm, For matrix trace, This represents the loss term in positive-label regression. This represents the negative-labeled regression loss term. This indicates label alignment constraints. To and A matrix of all ones of the same dimension For graph regularization terms, For sparse constraint terms, To balance the hyperparameters of each constraint term.

4. The dual-channel label-guided multi-label feature selection method according to claim 3, characterized in that, Step S3 specifically includes: Will relaxation ,in It is a diagonal matrix; Non-negativity constraints W ≥0 and ≥0 is incorporated into the objective function through Lagrange multipliers to construct an unconstrained Lagrange function; The Lagrange function is respectively about W and Find the partial derivatives and, using the KKT conditions, derive the following: W and Multiplication update rules; initialization W and Then, iterative updates are performed according to the multiplication update rule, and the diagonal matrix is ​​updated after each iteration. D The iteration continues until the change in the objective function value is less than a preset threshold.

5. The dual-channel label-guided multi-label feature selection method according to claim 4, characterized in that, In step S4, the top-k features with non-zero norm or features with a norm greater than a preset threshold are selected to form the optimal feature subset, and the index of the optimal feature subset is output.

6. A dual-channel label-guided multi-label feature selection system, applied to text classification, characterized in that, include: The data preprocessing module is used to obtain the feature matrix and positive label matrix of the multi-label dataset in text classification, generate a mirror negative label matrix by performing a logical inversion on the positive label matrix, and construct a graph Laplacian matrix based on the feature matrix; the feature matrix is ​​as follows: Where n represents the number of samples, including the number of documents in text classification, and d represents the feature dimension of each sample, including the number of bag-of-words features of the text in text classification; The positive label matrix is Where l represents the total number of tags, including the number of tags in the text category; The model building module is used to construct a multi-label model based on the preprocessed data. The objective function of the multi-label model includes at least the following: Positive label regression loss term: used to constrain the error between the feature matrix and the positive label matrix after mapping through the first projection matrix; Negative label regression loss term: used to constrain the error between the feature matrix and its mirror image negative label matrix after being mapped by the second projection matrix; Label alignment constraint: Used to constrain how closely the sum of the mapping results of the first projection matrix and the second projection matrix approximates the all-one matrix; Graph regularization term: The difference between the first projection matrix and the second projection matrix is ​​used to alleviate the interference of blurred labels, and the local popular structure of features is preserved by combining the graph Laplacian matrix; Sparse constraint term: Apply constraints to the first projection matrix and the second projection matrix; The model optimization and solution module is used to construct an optimization function by relaxing constraints and combining it with the Lagrange multiplier method, and then iteratively solve the objective function according to the KKT conditions until convergence is obtained to obtain the optimal first projection matrix and second projection matrix. The feature selection module is used to calculate the norm of each row of the optimal first projection matrix, sort all features in descending order of norm, and select features with non-zero norms to form the optimal feature subset for output.

Images