Sparse low-redundancy multi-label feature selection method with dynamic graph constraint

Through the combination of dynamic graph learning and L2,1 norms, the calculation error problem caused by fixed graph matrix in the prior art is solved, and more efficient multi-label feature selection is achieved, feature redundancy is eliminated, and the performance of the learning model is improved. stability.

CN119066388APending Publication Date: 2024-12-03SHAANXI AGRICULTURE & FORESTRY VOCATIONAL & TECHNICAL UNIVERSITY
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411094196.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-09
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The existing graph-based multi-label feature selection method uses a fixed graph matrix, which leads to the accumulation of computational errors of the learning model, limiting the performance of the learning model.

Method used

A sparse low redundant multi-label feature selection method with dynamic graph constraints is proposed. Through the combination of dynamic graph learning and L2,1 norms, the objective function is constructed to remove redundant information between features, and the cosine theorem and k- Nearest neighbor technology optimizes feature weight matrix.

Benefits of technology

A graph matrix is ​​more suitable for multi-label feature selection, effectively removing redundant information between features, improving the performance and stability of the learning model, and converging in multiple iterations compared with the existing methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119066388A_ABST
    Figure CN119066388A_ABST
Patent Text Reader

Abstract

The invention discloses a sparse low-redundancy multi-label feature selection method with dynamic graph constraint, which is characterized in that a sparse low-redundancy penalty term is designed by combining a dynamic graph learning technology of graph Laplacian rank constraint, a cosine theorem and an L2, 1 norm according to a graph learning mode, and is used for removing redundant information among features; according to the method, the effectiveness and superiority of the SLDMG are verified through a large number of contrast experiments compared with an existing advanced method. Compared with DSMFS, DMMFS and SLMDS of a dynamic graph in the prior art, the method has the advantages that a graph matrix more suitable for multi-label feature selection can be obtained by a dynamic graph learning technology in SLDMG; compared with RFSFS and SLDMS considering feature redundancy, the sparse low-redundancy penalty term designed in the SLDMG can better remove redundant information between the features.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of feature selection, and in particular to a sparse low-redundancy multi-label feature selection method with dynamic graph constraints. Background Art

[0002] Multi-label feature selection, that is, feature selection methods suitable for multi-label data, is one of the main technologies for high-dimensional data dimensionality reduction. Its purpose is to remove irrelevant and redundant features in the original feature space to improve the learning efficiency of the model and avoid the dimensionality disaster problem. In recent years, with the rapid increase of multi-label data in computer science, biomedicine and other fields, multi-label feature selection methods have received widespread attention and application in text classification, image classification, genetic data analysis and audio recognition.

[0003] In addition, since the Laplacian graph in manifold learning can effectively mine the supervision information of multi-label data, more and more advanced graph-based multi-label feature selection methods have been proposed. For example, some studies have proposed to use the instance similarity graph and the low-order similarity graph of the real label pair to jointly constrain the learning of pseudo labels, and combine L 2,1 Using the norm as a sparse constraint, MDFS (Manifold regularized discriminative feature selection for multi-label learning) was proposed. Similarly, DRMFS (Robust multi-label feature selection with dual-graph regularization) obtained a more accurate feature weight matrix by constraining the feature similarity graph and the low-dimensional similarity graph of the true label. In order to better explore the learning of pseudo-labels, some studies have proposed to use latent representation learning to design a pseudo-label learning technology with a shared structure between instances and true labels, and combined it with manifold learning based on instance manifold graphs to propose SSFS (Multi-label feature selection with constrained latent structure shared term). Subsequently, MRDM (Multi-label feature selection via manifold regularization and dependence maximization) constrained the learning of pseudo-labels by introducing the HSIC (Hilbert-Schmidt Independence Criterion) technology.

[0004] However, the graph matrices used in the above graph-based methods are fixed and do not change with the increase in iterations. These low-quality graph matrices may cause the computational errors of the learning model to accumulate due to uncorrected errors, thereby limiting the performance of the learning model. Summary of the invention

[0005] The present invention provides a sparse low-redundancy multi-label feature selection method with dynamic graph constraints, aiming to solve the above-mentioned problems existing in the background technology.

[0006] The technical solution provided by the present invention is as follows:

[0007] A sparse low-redundancy multi-label feature selection method with dynamic graph constraints, comprising the following steps:

[0008] S1. Select a label matrix Y and an instance matrix X, and determine a mapping function F between the instance matrix X and the label matrix Y. The expression of the mapping function F is as follows:

[0009]

[0010] Where X∈R n×d , n and d represent the number of instances and feature dimensions respectively; X i* is the i-th row vector of X, representing the i-th instance; Y∈R n×m , m represents the number of label categories; Y ij is the i-th row and j-th column element of Y, Y ij =1 indicates instance X i* Belongs to the jth class; W∈R d×m is the feature weight matrix; W *j is the j-th column vector of W;

[0011] S2. Incorporating a bias vector into the mapping function F and applying constraints to W to obtain a multi-label feature selection model. The expression of the multi-label feature selection model is:

[0012]

[0013] In the formula, 1 n is an n-dimensional column vector with all elements being 1, b is an m-dimensional bias row vector, and R(W) is the penalty term for W;

[0014] S3, according to the multi-label feature selection model, dynamic graph learning is performed to remove redundant information in W and construct an objective function, the expression of which is:

[0015]

[0016]

[0017] Where Tr(·) is the trace operation of the matrix; Q∈R d×d is the cosine similarity matrix between features; S∈R d×d is a dynamic similarity graph between features, S ij is the i-th row and j-th column element of S; L S =P S -S is the Laplacian matrix of S; rank(L S ) indicates L S Rank; S0∈R d×d is a fixed feature similarity graph; α, β and γ are regularization parameters;

[0018] Step S4: Optimize and solve the objective function, construct a sparse low-redundancy multi-label feature selection algorithm with dynamic graph constraints, and complete the sparse low-redundancy multi-label feature selection with dynamic graph constraints.

[0019] Furthermore, in step S3, the dynamic graph learning is specifically performed by using the L1 norm between the fixed feature similarity graph to obtain a dynamic graph learning expression of the Laplace rank constraint, and the dynamic graph learning expression is as follows:

[0020]

[0021] Where S∈R d×d is a dynamic similarity graph between features, S ij is the i-th row and j-th column element of S, representing the i-th feature X *i and the jth feature X *j The similarity between S =P S -S is the Laplacian matrix of S, P S is a diagonal matrix, rank(L S ) indicates L S Rank; S0∈R d×d is a fixed feature similarity graph;

[0022] S0 is calculated by the following formula:

[0023]

[0024] Where η is the regularization parameter; (S0) ij is the element in the i-th row and j-th column of S0.

[0025] Furthermore, in step S3, removing redundant information in W and constructing the objective function includes the following steps:

[0026] S3.1. Use the cosine theorem to construct a penalty term for W to avoid the drawbacks caused by inner product operations. The formula is as follows:

[0027]

[0028] Where Tr(·) is the trace operation of the matrix; Q∈R d×d is the cosine similarity matrix between features;

[0029] S3.2, combined with k-nearest neighbor technology, if X *i Yes X *j k-nearest neighbors or X *j Yes X *i The k-nearest neighbors of , the calculation formula for Q is as follows:

[0030]

[0031] On the contrary, Q ij =0;

[0032] In the formula, Q ij is the i-th row and j-th column element of Q;

[0033] S3.3, Introduction of L 2,1 The norm is used as a sparse constraint, and the learning expression of the sparse low-redundancy penalty constraint W is obtained as follows:

[0034]

[0035] In the formula, Represents L about W 2,1 norm;

[0036] S3.4. Combine the expression of the multi-label feature selection model, the dynamic graph learning expression and the learning expression of the sparse low-redundancy penalty term constraint W to obtain the expression of the objective function.

[0037] Furthermore, in step S4, the objective function is optimized and solved, including the following steps:

[0038] S4.1. Fix S and W, and obtain the derivative function with respect to b through the expression of the objective function:

[0039]

[0040] make The update formula of the objective function with respect to b is:

[0041]

[0042] S4.2, fix b and W, and obtain the objective function about S through the expression of the objective function:

[0043]

[0044] S4.3, the objective function of S is split and solved for different j, and its vector form is:

[0045]

[0046] Where V∈R d×d is the Euclidean distance matrix about W, V *j is the j-th column vector of V, D∈R d ×d is a diagonal matrix,

[0047] Simplify the vector form obtained after splitting and solving, and the simplified expression is as follows:

[0048]

[0049] In the formula,

[0050] S4.4, fix b and S, and get the objective function about W as:

[0051] F(W)=minTr[(XW-Y) T H(XW-Y)]+

[0052] βTr(W T L S W)+γ(Tr(W T QW)+Tr(W T UW)

[0053] stW ≥ 0,

[0054] In the formula, is the center matrix, E∈R n×n is the n-dimensional identity matrix; U∈R d×d is a diagonal matrix,

[0055] S4.5. According to the objective function about W, the Lagrangian function of the objective function about W is obtained and solved:

[0056]

[0057] In the formula, is the Lagrange multiplier; find the derivative with respect to W and set it to 0.

[0058] Furthermore, in step S4, in the sparse low-redundancy multi-label feature selection algorithm with dynamic graph constraints, the instance matrix X, the label matrix Y, and the regularization parameter group (α, β, γ) are input, and the feature ranking result F is output. S .

[0059] Furthermore, the sparse low-redundancy multi-label feature selection algorithm with dynamic graph constraints converges within 20 iterations.

[0060] Compared with the prior art, the present invention has the following beneficial effects:

[0061] The present invention provides a sparse low-redundancy multi-label feature selection method with dynamic graph constraints. The method combines the dynamic graph learning technology of graph Laplace rank constraint with the cosine theorem and L 2,1 Norm designed a sparse low-redundancy penalty term to remove redundant information between features. This method verified the effectiveness and superiority of SLDMG through a large number of comparative experiments with existing advanced methods. Compared with DSMFS, DMMFS and SLMDS in the existing technology that use dynamic graphs, the dynamic graph learning technology in SLDMG can obtain a graph matrix that is more suitable for multi-label feature selection; compared with RFSFS and SLDMS that consider feature redundancy, the sparse low-redundancy penalty term designed in SLDMG can better remove redundant information between features. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is a flowchart of a sparse low-redundancy multi-label feature selection method with dynamic graph constraints in an embodiment of the present invention;

[0063] Figure 2 The experimental result diagram of the Bonferroni-Dunn test (α=0.1) in the form of an average rank diagram in the embodiment of the present invention is shown, and the horizontal axis represents the ranking result of the overall performance;

[0064] Figure 3 This is a graph showing the results of a convergence experiment in an embodiment of the present invention;

[0065] Figure 4 This is a graph of experimental results under the Hamming loss indicator in an embodiment of the present invention;

[0066] Figure 5 This is a graph of experimental results under the One-error indicator in an embodiment of the present invention;

[0067] Figure 6 This is a graph of experimental results under the Average precision indicator in an embodiment of the present invention;

[0068] Figure 7 This is a graph of experimental results under the Macro-F1 indicator in an embodiment of the present invention;

[0069] Figure 8 This is a graph of experimental results under the Micro-F1 indicator in an embodiment of the present invention. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the embodiments described below are part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0071] Therefore, the detailed description of the embodiments of the present application provided below in conjunction with the accompanying drawings is intended to represent only selected embodiments of the present application, and is not intended to limit the scope of protection claimed in the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0072] like Figure 1 As shown, the present invention provides a sparse low-redundancy multi-label feature selection method with dynamic graph constraints, comprising the following steps:

[0073] S1. Select a label matrix Y and an instance matrix X, and determine a mapping function F between the instance matrix X and the label matrix Y. The expression of the mapping function F is as follows:

[0074]

[0075] (1) In the formula, X∈R n×d , n and d represent the number of instances and feature dimensions respectively; X i* is the i-th row vector of X, representing the i-th instance; Y∈R n×m , m represents the number of label categories; Y ij is the i-th row and j-th column element of Y, Y ij =1 indicates instance X i* Belongs to the jth class; W∈R d×m is the feature weight matrix; W *j is the j-th column vector of W;

[0076] represents the Frobenius norm of an arbitrary matrix M.

[0077] S2. Incorporating a bias vector into the mapping function F and applying constraints to W to obtain a multi-label feature selection model. The expression of the multi-label feature selection model is:

[0078]

[0079] (3) In the formula, 1 n is an n-dimensional column vector whose elements are all 1, b is an m-dimensional bias row vector, and R(W) is the penalty term for W.

[0080] Considering that the binary nature of Y in step S1 will affect the accuracy of linear mapping, this problem is usually addressed by incorporating a bias vector into the linear mapping. Thus:

[0081]

[0082] (2) In the formula, 1 n is an n-dimensional column vector whose elements are all 1. b is an m-dimensional bias row vector.

[0083] For the multi-label feature selection model, it is also necessary to impose certain constraints on W to improve the learning performance of the model, such as sparse constraints, manifold regularization, etc. Thus, the linear framework of multi-label feature selection can be obtained as shown in formula (3).

[0084] S3, according to the multi-label feature selection model, dynamic graph learning is performed to remove redundant information in W and construct an objective function, the expression of which is:

[0085]

[0086]

[0087] Where Tr(·) is the trace operation of the matrix; Q∈R d×d is the cosine similarity matrix between features; X *i Yes X *j k-nearest neighbors or X *j Yes X *i k-nearest neighbors; S∈R d×d is a dynamic similarity graph between features, S ij is the i-th row and j-th column element of S; L S =P S -S is the Laplacian matrix of S; rank(L S ) indicates L S Rank; S0∈R d×d is a fixed feature similarity graph; α, β and γ are regularization parameters.

[0088] Step S4: Optimize and solve the objective function, construct a sparse low-redundancy multi-label feature selection algorithm with dynamic graph constraints, and complete the sparse low-redundancy multi-label feature selection with dynamic graph constraints.

[0089] Optionally, in step S3, the dynamic graph learning is specifically performed by using the L1 norm between the fixed feature similarity graph and obtaining a dynamic graph learning expression of the Laplace rank constraint, and the dynamic graph learning expression is as follows:

[0090]

[0091] (4) In the formula, S∈R d×d is a dynamic similarity graph between features, S ij is the i-th row and j-th column element of S, representing the i-th feature X *i and the jth feature X *j The similarity between S =P S -S is the Laplacian matrix of S, P S is a diagonal matrix, rank(L S ) indicates L S Rank; S0∈R d×d is a fixed feature similarity graph;

[0092] S0 is calculated by the following formula:

[0093]

[0094] (5) where η is the regularization parameter; (S0) ij is the element in the i-th row and j-th column of S0.

[0095] Optionally, in step S3, removing redundant information in W and constructing an objective function comprises the following steps:

[0096] S3.1. Use the cosine theorem to construct a penalty term for W to avoid the drawbacks caused by inner product operations. The formula is as follows:

[0097]

[0098] (6) where Tr(·) is the trace operation of the matrix; Q∈R d×d It is the cosine similarity matrix between features, which can effectively quantify the degree of redundancy between features.

[0099] S3.2, combined with k-nearest neighbor technology, if X *i Yes X *j k-nearest neighbors or X *j Yes X *iThe k-nearest neighbors of , the calculation formula for Q is as follows:

[0100]

[0101] On the contrary, Q ij =0;

[0102] (7) In the formula, Q ij is the i-th row and j-th column element of Q;

[0103] S3.3, Introduction of L 2,1 The norm is used as a sparse constraint, and the learning expression of the sparse low-redundancy penalty constraint W is obtained as follows:

[0104]

[0105] (8) In the formula, Represents L about W 2,1 Norm.

[0106] Considering that sparse representation technology reduces the computational cost, L 2,1 norm as a sparse constraint. On the one hand, L 2,1 The intra-row stability of the norm can be well promoted with formula (7), thereby better removing the redundant information in W. On the other hand, L 2,1 The inter-row sparsity of the norm can better fit the subsequent feature selection strategy.

[0107] S3.4, combining equations (3), (4) and (8), the objective function of the proposed method SLDMG is constructed:

[0108]

[0109] Where α and γ are regularization parameters.

[0110] In addition, considering that the rank constraint problem in the above formula is difficult to solve directly, according to KyFan's theorem, the approximate solution of the rank constraint can be obtained by the following formula:

[0111]

[0112] In the formula, σ i (L S ) indicates L S The i-th smallest eigenvalue of S is a positive semidefinite matrix, so σ i (L S ) is naturally non-negative. In addition, in order to ensure the non-negativity of S, we constrain W to be a non-negative matrix. In summary, the objective function of SLDMG can be restated as:

[0113]

[0114] Where β is the regularization parameter.

[0115] It can be seen from formula (11) that the objective function of SLDMG consists of a convex function about b, a non-convex function about S, and a non-convex function about W. Therefore, the next step is to optimize and solve formula (11) through an alternating iterative strategy.

[0116] First, when S and W are fixed, the derivative of equation (11) with respect to b is:

[0117]

[0118] make The update formula of the objective function with respect to b can be obtained:

[0119]

[0120] Secondly, when b and W are fixed, the objective function of equation (11) with respect to S is:

[0121]

[0122] Considering that the above equation is independent of each other for different j, and S is a symmetric matrix. Therefore, the above equation can be split and solved for different j, and its vector form can be written as:

[0123]

[0124] Where V∈R d×d is the Euclidean distance matrix about W, V *j is the j-th column vector of V, D∈R d ×d is a diagonal matrix,

[0125] The above formula can be simplified as:

[0126]

[0127] In the formula,

[0128] Integrating the constraints in the above formula into the objective function, we can obtain the Lagrangian function of the above formula for solution:

[0129]

[0130] In the formula, μ and ψ ≥ 0 are both Lagrange multipliers. Calculate the above formula about Si j The derivative function of , and set the derivative function equal to zero:

[0131] αD ii S ij -P ij -μ-ψ ij =0. (18)

[0132] According to KKT (Karush Kuhn Tucker) conditions and S ij ≥0, we know ψ ij = 0. Thus, we can get a j The update formula is:

[0133]

[0134] According to the above formula, a piecewise monotone increasing function about μ is defined to solve μ:

[0135]

[0136] Regarding the above formula, Newton iteration can be used to solve it, and the optimal value of is substituted into formula (19) to obtain the optimal solution of formula (19).

[0137] Finally, when b and S are fixed, according to equation (13), ||W|| 2,1 =Tr(W T UW) and The objective function of formula (11) with respect to W is:

[0138] F(W)=minTr[(XW-Y) T H(XW-Y)]+

[0139] βTr(W T L S W)+γ(Tr(W T QW)+Tr(W T UW)

[0140] stW≥0,(21)

[0141] In the formula, is the center matrix, E∈R n×n is the n-dimensional identity matrix. U∈R d×d is a diagonal matrix,

[0142] Integrating the constraints in the above formula into the objective function, we can obtain the Lagrangian function of the above formula for solution:

[0143]

[0144] In the formula, is the Lagrange multiplier. Find the derivative of the above formula with respect to W and set the derivative to 0. Then, according to the KKT condition and non-negative matrix decomposition technology, we can get the update formula of the objective function with respect to W:

[0145]

[0146] Based on the optimization solution of the above model, a sparse and low-redundancy multi-label feature selection algorithm with dynamic graph constraints was designed.

[0147] The sparse and low-redundancy multi-label feature selection with dynamic graph constraints algorithm is shown in Algorithm 1:

[0148] Algorithm 1 SLDMG algorithm.

[0149] Input instance matrix X, label matrix Y, regularization parameter group (α, β, γ).

[0150] Output feature sorting result FS.

[0151] 1. Calculate S0 according to formula (5).

[0152] 2. Initialize the center matrix H.

[0153] 3. Initialize the number of iterations t=0.

[0154] 4. Initialize L 2,1 Norm diagonal matrix U t is the identity matrix.

[0155] 5. Randomly initialize the weight matrix W t and the dynamic graph matrix S t .

[0156] 6. According to calculate and

[0157] 7. Repetition:

[0158] Update W:

[0159]

[0160] According to equation (20), μ is calculated using Newton iteration.

[0161] Update S:

[0162] Update b according to formula (13).

[0163] Update U:

[0164] t=t+1

[0165] t=t+1.

[0166] 8. Until convergence.

[0167] 9. Calculate and sort || W i* ||2, assign the sorting result to F S .

[0168] As can be seen from Algorithm 1, Algorithm 1 mainly consists of matrix operations, maximum value comparison and summation operations, and mainly updates W, S, U, b and μ. According to the optimization solution in Section 2, the time complexity of Algorithm 1 to update W in each iteration is O(nd 2 ), the time complexity of updating S is O(d 2 ), the time complexity of updating U is O(d 2 ), the time complexity of updating b is O(nm), and the time complexity of updating μ is O(d 2 ). Therefore, the time complexity of each iteration of Algorithm 1 is O(nd 2 +d 2 +dm). In addition, Algorithm 1 generally converges within 20 iterations (this conclusion is discussed below). Figure 1 So the total time complexity of Algorithm 1 is O(nd 2 +d 2 +dm).

[0169] In this application, a theoretical proof of the convergence of Algorithm 1 is also given. First, from formula (11), we can see that F(W, S, b) is a convex function of b and S. Therefore, in the tth iteration of Algorithm 1, we have:

[0170] F(W t ,S t+1 ,b t+1 )≤F(W t ,S t ,b t ). (twenty four)

[0171] Secondly, F(W,S,b) is a non-convex function with respect to W. Therefore, before proving the convergence of Algorithm 1 with respect to W, we need to understand some related concepts.

[0172] Definition 1: If there is g1(W,W t )≥g2(W) and g1(W,W)=g2(W), then g1(W,W t ) is an auxiliary function of g2(W). t is the updated value of W in the previous iteration.

[0173] Lemma 1: If g1(W,W t ) is an auxiliary function of g2(W), then g2(W) is The form of is monotonically decreasing. Among them, W t+1 is the optimal solution for the tth iteration.

[0174] Proposition 1: If M1∈R m×m and M2∈R d×d is a symmetric matrix, then for any matrix W∈R d×m and W t ∈R d×m , the following inequality holds:

[0175]

[0176] According to formula (11), g2(W) is defined as:

[0177]

[0178] In the formula, G1 = X T HX and G2 = X T HY are symmetric matrices. G3 is the m-dimensional identity matrix.

[0179] According to the above formula, g1(W,W t ) is defined as:

[0180]

[0181] From Proposition 1, we know that for equations (26) and (27), we have:

[0182]

[0183] From the calculation mechanism of matrix trace, we can know that for equations (26) and (27), we have:

[0184]

[0185] Combining equation (28) and equation (29), we can get:

[0186] g1(W,W t )≥g2(W). (30)

[0187] When W t=W, equation (28) can be restated as:

[0188]

[0189] From equations (29) and (31), we can see that when W t =W, g1(W,W) = g2(W). Combining with equation (30), we can prove that g1(W,W t ) is an auxiliary function of g2(W). Finally, according to Lemma 1, it can be proved that Algorithm 1 converges with respect to W.

[0190] In summary, the convergence of Algorithm 1 is proved.

[0191] Example 1

[0192] This example obtains 8 classic multi-label datasets from the Mulan library (http: / / mulan.sourceforge.net / datasets.html) as experimental data, and compares the method provided by the present invention with 8 multi-label feature selection methods, including the classic method SCLS based on information theory, the advanced method G3WI based on information theory, and the advanced graph-based methods SSFS, RFSFS, DRMFS, DSMFS, DMMFS and SLMD. In addition, ML-KNN is used as a representative of the multi-label classification algorithm in the experiment, and the performance of each feature selection method is evaluated using five multi-label evaluation indicators: Hamming loss, One-error, Average precision, Macro-F1 and Micro-F1.

[0193] 1.1 Dataset and Experimental Details

[0194] Table 1 Dataset information

[0195]

[0196]

[0197] In the experiment, 8 multi-label experimental datasets come from 3 different fields, including: biological dataset PlantGo; text datasets Enron, Computer, Recreation and Social; image datasets Scene, Image and Flags. The specific properties of all datasets are shown in Table 1.

[0198] In terms of experimental settings: First, the programming software Matlab R2016a is used in the experiment to reproduce all algorithms on a Microsoft Windows 11 system with a processor: ADM Ryzen 53600 6-Core Processor 3.59 GHz.

[0199] Secondly, for the ML-KNN classifier, its parameters are set to the default values, that is, the smoothing parameter is 1 and the neighbor parameter is 10. In order to facilitate the operation of SCLS and G3WI, the data set is discretized using equal-width intervals in the experiment. For all graph-based feature selection methods, the default techniques of each method are used to learn the graph matrix in the experiment, and the neighbor parameter of graph learning is set to 5. As shown in Table 1, the number of labels in the Image dataset is only 5, so the label neighbor parameter of the Image dataset is set to 3. In addition, the "grid search" strategy is used in the experiment to search the regularization parameters of all methods in the range of [0.001, 0.01, 0.1, 1, 10, 100, 1000].

[0200] Finally, it can be seen from Table 1 that the number of features of the Flags dataset is only 19, so the feature selection norm of Flags is set to 2 to 18, and the feature selection range of the other 7 experimental datasets is set to the first 2% to 20% of the instance features, with a step size of 2%.

[0201] 1.2 Results Analysis

[0202] Tables 2-6 show the mean of the five-fold cross-validation experimental results of all experimental methods under the five multi-label indicators of Hamming loss, One-error, Averageprecision, Macro-F1 and Micro-F1. For the convenience of comparison, the best results and sub-optimal results on the same dataset are marked in bold and underlined in Tables 2-6. In addition, the last two rows of Tables 2-6 show the mean of the results of each algorithm on all data and the average ranking results of the overall performance of each algorithm using the Bonferroni-Dunn test with a significance level of α=0.1.

[0203] In Tables 2-6, "↓" indicates that the smaller the index value, the better the algorithm performance. On the contrary, "↑" indicates that the larger the index value, the better the algorithm performance. As can be seen from Tables 2-6, the performance of SLDMG on the Enron dataset is slightly inferior to DRMFS and DSMFS under the One-error and Micro-F1 indicators; the performance of SLDMG on the Recreation dataset is slightly inferior to DSMFS under the Macro-F1 indicator, but in other comparison cases, the performance of SLDMG is the best. In addition, the overall performance and the average ranking of the overall performance of SLDMG are always the best under all indicators. In particular, compared with SCLS, G3WI, SSFS, RFSFS, DRMFS, DSMFS, DMMFS and SLMDS, the overall performance of SLDMG under the One-error indicator is improved by 22.79%, 14.84%, 16.35%, 14.67%, 4.91%, 6.88%, 5.66% and 13.50%, respectively; the overall performance of SLDMG under the Macro-F1 indicator is improved by 27.92%, 17.83%, 27.12%, 20.76%, 6.62%, 9.97%, 4.73% and 16.64%, respectively.

[0204] Table 2 Comparison of Hammingloss (↓) of different algorithms under different data sets

[0205]

[0206] Table 3 Comparison of One-error (↓) of different algorithms under different data sets

[0207]

[0208] Table 4 Comparison of Average precision (↑) of different algorithms under different data sets

[0209]

[0210]

[0211] Table 5 Comparison of Macro-F1(↑) of different algorithms under different data sets

[0212]

[0213] Table 6 Comparison of Micro-F1(↑) of different algorithms under different data sets

[0214]

[0215] In addition, in order to intuitively observe the overall performance comparison of each algorithm under the same indicator, Figure 2The experimental results of the Bonferroni-Dunn test (α = 0.1) are shown in the form of an average rank plot. Figure 2 As shown in the figure, the horizontal axis represents the ranking result of the overall performance, and the overall performance of the algorithms is getting better from right to left.

[0216] In addition, the length of the red line indicates the critical value CD of the Bonferroni-Dunn test. Among them, q α =2.855, K is the number of experimental algorithms, N is the number of experimental data. There is no significant difference between the methods connected by the red line. Figure 2 It can be seen that although the overall performance of SLDMG is the best in all indicators, and there are significant differences between SLDMG and SCLS, SSFS, G3WI, RFSFS and SLMDS, there is no significant difference between SLDMG and DRMFS, DSMFS and DMMFS.

[0217] Convergence experiments were conducted on the Flags, Image, Scene, and Social datasets. The experimental results are shown in Figure 3 As shown in the figure, the horizontal axis represents the number of iterations of the algorithm, and the vertical axis represents the objective function value. It can be seen that the proposed algorithm converges on all experimental data sets, and the convergence speed is very fast, generally reaching convergence within 20 iterations. This phenomenon also occurs on other data sets. In addition, Figure 3 This also confirms the accuracy of the above convergence proof.

[0218] In order to further explore the differences between SLDMG and the comparison algorithms, the Friedman test with a significance level of 0.05 was used to analyze the differences of the experimental algorithms. The experimental results and critical values ​​of the Friedman test are shown in Table 7. It can be seen that the Friedman test results on all indicators are much larger than the critical values, which indicates that the original hypothesis (the performance of all methods is the same) is rejected. Figure 3 The results in verify that the overall performance of SLDMG in all indicators is better than that of all the compared methods, and further prove that SLDMG has certain advantages in feature selection compared with the existing advanced methods.

[0219] Table 7 Friedman test results

[0220]

[0221] In order to intuitively compare the performance of each algorithm when selecting different feature numbers, Figure 3-Figure 7The performance curves of each algorithm as the number of selected features increases are plotted in Figure 1. The horizontal axis represents the number of selected features, and the vertical axis represents the experimental results under the corresponding indicators. It can be seen that as the number of selected features increases, the performance curves of SLDMG on most data sets are in the optimal position, which shows that SLDMG is effective in dealing with multi-label feature selection problems and has certain advantages. In addition, by observing Figure 4-Figure 8 From the performance curves of the algorithms on the Flags dataset, we can find that with the increase in the number of selected features, the performance of the proposed method SLDMG on all indicators first increases and then slowly decreases. This phenomenon shows that the feature subset selected by SLDMG on the Flags dataset is effective and can effectively remove irrelevant and redundant features.

[0222] In summary, through the above-mentioned large number of comparative experiments and statistical analysis, it can be seen that compared with DSMFS, DMMFS and SLMDS that use dynamic graphs, the dynamic graph learning technology in SLDMG can obtain a graph matrix that is more suitable for multi-label feature selection; compared with RFSFS and SLDMS that consider feature redundancy, the sparse low-redundancy penalty item designed in SLDMG can better remove redundant information between features.

[0223] The above is only the best specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A sparse low-redundancy multi-label feature selection method with dynamic graph constraints, characterized in that: The following steps are involved: S1. Select a label matrix Y and an instance matrix X, and determine a mapping function F from the instance matrix X to the label matrix Y. The expression of the mapping function F is as follows: Where X∈R n×d , n and d represent the number of instances and feature dimensions respectively; X i* is the i-th row vector of X, representing the i-th instance; Y∈R n×m , m represents the number of label categories; Y ij is the i-th row and j-th column element of Y, Y ij =1 indicates instance X i* Belongs to the jth class; W∈R d ×m is the feature weight matrix; W *j is the j-th column vector of W; S2. Incorporating a bias vector into the mapping function F and applying constraints to W to obtain a multi-label feature selection model. The expression of the multi-label feature selection model is: In the formula, 1 n is an n-dimensional column vector with all elements being 1, b is an m-dimensional bias row vector, and R(W) is the penalty term for W; S3, according to the multi-label feature selection model, dynamic graph learning is performed to remove redundant information in W and construct an objective function, the expression of which is: Where Tr(·) is the trace operation of the matrix; Q∈R d×d is the cosine similarity matrix between features; S∈R d×d is a dynamic similarity graph between features, S ij is the i-th row and j-th column element of S; L S =P S -S is the Laplacian matrix of S; rank(L S ) indicates L S Rank; S0∈R d×d is a fixed feature similarity graph; α, β and γ are regularization parameters; Step S4: Optimize and solve the objective function, construct a sparse low-redundancy multi-label feature selection algorithm with dynamic graph constraints, and complete the sparse low-redundancy multi-label feature selection with dynamic graph constraints.

2. The sparse low-redundancy multi-label feature selection method with dynamic graph constraints according to claim 1, characterized in that: In step S3, the dynamic graph learning is specifically performed by using the L1 norm between the fixed feature similarity graph to obtain the dynamic graph learning expression of the Laplace rank constraint, and the dynamic graph learning expression is as follows: Where S∈R d×d is a dynamic similarity graph between features, S ij is the i-th row and j-th column element of S, representing the i-th feature X *i and the jth feature X *j The similarity between S =P S -S is the Laplacian matrix of S, P S is a diagonal matrix, rank(L S ) indicates L S Rank; S0∈R d×d is a fixed feature similarity graph; S0 is calculated by the following formula: Where η is the regularization parameter; (S0) ij is the element in row i and column j of S0.

3. The sparse low-redundancy multi-label feature selection method with dynamic graph constraints according to claim 2, characterized in that: In step S3, removing redundant information in W and constructing an objective function includes the following steps: S3.

1. Use the cosine theorem to construct a penalty term for W to avoid the drawbacks caused by inner product operations. The formula is as follows: Where Tr(·) is the trace operation of the matrix; Q∈R d×d is the cosine similarity matrix between features; S3.2, combined with k-nearest neighbor technology, if X *i Yes X *j k-nearest neighbors or X *j Yes X *i The k-nearest neighbors of , the calculation formula for Q is as follows: On the contrary, Q ij =0; In the formula, Q ij is the i-th row and j-th column element of Q; S3.3, Introduction of L 2,1 The norm is used as a sparse constraint, and the learning expression of the sparse low-redundancy penalty constraint W is obtained as follows: In the formula, Represents L about W 2,1 norm; S3.

4. Combine the expression of the multi-label feature selection model, the dynamic graph learning expression and the learning expression of the sparse low-redundancy penalty term constraint W to obtain the expression of the objective function.

4. The sparse low-redundancy multi-label feature selection method with dynamic graph constraints according to any one of claims 1 to 3, characterized in that: In step S4, the objective function is optimized and solved, including the following steps: S4.

1. Fix S and W, and obtain the derivative function with respect to b through the expression of the objective function: make The update formula of the objective function with respect to b is: S4.2, fix b and W, and obtain the objective function about S through the expression of the objective function: S4.3, the objective function of S is split and solved for different j, and its vector form is: Where V∈R d×d is the Euclidean distance matrix about W, V *j is the j-th column vector of V, D∈R d×d is a diagonal matrix, Simplify the vector form obtained after splitting and solving, and the simplified expression is as follows: In the formula, S4.4, fix b and S, and get the objective function about W as: F(W)=minTr[(XW-Y) T H(XW-Y)]+ βTr(W T L S In)+γ(Tr(In T QW)+Tr(W T UW)) stW≥0, In the formula, is the center matrix, E∈R n×n is the n-dimensional identity matrix; U∈R d×d is a diagonal matrix, S4.

5. According to the objective function about W, the Lagrangian function of the objective function about W is obtained and solved: In the formula, is the Lagrange multiplier; find the derivative with respect to W and set it to 0.

5. The sparse low-redundancy multi-label feature selection method with dynamic graph constraints according to claim 4, characterized in that: In step S4, in the sparse low-redundancy multi-label feature selection algorithm with dynamic graph constraints, the instance matrix X, the label matrix Y, and the regularization parameter group (α, β, γ) are input, and the feature ranking result F is output. S .

6. The sparse low-redundancy multi-label feature selection method with dynamic graph constraints according to claim 5, characterized in that: The sparse low-redundancy multi-label feature selection algorithm with dynamic graph constraints converges within 20 iterations.

Citation Information

Cited By

  • Multi-label learning behavior feature selection method and system based on shared structure

    CN122571031A