A text classification method based on sparse feature selection

Abstract

The application belongs to the technical field of data classification, and particularly relates to a text classification method based on sparse feature selection. The method is characterized in that: sample matrix and label matrix are input into a target function for iterative updating, the target function preselects a feature subset according to the sample matrix and the label matrix; the correlation between two labels is determined according to a local feature correlation function and a high-order label information function; and the iterative updating process is stopped when a set stop rule is reached, and the feature subset output according to the set order. The method combines the characteristics of high-order labels, fully utilizes and mines the hidden information behind the labels, and explicitly indicates the correlation between two labels, thereby accurately reflecting the correlation between local features, improving the classification performance of multi-label data, and improving the accuracy of the feature selection result.

Description

Technical Field

[0001] This invention belongs to the field of data classification technology, specifically relating to a text classification method based on sparse feature selection. Background Technology

[0002] Multi-label classification can solve real-world tasks with rich semantics. Specifically, in multi-label classification, an instance may be associated with multiple labels. For example, an image may be associated with a set of labels, and a news article may belong to multiple topics. Compared to traditional single-label classification problems, multi-label classification is a more challenging task due to the enormous size of the output space. Research shows that high-dimensional data with a large amount of redundancy and noise is widespread in real life. The "curse of dimensionality" caused by high-dimensional data degrades model performance. Therefore, researchers have proposed two dimensionality reduction techniques to address this problem: feature extraction and feature selection. Feature extraction projects samples from a high-dimensional output space to a low-dimensional space through linear or nonlinear mapping to obtain a new set of features. Obviously, while this type of method reduces the dimensionality of the samples, it loses the original semantic information of the features and cannot effectively remove irrelevant and redundant features. Feature selection effectively solves this problem. Feature selection refers to using computational methods to select some features with good discriminative properties from the original feature set according to evaluation criteria for classification. Its purpose is to select the smallest subset of features based on certain criteria, so that classification and other tasks achieve results similar to or even better than before feature selection. Therefore, feature selection is more interpretable and meaningful, and has been widely applied in various fields such as bioinformatics, medicine, materials chemistry, and economics.

[0003] Most feature selection methods are primarily designed for single-label learning. However, with the increasing popularity of multi-label learning, more and more scholars are focusing on multi-label problems. Currently, many feature selection algorithms for multi-label data have been proposed: For example, the paper "Multi-label feature selection based on label correlations and feature redundancy" by Fan et al. uses l in the regularization part of the loss function. 2,1Norms are used to select features. Braytee et al.'s "Multi-label feature selection using correlation information" decomposes the feature and label space into a low-dimensional space for multi-label feature selection (CMFS). Wang et al.'s "A survey of sparse representation: Algorithms and applications" filters label-specific features by controlling the sparsity of the coefficient matrix and applies low-rank constraints to the label matrix to mine local correlations between class labels. However, the label matrix in multi-label learning is usually a full-rank matrix, which cannot be well approximated by low-rank decomposition. Moreover, using this method in the latent space can make label correlations indirect and semantically unclear. In real-world scenarios, for most tasks, only a small portion of the original features are needed to train the model, while most features are irrelevant or redundant. Melo and Paulheim's "Local and global feature selection for multilabel classification with binary relevance" compares the effects of local and global feature selection. Experimental results show that local feature selection not only outperforms global feature selection during the feature selection process but also performs better in terms of runtime performance. Therefore, local feature correlation should be considered more in feature selection. For example, the Multi-label Informed Feature Selection (MIFS) framework by Jian et al. projects label information into a low-dimensional space, giving it a local geometric structure similar to the feature space. Huang et al.'s Manifold-based constraint Laplacian score for multi-label feature selection proposed a feature selection method (MCLS) based on manifold constraints and Laplacian scores, projecting the binary label space into the digit label space and then constraining the similarity between instances through corresponding digit labels. However, these methods only obtain label correlation and local feature correlation through similarity metrics, which fails to accurately reflect the relationships between local features. Therefore, in existing techniques for studying the hidden information behind labels, using low-rank matrix factorization may destroy the rank of the high-order label matrix, thus failing to utilize some useful information; furthermore, studying global feature correlation is not easy. Summary of the Invention

[0004] The purpose of this invention is to provide a text classification method based on sparse feature selection, in order to solve the problem that existing feature selection methods cannot accurately reflect the correlation between local features, resulting in inaccurate feature selection results.

[0005] To address the aforementioned technical problems, this invention provides a text classification method based on sparse feature selection, comprising the following steps:

[0006] 1) Input the sample matrix and label matrix into the objective function for iterative updates. The objective function pre-selects a feature subset based on the sample matrix and label matrix; and determines the correlation between two labels based on the local feature correlation function and the higher-order label information function.

[0007] 2) When the set stopping rule is reached, stop the iterative update process, output the feature subsets according to the set sorting, select the first set number of subsets in the output feature subsets to form a new subset, and use the new subset as the final feature subset.

[0008] Its beneficial effects are as follows: The method of the present invention, based on l 2,1 The regularized loss function explores the potential relationship between features and labels and selects a discriminative subset of features. By combining the characteristics of higher-order labels, it fully utilizes and mines the hidden information behind the labels and explicitly points out the correlation between two labels. Furthermore, it combines manifold learning with Laplacian scores as a method to process local features, better utilize the correlation between local features, and thus accurately reflect the correlation between local features, improving the accuracy of feature selection results.

[0009] Further, in step 1), the objective function is based on l 2,1 The norm-based regularized loss function explores the relationship between labels and features.

[0010] The method of the present invention takes into account that l 2,1 Regularization can enhance the sparsity of the feature selection matrix in the rows; therefore, by employing l-based regularization... 2,1 The norm-based regularized loss function not only considers the interpretability and constraints of the model, but also explores the potential relationship between features and labels, thus making the feature selection matrix sparser and easier to solve.

[0011] Furthermore, based on l 2,1 The regularization loss function of the norm is: Where W is the eigenvalue matrix; ||XW-Y|| F Denotes the Frobenius norm; λ is a hyperparameter; ||W|| 2,1 l represents W 2,1 Norm.

[0012] Furthermore, in step 1), the local feature correlation function is established based on manifold constraints and Laplace fractions.

[0013] Furthermore, the function established based on the Laplace score in the local feature correlation function is as follows:

[0014] , where Lp = (L + L T ) / 2, L = G - Q R For the graph Laplace matrix, Q is a diagonal matrix; R ∈R n×n For example, a k-nearest neighbor graph instance with respect to Q; Represents the trace of a matrix.

[0015] Furthermore, the function established based on manifold constraints in the local feature correlation function is:

[0016]

[0017] in, For the graph Laplace matrix, It is a diagonal matrix.

[0018] Further, in step 1), the higher-order tag information function is: Where Y is the label matrix; E n H represents a column vector consisting entirely of 1s; H is the label-related information matrix; P is the deviation regression coefficient matrix; and λ is a hyperparameter.

[0019] Further, in step 1), the objective function is:

[0020]

[0021] Where α, β, γ, and λ represent different hyperparameters, and Y is the label matrix; E n H represents a column vector consisting entirely of 1s; P represents the label-related information matrix; W represents the deviation regression coefficient matrix; and X represents the characteristic coefficient matrix.

[0022] Furthermore, the values ​​of the three matrices W, H, and P in the objective function are updated iteratively, and the process stops when the set iteration conditions are met or the maximum number of iterations is reached.

[0023] Furthermore, the set iteration condition is met when the difference between two consecutive function values ​​is less than a preset value. Attached Figure Description

[0024] Figures 1a-1fThis is a comparison chart of the AP of the method of this invention and existing methods on six different datasets;

[0025] Figures 2a-2f This is a comparison chart of the CV of the method of this invention and existing methods on six different datasets;

[0026] Figures 3a-3f This is a comparison chart of the RL method of this invention and existing methods on six different datasets;

[0027] Figures 4a-4f This is a comparison chart of the HL of the method of this invention and existing methods on six different datasets;

[0028] Figures 5a-5f This is a comparison chart of the MA of the method of this invention and existing methods on six different datasets;

[0029] Figures 6a-6f This is a comparison chart of the MI between the method of this invention and existing methods on six different datasets;

[0030] Figure 7 This is a flowchart of the text classification method based on sparse feature selection according to the present invention. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0032] Example of a text classification method based on sparse feature selection:

[0033] To address the issues of model interpretability and constraint, the method in this embodiment employs l 2,1 The norm-regularized loss function performs sparse operations on the feature coefficient matrix, establishing a connection between labels and features. This facilitates the determination of potential relationships between features and labels, making the feature coefficient matrix sparser and easier to solve. Furthermore, considering the full-rank characteristic of most label matrices, this embodiment uses a self-representation strategy to explicitly indicate the correlation between two labels without compromising the matrix rank. In addition, to better utilize local feature correlations, manifold learning and Laplacian scores are combined as methods for processing local features, allowing for the selection of different features for the true label and individual labels. This embodiment also employs an iterative optimization process to improve the classification accuracy of its multi-label feature selection method.

[0034] The specific process of the text classification method based on sparse feature selection in this embodiment is as follows: Figure 7 As shown:

[0035] 1) Adopting l-based 2,1The norm-based regularization loss function determines the potential relationship between features and labels.

[0036] This implementation takes into account the partial non-differentiability of the l1 norm for feature selection methods, and while l2 norm regularization has discriminative capabilities, it is not conducive to obtaining strictly sparse solutions. Therefore, in order to select high-resolution feature subsets through sparse learning, this embodiment designs a new function, namely, based on l... 2,1 The norm-based regularization loss function is constructed as follows:

[0037] Define X = [x1, x2, ..., x] n ]∈R n×d Let be the sample matrix, where n is the number of instances and d is the number of features;

[0038] Define Y = {y1, y2, ..., y} n}∈{1,−1} n×m Let x be the label matrix, where for each sample x i y i It is the label vector relative to class m; if x i If it belongs to the j-th class, then y ij =1, otherwise y ij =−1. The loss function based on the l1 norm constraint is expressed as follows:

[0039] (1)

[0040] However, l 2,1 The solution obtained by the norm is more efficient than that obtained by the l1 norm, because l 2,1 Regularization can enhance the sparsity of the eigenvalue matrix in rows, i.e., w i It is a column vector that quantifies the contribution of the i-th feature to each label. Therefore, the l1 norm in equation (1) is changed to l 2,1 Norm sparsity regularization results in a new loss function as follows:

[0041] (2)

[0042] Where W∈R d×m The feature coefficient matrix represents the mapping relationship between the feature space and the labels; the "|| ||" in the above formula F The calculation method for “” is as follows: , denotes the Frobenius norm; λ is a hyperparameter; ||W|| 2,1 Indicate l 2,1 Norms, specifically represented as: .

[0043] This is based on l2,1 The norm-based regularized loss function not only considers the interpretability and constraints of the model, but also explores the potential relationship between features and labels, thus making the feature coefficient matrix sparser and easier to solve.

[0044] 2) Based on manifold learning and Laplace fractions, a functional expression of local feature structure is established. By effectively utilizing the correlation of local features, it provides guidance for exploring the underlying true label and selecting different features for a single label.

[0045] Compared to global correlation, local correlation is more beneficial to the classification accuracy of feature selection because if vector x i With vector x j If they are similar, the label matrices predicted by the two vectors may have the same points. According to the smoothness assumption, the local topology can be transferred from the feature space to the label space.

[0046] (3)

[0047] In equation (3), X = [x1, x2, ..., x n ]∈R n×d Let be the sample matrix, and “|| ||2” denotes the l2 norm.

[0048] In this embodiment, the local label information is processed as shown in equation (4).

[0049] (4)

[0050] In equation (4), f i It is a numeric label, and a logical label y. i In comparison, f i It can indicate the different importance of labels associated with the same sample and help to utilize label correlation.

[0051] In constraint scoring, label information is used as two pairs of constraints to guide graph construction. This embodiment's method considers that these constraints are not applicable to multi-label data; therefore, it implements guidance by selecting different features for the underlying true labels and individual labels through the following process.

[0052] Specifically define x i With x j The constraint affinity between them is shown in equation (5):

[0053] (5)

[0054] in, It represents the number of nearest neighbors. x i With x jThe similarity between them depends on the distance between them and the affinity between their tags. The latter (i.e., the affinity between tags) can be considered as x influenced by tag relevance. i With x j The strength of the connection between them. In the affinity matrix, x i With x j Higher similarity between them corresponds to a larger Q. ij .

[0055] The local attributes of the features are validated. Excellent features should be able to preserve the local structure of the data. Specifically, if two samples are similar or close, the feature should keep the two samples close to each other; otherwise, the feature should keep them far apart. Therefore, in this embodiment, the adjacency matrix is ​​defined as shown in equation (6):

[0056] (6)

[0057] Among them, Q R ∈R n×n Let N(x) represent the k-nearest neighbor graph instance with respect to Q. i ) indicates the xth i The set of nearest instances of an instance. Then, based on the local geometry, construct the corresponding mapping function f(x). i ) = x i W + y i H + P T Construct matrix Z as a set of functions, as shown in equation (7):

[0058] (7)

[0059] In equation (7), X∈R n×d Let W be the sample matrix; W ∈ R d×m The characteristic coefficient matrix; Y∈R n×m The label matrix; H∈R m ×m For the tag-related information matrix; E n ∈R n Represents a column vector consisting entirely of 1s; P∈R m P is the error vector; T Let P be the transpose of matrix P.

[0060] Based on the above formula, the final expression is constructed as shown in equation (8). When it is minimized, if two local features are similar, then the labels predicted by these two local features are also the same.

[0061] (8)

[0062] In equation (8), L = G - QR For the graph Laplace matrix, Let X be a diagonal matrix, X∈R n×d The sample matrix; Q R ∈R n×n Represents a k-nearest neighbor graph instance with respect to Q; The trace of the matrix is ​​represented by "|||2", which represents the l2 norm. In this embodiment, to more reasonably learn the characteristics of local geometric structures, the matrix L is symmetricized, i.e., L p =(L+L T ) / 2.

[0063] Finally, introducing the idea of ​​manifold learning on the basis of embedded feature selection can provide guidance for exploring potential true labels and selecting different features for individual labels. For example, if some instances have the same label, they may share a common subset of features closely related to that label. That is, the relevant feature subsets for different labels are different. Therefore, for feature selection, it is not necessary to process the global feature subset, but only to judge the local features and find the feature subsets belonging to different labels. Based on this, this embodiment introduces the idea of ​​manifold regularization learning related to local features to process the feature coefficient matrix W. The instance dataset X is divided into s groups by the K-means algorithm, i.e., {X1, ⋯, X c ,⋯,X s}∈X where X c ∈R c×d Let X represent the c-th sample group. c The label model of the medium sample is a discriminative feature equation as follows (9):

[0064] (9)

[0065] Among them, U c ∈R d×d Let c be the local feature discrimination correlation matrix of the c-th sample group. It represents the i-th column of the c-th sample; "|| ||2" represents the l2 norm.

[0066] Based on the above formula, s different local feature correlation matrices can be obtained. Then, these matrices are weighted according to the different sample groups, and the processed sample groups are merged, as shown in equation (10):

[0067] (10)

[0068] To ensure the symmetry of the matrix dimensions, U = U T × U. Finally, a manifold regularization term based on the local feature manifold of s is constructed, and its expression is shown in equation (11):

[0069] (11)

[0070] Among them, L ο = D - U is the graph Laplacian matrix, where It is a diagonal matrix.

[0071] 3) Combining the characteristics of higher-order label matrices, a self-representation strategy is used to fully utilize and mine the hidden information behind the labels, and to explicitly indicate the correlation between two labels.

[0072] In this embodiment, considering that effectively utilizing label information is an important step in the feature selection process, and facing the high-dimensional pressure of data, label information is processed based on the idea of ​​low-rank matrix decomposition, that is, the label matrix is ​​decomposed into a latent space. However, most label matrices have the characteristic of full rank. This is because although there is correlation among the labels, the related labels may also exist alone in some samples. Therefore, most label matrices are not low-rank. However, although the method of low-rank decomposition can obtain label information, for full-rank matrices, this method will destroy the rank of the label matrix, thereby destroying some information. Therefore, in this embodiment, while using the self-representation strategy to mine high-order label information, a penalty term is added to matrices H and P to avoid trivial solutions and overfitting. The function used to study high-order label information is as follows (12):

[0073] (12)

[0074] Where Y∈R n×m For the label matrix; E n ∈R n Represents a column vector consisting entirely of 1s; H∈R m×m This is a label-related information matrix, representing the correlation between higher-order labels; P∈R m Let be the deviation regression coefficient matrix, representing the deviation between the algorithm and the actual labels; λ is a hyperparameter; where Describing the Frobenius norm, This represents the l2 norm.

[0075] 4) Establish a multi-label feature selection algorithm to improve the classification performance of multi-label data.

[0076] Combining the above modules, the objective function SFSLH can be derived from the loss function in equation (2), the local feature correlation function in equations (8) and (11), and the expression in equation (12). Finally, the objective function of SFSLH is shown in equation (13):

[0077] (13)

[0078] Where α, β, γ, and λ represent different hyperparameters used to balance different terms. It is worth noting that SFSLH uses the Laplace regularization term of the graph to integrate the utilization of higher-order correlations and model prediction into a joint model, which can enhance both aspects simultaneously during the execution of the algorithm. Although not all variables in equation (13) are convex, each variable is convex, and the remaining non-convex variables can be fixed. Therefore, the optimal solution is sought through the following alternating optimization steps:

[0079] Update W: Fix H and P, solve W, and transform equation (13) into equation (14):

[0080] (14)

[0081] Taking the partial derivative of equation (14) with respect to W, we can obtain equation (15):

[0082] (15)

[0083] in V R d×d It is a diagonal matrix. This is a binary optimization problem. The solution tends to be optimal when the reciprocal is 0. Therefore, let... Therefore, we get the following equation (16):

[0084] (16)

[0085] Update H: Fix W and P, solve for H, and transform equation (13) into equation (17):

[0086] (17)

[0087] Taking the partial derivative of equation (17) with respect to H, since it is only a binary optimization problem, there exists an optimal solution when the partial derivative equals 0. Therefore, we can obtain... Solving for:

[0088] (18)

[0089] Where, I∈R m×m It is an identity matrix.

[0090] Update P: Fix W and H and solve P, transforming equation (13) into equation (19):

[0091] (19)

[0092] This is also a binary optimization problem. An optimal solution exists when the reciprocal is 0. Therefore, we can obtain:

[0093] (20)

[0094] Based on the above process, the SFSLH algorithm first randomly initializes the values ​​of W, H, and P, and then updates these three variables using an iterative optimization method. The iteration stops when the difference between two continuous function values ​​is less than 0.001 (i.e., the set iteration stopping condition is met). Finally, feature selection is performed based on the feature ranking obtained from the results. Table 1 below illustrates the SFSLH algorithm process in this embodiment:

[0095] Table 1

[0096]

[0097] As can be observed from the algorithm description, the ultimate goal of each iteration is to update the values ​​of the three matrices W, H, and P. Therefore, the time complexity of Algorithm 1 can be divided into three parts: First, the calculation of the eigenvalue matrix W, which requires the calculation of matrix L. p Therefore, the computational complexity of W is O(W). max (d 3 , m 3 , n 3 +n 2 d). Similarly, updating H and P requires matrix multiplication and inversion operations with time complexities of O(n log n). max (m 2 n, m 3 , dmn) and O max (m 3 , n 2 Therefore, the overall complexity of SFSLH is O(dmn). max (n 3 +nd, m 3 , dmn).

[0098] Experimental Verification: To verify the superiority of the feature selection method of this invention, this embodiment selected six multi-label text datasets from http: / / mulan.sourceforge.net / datasets.html to verify the effectiveness of the SSFSLH algorithm, based on experimental requirements. Table 2 shows the attributes of the six datasets. These datasets were combined into a corpus, and on this basis, test and training sets were divided to conduct experiments, and the experimental results were evaluated.

[0099] Table 2

[0100]

[0101] Comparative experiments were conducted between SSFSLH and six other methods: GRRO, MFS-MCDM, WFSNR, MFSR, MDFS, and MCLS. GRRO is a general multi-label learning framework based on global optimization and redundancy; MFS-MCDM is a multi-label feature selection method using multi-criteria decision-making, with parameters set to 0.1; WFSNR is a weakly labeled feature selection algorithm based on neighborhood rough sets and ReliefF; MFSR is a multi-label feature selection algorithm based on an improved ReliefF; MDFS is a discriminative feature selection method based on manifold regularization in multi-label learning, with parameters set to 1, 1, and 0.1; and MCLS is a multi-label feature selection method based on manifold constraints and Laplacian scores.

[0102] The classification results of the above methods and the SFSLH method, which uses a sorted subset of the top 100 features, are compared in an ML-KNN classification dataset. To quantify the performance of these multi-label feature selection methods, six multi-label learning metrics are used: Micro-average F1 (MI), Macro-average F1 (MA), Ranking Loss (RL), Hamming Loss (HL), Coverage (CV), and Average Precision (AP). Hamming Loss, Ranking Loss, and Coverage are better when they are closer to 0; Average Precision, Macro-average F1, and Micro-average F1 are better when they are closer to 1. In the experiments, "↑" indicates a larger value and "↓" indicates a smaller value and better performance for each evaluation criterion. The bold text in the table indicates the optimal performance value. These six evaluation metrics each have different focuses, therefore few feature selection methods can outperform other methods in all of these metrics. The comparison results of SFSLH with other methods on six datasets are shown in Table 3.

[0103] Table 3 shows the comparison results of SFSLH with other methods on 6 datasets.

[0104]

[0105]

[0106] Table 3 presents the experimental results of seven algorithms on the Business, Arts, Computers, Enron, Science, and Bookmarks datasets for text classification. On the Business, Arts, Enron, Science, and Bookmarks datasets, our algorithm achieves the best performance on six metrics: HL, RL, CV, AP, MI, and MA. On the Computers dataset, our algorithm achieves the best performance on HL, RL, CV, AP, and MA, while GRRO achieves the best performance on MI. However, SFSLH performs second only to GRRO on MI and significantly outperforms GRRO on other datasets. Although SFSLH's performance on MI is slightly inferior to other methods, it is still superior overall.

[0107] Based on the 36 comparison results in the table (6 datasets and 6 evaluation metrics), the present invention achieves optimal values ​​in over 97.2% of cases. The above experimental results clearly demonstrate that the classification performance induced by the feature subset obtained by the feature selection method of the present invention is significantly superior to other comparative algorithms.

[0108] Figures 1a-1f , Figures 2a-2f , Figures 3a-3f , Figures 4a-4f , Figures 5a-5f as well as Figures 6a-6f The experimental classification results of seven algorithms on six multi-label datasets are presented. For each graph, the X-axis (horizontal axis) represents the number of selected feature subsets, and the Y-axis (vertical axis) represents the performance of the evaluation metric. Figures 1a-6f (As shown in the diagram illustrating the experimental classification results of seven algorithms on six multi-label datasets) it can be seen that, for the three metrics of AP, CV, and RL, SFSLH significantly outperforms all other algorithms on the five text datasets. For the Computers dataset, SFSLH's performance on the MI metric is not outstanding, possibly due to the high redundancy of labels in this dataset. For the Business, Science, and Enron datasets, on the HL and MA metrics, SFSLH's advantage is not significant when the number of selected feature subsets is small; however, as the number of features increases, the SFSLH algorithm outperforms all other algorithms. Regarding the MI metric, on the Enron dataset, while SFSLH outperforms GRRO, MFS-MCDM, MCLS, MDFS, and WFSNR, it does not have a significant advantage over the MFSR method. For other datasets, SFSLH shows a good advantage. Overall, from... Figures 1a-6f It can be concluded that the method of the present invention is superior to the other six advanced multi-label feature selection methods.

[0109] In this embodiment, a multi-label feature selection method considering high-order label information and local feature geometry is proposed. First, considering the interpretability and constraints of the model, a method based on l is adopted. 2,1 First, a norm-regularized loss function is used to explore the latent relationship between features and labels, enabling the selection of discriminative feature subsets. Second, based on the self-representation strategy, the characteristics of higher-order labels are combined to fully utilize and mine the hidden information behind the labels, and explicitly indicate the correlation between two labels. Third, to better utilize local feature correlations, manifold learning is combined with Laplacian scores as a method to process local features, providing guidance for exploring the underlying true labels and selecting different features for individual labels. Furthermore, the combination of topological structure and graph Laplacian further constructs the geometric structure of local features. Finally, a multi-label feature selection algorithm is designed to obtain the optimally ranked feature subset. Experimental results show that our method outperforms other comparative methods in classification across multiple datasets. In our future work, we will extend this method to study feature selection algorithms for more complex multi-label classification, and further explore the hidden information of higher-order labels to obtain even better feature subset ranking sequences.

[0110] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. The scope of patent protection of the present invention shall be determined by the claims. Similarly, any equivalent structural changes made based on the description and drawings of the present invention shall also be included within the scope of protection of the present invention.

Claims

1. A text classification method based on sparse feature selection, characterized in that, Includes the following steps: 1) Based on a corpus containing multiple multi-label text datasets, input the sample matrix and label matrix into the objective function for iterative updates. The objective function pre-selects a subset of features based on the sample matrix and label matrix. The correlation between two labels is determined based on the local feature correlation function established by manifold constraints and Laplace fractions, as well as the higher-order label information function. The objective function is based on the following... Norm-based regularized loss functions explore the relationship between labels and features: , The characteristic coefficient matrix, Denotes the Frobenius norm. express of Norm; For the sample matrix, The label matrix; The local feature correlation function based on the Laplace score is as follows: , , For the graph Laplace matrix, It is a diagonal matrix. For about of Nearest neighbor graph example, Represents the trace of a matrix; , For the tag-related information matrix, This is the error vector; ; The local feature correlation function, based on manifold constraints, is as follows: , For the graph Laplace matrix, It is a diagonal matrix; The different local feature correlation matrices are weighted according to the different sample groups they belong to, and the processed sample groups are then merged to obtain the result. , For the first Local feature discrimination correlation matrix of each sample group; The higher-order tag information function is: , This represents a column vector consisting entirely of 1s. This is the deviation regression coefficient matrix; The objective function is: α, β, γ, and λ represent different hyperparameters; 2) When the set stopping rule is reached, stop the iterative update process, output the feature subsets according to the set sorting, select the first set number of subsets in the output feature subsets to form a new subset, and use the new subset as the final feature subset to classify the corpus.

2. The text classification method based on sparse feature selection according to claim 1, characterized in that, The system iterates through the values ​​of the three matrices W, H, and P in the objective function, stopping when the set iteration conditions are met or the maximum number of iterations is reached.

3. The text classification method based on sparse feature selection according to claim 2, characterized in that, The set iteration condition is met when the difference between two consecutive function values ​​is less than a preset value.

4. The text classification method based on sparse feature selection according to claim 3, characterized in that, First, the values ​​of W, H, and P are randomly initialized. Then, the values ​​of these three matrices are updated using an iterative optimization method.

5. The text classification method based on sparse feature selection according to claim 4, characterized in that, The optimal solution is sought through the following alternating optimization steps: Update W: Fix H and P, solve for W; Update H: Fix W and P, solve for H; Update P: Fix W and H, solve for P.

6. The text classification method based on sparse feature selection according to claim 5, characterized in that, When updating W, V is a diagonal matrix. .

7. The text classification method based on sparse feature selection according to claim 5, characterized in that, When updating H, , where I is the identity matrix.

8. The text classification method based on sparse feature selection according to claim 5, characterized in that, When updating P, , where n is the number of instances.

9. The text classification method based on sparse feature selection according to claim 1, characterized in that, , Let X represent the i-th column of the c-th sample group, where d is the feature number. X is divided into s groups, i.e. ,in Let c represent the c-th sample group.

10. The text classification method based on sparse feature selection according to claim 9, characterized in that, X is divided into s groups using the K-means algorithm.

Images