A feature selection method for multi-topic text data with relaxed labeling

Abstract

The present invention relates to a method for selecting multi-topic text data with relaxed labels, and relates to the technical field of natural language processing. The method comprises: given multi-topic text data consisting of a feature space and a label space; using a non-negative label relaxation matrix to obtain a relaxed label space; constructing a feature selection matrix and decomposing it to obtain a structured subspace and a factor matrix, and mining the structured relationship between features and between labels; constructing a loss function and applying a norm constraint, performing norm sparse regularization on the feature selection matrix, and constructing a total objective function based on this; then solving the total objective function iteratively to obtain a final feature selection matrix; finally, using the final feature selection matrix to select features to characterize the original text data. The present invention significantly improves the accuracy, interpretability, generalization ability and computational efficiency of feature selection of multi-topic text data through technical means such as label relaxation, structured subspace decomposition, and sparse regularization.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method for selecting features of multi-topic text data with loose markings. Background Art

[0002] Multi-label learning is an important field in machine learning, and its research goal is to determine the most appropriate set of labels for each instance. With the development of the information age, the dimensionality of multi-label data has surged in a variety of rich application environments, especially text classification. For example, a news report may involve both entertainment and sports, and therefore needs to be classified under these two labels. This classification method allows text to belong to multiple related fields or topics at the same time, thereby more accurately reflecting the content and nature of the text. High-dimensional text data may be mixed with many redundant and irrelevant features, which will lead to expensive computational overhead and unsatisfactory learning performance. Therefore, feature selection techniques are used to accurately locate the most salient features from high-dimensional text datasets, aiming to create a compact and accurate representation of text data in low dimensions.

[0003] Multi-label feature selection methods are roughly divided into three modes, namely packaging mode, filtering mode and embedding mode. Compared with the first two modes, the embedding mode requires lower computational cost because it can optimize the model and learn the feature matrix at the same time. In recent years, the embedding mode has attracted the attention of researchers for its unique advantages. The embedding method usually selects the most informative features by exploring the three structural relationships of multi-label data, namely, the relationship between features and labels, between labels, and between features. Therefore, the present invention focuses on an embedded multi-label feature selection method, which selects significant features to characterize the original multi-topic text data, thereby improving the classification performance of the multi-topic text data. Summary of the invention

[0004] In order to solve the above problems, the present invention proposes a method for selecting features of multi-topic text data with relaxed labeling.

[0005] The specific plan is as follows:

[0006] A method for selecting features from multi-topic text data with relaxed labeling, comprising:

[0007] S1, given multi-topic text data consisting of feature space and tag space;

[0008] S2, based on multi-topic text data, uses a non-negative label relaxation matrix to obtain the relaxed label space;

[0009] S3, construct a feature selection matrix and decompose it to obtain a structured subspace and a factor matrix; based on the structured subspace and the factor matrix, mine the structured relationships between features and between tags in multi-topic text data;

[0010] S4, constructing a loss function, applying a norm constraint to the constructed loss function; performing norm sparse regularization on the feature selection matrix based on the loss function; and constructing a total objective function based on the feature selection matrix;

[0011] S5, solving the total objective function based on the relaxed label space and the structured relationship between features and labels; iterating the solution of the total objective function to obtain the final feature selection matrix;

[0012] S6, features are selected through the final feature selection matrix to represent the original text data.

[0013] Furthermore, in S1, the multi-topic text data includes:

[0014] A feature space with d-dimensional features and a label space with l categories of topics Where n represents the number of texts in the multi-topic text data, represents the feature description of the i-th text; if the feature description x of the i-th text i is associated with the jth topic, then Y ij =1, otherwise, Y ij =0.

[0015] Furthermore, the S2 specifically includes:

[0016] S21, design a directional traction matrix To ensure the relaxation direction, B ij Defined as:

[0017]

[0018] Among them, each element B ij Represents the relaxation direction, a value of "+1" indicates the positive direction of relaxation, and "-1" indicates the negative direction of relaxation;

[0019] S22, randomly initialized non-negative labeled relaxation matrix Among them, S ij Represents the value of the i-th row and j-th column of the matrix S;

[0020] S23, construct the relaxed labeling space The calculation formula is as follows:

[0021]

[0022] Furthermore, the S3 specifically includes:

[0023] S31, decompose the constructed feature selection matrix W into structured subspace Q and factor matrix A, and get Where, F represents the norm;

[0024] S32, impose orthogonal constraints Q on the structured subspace Q T Q=I l ; Among them, I l Represents the l*l identity matrix;

[0025] S33, obtaining the relationship between features in the text, that is, the Laplacian graph representing the feature space; wherein, if q in the structured subspace Q a and q b is closely related, and the following regularization term is obtained:

[0026]

[0027] Among them, q a represents the ath row of the structured subspace Q, i.e., the ath feature position in the structured subspace Q, q b represents the bth row of the structured subspace Q, Q represents a d*l matrix, d represents the feature dimension, represents the Laplacian graph of the feature space, represents a diagonal matrix with diagonal elements represents an affinity matrix based on the feature space, which is constructed by the heat kernel function as follows:

[0028]

[0029] Where σ represents the bandwidth of the heat kernel function, Indicated by X b. Constructed k nearest neighbor set;

[0030] S34, obtaining the relationship between topic tags in the text, that is, the Laplacian graph of the tag space; wherein, if the c-th column q in the structured subspace Q c and the fth column q f is closely related, and the following regularization term is obtained:

[0031]

[0032] in, The Laplacian graph representing the labeling space; represents a diagonal matrix whose diagonal elements are matrix The heat kernel function is constructed as follows:

[0033]

[0034] Among them, Y .c and Y .f represents the cth label vector and the fth label vector in the label space, Indicated by Y .c Constructed k nearest neighbor set.

[0035] Furthermore, the S4 specifically includes:

[0036] S41, construct the loss function and apply Norm constraint, the calculation formula is as follows:

[0037]

[0038] in, express norm;

[0039] S42, perform feature selection on the matrix W Norm sparse regularization, we get ||W|| 2,1 ;

[0040] S43, constitutes the overall objective function, and the calculation formula is as follows:

[0041]

[0042] QUR T Q=I l

[0043] Among them, α, β, γ and λ represent trade-off parameters.

[0044] Furthermore, the S5 specifically includes:

[0045] S51, based on the total objective function, fix other variables and update the factor matrix A, and get:

[0046]

[0047] S52, set W = [w 1 ;w 2 ;...;w d ]and Fix other variables and update the feature selection matrix W to get:

[0048]

[0049] Among them, w 1 ;w 2 ;...;w d Respectively represent the 1st, 2nd, ..., dth rows of the feature selection matrix W; v 1 ;v 2 ;...;vn Respectively represent matrices The 1st, 2nd, ..., nth rows of ; represents a diagonal matrix whose diagonal elements are D represents a diagonal matrix whose diagonal elements are D ee =1 / 2||w e ||2; Yes Taking the derivative of W in the equation, we get:

[0050]

[0051] Where G = C-αQQ T , I d Represents the identity matrix of d*d;

[0052] S53, fix other variables and update the structured subspace Q, the formula is as follows:

[0053]

[0054] According to the Sherman-Morrison-Woodbury formula, G -1 =C -1 +αC -1 Q(I l -αQ T CQ) -1 Q T C -1 ,have to:

[0055]

[0056] in, Setting the derivative of Q to zero gives:

[0057]

[0058] Then solve Q through Matlab's Lyapunov function;

[0059] S54, assuming X T WY=K, fix other variables and solve for S, the formula is as follows:

[0060]

[0061] Among them, K is an intermediate variable, and the optimal solution of S is calculated as:

[0062]

[0063] S55, based on the calculated factor matrix A, feature selection matrix W, structured subspace Q and non-negative labeled relaxation matrix S, iteratively execute S51-S55 to obtain the final feature selection matrix.

[0064] Furthermore, the S6 specifically includes:

[0065] S61, according to ||W o ||2(o=1→d) in descending order, the top 20% of features are selected as feature subsets; where i represents the i-th row of the final feature selection matrix W;

[0066] S62, using MLKNN to test the selected feature subset, and recording the average performance of the multi-topic text dataset Arts on Hamming loss and Micro-F1 indicators through five-fold cross validation.

[0067] The present invention adopts the above technical solution and has the following beneficial effects:

[0068] (1) The present invention uses a non-negative label relaxation matrix to convert the original label space into a relaxation variable matrix, so that an accurate feature selection matrix can be obtained at the beginning of the fitting process;

[0069] (2) The feature selection matrix of the present invention is decomposed into structured subspaces, which can simultaneously explore the structured relationships between features and between tags;

[0070] (3) The present invention constructs an objective function for feature selection and proposes an effective optimization method to solve the overall objective function, thereby obtaining a more robust feature selection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 A method for selecting features of multi-topic text data with relaxed marking according to an embodiment of the present invention;

[0072] Figure 2 Flow chart of a multi-topic text data feature selection algorithm according to an embodiment of the present invention. DETAILED DESCRIPTION

[0073] The present invention is further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0074] like Figure 1 and Figure 2 As shown, the present invention provides a method for selecting features of multi-topic text data with relaxed labeling, comprising:

[0075] S1, given multi-topic text data consisting of feature space and tag space.

[0076] Specifically, the multi-topic text data includes:

[0077] A feature space with d-dimensional features and a label space with l categories of topics Where n represents the number of texts in the multi-topic text data, represents the feature description of the i-th text; if the feature description x of the i-th text i is associated with the jth topic, then Y ij =1, otherwise, Y ij =0;

[0078] Given any matrix The trace of U is denoted as Tr(U), and the transpose of U is denoted as U T , the Frobenius norm of U is defined as U's The norm is defined as Among them U ij represents the element in the i-th row and j-th column of matrix U, is a positive definite matrix with k variables.

[0079] S2, based on multi-topic text data, uses a non-negative label relaxation matrix to obtain the relaxed label space.

[0080] The non-negative label relaxation matrix improves the flexibility of mapping between features and labels. Specifically, S2 specifically includes:

[0081] S21, design a directional traction matrix To ensure the relaxation direction, B ij Defined as:

[0082]

[0083] Among them, each element B ij Represents the relaxation direction, a value of "+1" indicates the positive direction of relaxation, and "-1" indicates the negative direction of relaxation;

[0084] S22, randomly initialized non-negative labeled relaxation matrix Among them, Sij represents the value of the i-th row and j-th column of the matrix S;

[0085] S23, construct the relaxed labeling space The calculation formula is as follows:

[0086]

[0087] To illustrate this operation with an example, suppose:

[0088]

[0089]

[0090] So, we can get:

[0091]

[0092] S3, constructs a feature selection matrix and decomposes it to obtain a structured subspace and a factor matrix; based on the structured subspace and the factor matrix, it mines the structured relationships between features and between tags in multi-topic text data.

[0093] Specifically, the S3 specifically includes:

[0094] S31, decompose the feature selection matrix W into the structured subspace Q and the factor matrix A, and get:

[0095]

[0096] Where, F represents the norm;

[0097] S32, impose orthogonal constraints Q on the structured subspace Q T Q=I l , which is convenient for model optimization; among them, I l Represents the l*l identity matrix;

[0098] S33, obtaining the relationship between features in the text, that is, the Laplacian graph representing the feature space; wherein, if q in the structured subspace Q a and q b is closely related, and the following regularization term is obtained:

[0099]

[0100] Among them, q a represents the ath row of the structured subspace Q, i.e., the ath feature position in the structured subspace Q, q b represents the bth row of the structured subspace Q, Q represents a d*l matrix, d represents the feature dimension, represents the Laplacian graph of the feature space, represents a diagonal matrix with diagonal elements represents an affinity matrix based on the feature space, which is constructed by the heat kernel function as follows:

[0101]

[0102] Where σ represents the bandwidth of the heat kernel function, Indicated by X b.Constructed k nearest neighbor set; S34, obtain the relationship between topic tags in the text, that is, the Laplacian graph of the tag space; where, if the c-th column q in the structured subspace Q c and the fth column q f is closely related, and the following regularization term is obtained:

[0103]

[0104] in, The Laplacian graph representing the labeling space; represents a diagonal matrix whose diagonal elements are matrix The heat kernel function is constructed as follows:

[0105]

[0106] Among them, Y .c and Y .f represents the cth label vector and the fth label vector in the label space, Indicated by Y .c Constructed k nearest neighbor set.

[0107] S4, construct a loss function, impose a norm constraint on the constructed loss function; perform norm sparse regularization on the feature selection matrix based on the loss function; and construct an overall objective function based on the feature selection matrix.

[0108] The S4 specifically includes:

[0109] S41, construct the loss function and apply Norm constraint, the calculation formula is as follows:

[0110]

[0111] in, express norm;

[0112] S42, regularize the feature selection matrix W The norm is sparse, and we get ||W|| 2,1 ;

[0113] S43, integrating formulas (6), (7), (9), (11) and ||W|| 2,1 , forming the overall objective function, the calculation formula is as follows:

[0114]

[0115] QUR T Q=I l

[0116] Among them, α, β, γ and λ represent trade-off parameters.

[0117] S5, solving the overall objective function based on the relaxed label space and the structured relationship between features and labels; iterating the solution of the overall objective function to obtain the final feature selection matrix.

[0118] Specifically, the S5 specifically includes:

[0119] S51, based on Q T Q=I l And the final objective function, by setting the derivative of A in formula (12) to zero, fixing other variables and updating A, we get:

[0120]

[0121] S52, set W = [w 1 ;w 2 ;...;w d ]and Substituting formula (13) into formula (12), fixing other variables and updating W, and simplifying irrelevant terms, the formula is as follows:

[0122]

[0123] Among them, w 1 ;w 2 ;...;w d Respectively represent the 1st, 2nd, ..., dth rows of the feature selection matrix W; v 1 ;v 2 ;...;v n Respectively represent matrices The 1st, 2nd, ..., nth rows of ; represents a diagonal matrix whose diagonal elements are D represents a diagonal matrix whose diagonal elements are D ee =1 / 2||w e ||2; Yes Taking the derivative of W in the equation, we get:

[0124]

[0125] Where G = C-αQQ T , C is the intermediate variable, I d Represents the identity matrix of d*d;

[0126] S53, substitute formula (13) and formula (15) into formula (12), simplify irrelevant terms, fix other variables and update Q, the formula is as follows:

[0127]

[0128] According to the Sherman-Morrison-Woodbury formula, G -1 =C -1 +αC -1 Q(I l -αQ T CQ) -1 Q T C -1 ,have to:

[0129]

[0130] in, Setting the derivative of Q to zero gives:

[0131]

[0132] The form of formula (18) satisfies U1Q+QU2=Z, and Z = 0, and finally solve Q by Matlab’s Lyapunov function;

[0133] S54, assuming X T WY=K, and simplify the irrelevant terms in formula (11), fix other variables and solve S, the formula is as follows:

[0134]

[0135] Among them, K is the intermediate variable, B represents the directional traction matrix, and the optimal solution of S is calculated as:

[0136]

[0137] S55, based on the calculated factor matrix A, feature selection matrix W, structured subspace Q and non-negative labeled relaxation matrix S, iteratively execute S51-S55 to obtain the final feature selection matrix.

[0138] S6, features are selected through the final feature selection matrix to represent the original text data, and the quality of the selected features is tested through the MLKNN classifier.

[0139] Specifically, the S6 specifically includes:

[0140] S61, according to ||W o ||2(o=1→d) in descending order, the top 20% of features are selected as feature subsets; where i represents the i-th row of the final feature selection matrix W;

[0141] S62, using MLKNN to test the selected feature subset, and recording the average performance of the multi-topic text dataset Arts on Hamming loss and Micro-F1 indicators through five-fold cross validation.

[0142] As shown in Table 1, taking the multi-emotional music dataset as an example, as shown in Table 2, the performance of MFSLL under the MLKNN classifier is the same as the five comparison methods on two indicators, among which the lower the value of Hamming loss, the better the performance, and the higher the value of Micro-F1, the better the performance, and the method LRMFS of the present invention has the best effect.

[0143] Table 1 Detailed characteristics of the multi-emotional music dataset;

[0144] Dataset Number of texts Number of features Number of markers Arts 5000 462 26

[0145] Table 2 Performance of MFSLL with 5 comparison methods on two indicators under MLKNN classifier;

[0146]

[0147] Although the present invention has been specifically shown and described in conjunction with the preferred embodiments, it should be understood by those skilled in the art that various changes may be made to the present invention in form and details without departing from the spirit and scope of the present invention as defined by the appended claims, all of which are within the scope of protection of the present invention.

Claims

1. A method for selecting features from multi-topic text data with loose labeling, characterized in that: include: S1, given multi-topic text data consisting of feature space and tag space; S2, based on multi-topic text data, uses a non-negative label relaxation matrix to obtain the relaxed label space; S3, construct the feature selection matrix and decompose it to obtain the structured subspace and factor matrix; Based on structured subspace and factor matrix, we mine the structured relationships between features and tags in multi-topic text data. S4, constructing a loss function, applying a norm constraint to the constructed loss function; performing norm sparse regularization on the feature selection matrix based on the loss function; Construct the overall objective function based on the feature selection matrix; S5, solving the overall objective function based on the relaxed label space and the structured relationships between features and labels; Iterate the solution of the total objective function to obtain the final feature selection matrix; S6, selecting features to represent the original text data through the final feature selection matrix; The multi-topic text data includes: A feature space with d-dimensional features and a label space with l categories of topics Where n represents the number of texts in the multi-topic text data, represents the feature description of the i-th text; if the feature description x of the i-th text i Associated with the jth topic, then Y ij =1, otherwise, Y ij =0; The S2 specifically includes: S21, design a directional traction matrix To ensure the relaxation direction, B ij Defined as: Among them, each element B ij Represents the relaxation direction, a value of "+1" indicates a positive relaxation direction, and a value of "-1" indicates a negative relaxation direction; S22, randomly initialized non-negative labeled relaxation matrix Among them, S ij The i-th text in the matrix S is associated with the j-th topic, and its position in the non-negative labeled relaxation matrix S is the i-th row and j-th column; S23, construct the relaxed labeling space The calculation formula is as follows: The S3 specifically includes: S31, decompose the constructed feature selection matrix W into structured subspace Q and factor matrix A, and get Where, F represents the norm; S32, impose orthogonal constraints Q on the structured subspace Q T Q=I l ; Among them, I l Represents the l*l identity matrix; S33, obtaining the relationship between features in the text, that is, the Laplacian graph representing the feature space; wherein, if q in the structured subspace Q a and q b is closely related, and the following regularization term is obtained: Among them, q a represents the ath row of the structured subspace Q, i.e., the ath feature position in the structured subspace Q, q b represents the bth row of the structured subspace Q, Q represents a d*l matrix, d represents the feature dimension, represents the Laplacian graph of the feature space, represents a diagonal matrix with diagonal elements represents an affinity matrix based on the feature space, which is constructed by the heat kernel function as follows: Where σ represents the bandwidth of the heat kernel function, Indicated by X b. Constructed k nearest neighbor set; S34, obtaining the relationship between topic tags in the text, that is, the Laplacian graph of the tag space; wherein, if the c-th column q in the structured subspace Q c and the fth column q f is closely related, and the following regularization term is obtained: in, The Laplacian graph representing the labeling space; represents a diagonal matrix whose diagonal elements are matrix The heat kernel function is constructed as follows: Among them, Y .c and Y .f represents the cth label vector and the fth label vector in the label space, Indicated by Y .c Constructed k nearest neighbor set; The S4 specifically includes: S41, construct the loss function and apply l 2,1 Norm constraint, the calculation formula is as follows: in, Indicates e 2,1 norm; S42, perform e on the feature selection matrix W 2,1 Norm sparse regularization, we get ||W|| 2,1 ; S43, constitutes the overall objective function, and the calculation formula is as follows: s.t.Q T Q=I l Among them, α, β, γ and λ represent trade-off parameters; The S5 specifically includes: S51, based on the total objective function, fix other variables and update the factor matrix A, and get: S52, set W = [w 1 ;w 2 ;...;w d ]and Fix other variables and update the feature selection matrix W to get: Among them, w 1 ;w 2 ;...;w d Respectively represent the 1st, 2nd, ..., dth rows of the feature selection matrix W; v 1 ;v 2 ;...;v n Respectively represent matrices The 1st, 2nd, ..., nth rows of ; represents a diagonal matrix whose diagonal elements are D represents a diagonal matrix whose diagonal elements are D ee =1 / 2||w e ||2; Yes Taking the derivative of W in the equation, we get: Where G = C-αQQ T , I d Represents the identity matrix of d*d; S53, fix other variables and update the structured subspace Q, the formula is as follows: According to the Sherman-Morrison-Woodbury formula, G -1 =C -1 +αC -1 Q(I l -αQ T CQ) -1 Q T C -1 ,have to: in, Setting the derivative of Q to zero gives: Then solve Q through Matlab's Lyapunov function; S54, assuming X T WY=K, fix other variables and solve for S, the formula is as follows: Among them, K is an intermediate variable, and the optimal solution of S is calculated as: S55, based on the calculated factor matrix A, feature selection matrix W, structured subspace Q and non-negative labeled relaxation matrix S, iteratively execute S51-S55 to obtain the final feature selection matrix.

2. The method for selecting features of multi-topic text data with label relaxation according to claim 1, characterized in that: The S6 specifically includes: S61, according to ||W o ||2(o=1→d) in descending order, select the top 20% features as the feature subset; where o represents the i-th row of the final feature selection matrix W; S62, using MLKNN to test the selected feature subset, and recording the average performance of the multi-topic text dataset Arts on Hamming loss and Micro-F1 indicators through five-fold cross validation.

Images