Multi-topic text data feature selection method with slack marks

Abstract

The invention relates to a multi-theme text data feature selection method with a slack mark, and relates to the technical field of natural language processing, and the method comprises the steps: giving multi-theme text data composed of a feature space and a mark space; obtaining a relaxed mark space by using a non-negative mark relaxation matrix; a feature selection matrix is constructed and decomposed to obtain a structured subspace and a factor matrix, and a structured relationship between features and between marks is mined; constructing a loss function and applying norm constraint, performing norm sparse regularization on the feature selection matrix, and constructing a total objective function based on the norm sparse regularization; solving the total objective function through iteration to obtain a final feature selection matrix; and finally, selecting features by utilizing the final feature selection matrix to represent the original text data. According to the method, the accuracy, the interpretability, the generalization ability and the calculation efficiency of feature selection of the multi-topic text data are remarkably improved through the technical means of label relaxation, structured subspace decomposition, sparse regularization and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method for selecting features of multi-topic text data with loose markings. Background Art

[0002] Multi-label learning is an important field in machine learning, and its research goal is to determine the most appropriate set of labels for each instance. With the development of the information age, the dimensionality of multi-label data has surged in a variety of rich application environments, especially text classification. For example, a news report may involve both entertainment and sports, and therefore needs to be classified under these two labels. This classification method allows text to belong to multiple related fields or topics at the same time, thereby more accurately reflecting the content and nature of the text. High-dimensional text data may be mixed with many redundant and irrelevant features, which will lead to expensive computational overhead and unsatisfactory learning performance. Therefore, feature selection techniques are used to accurately locate the most salient features from high-dimensional text datasets, aiming to create a compact and accurate representation of text data in low dimensions.

[0003] Multi-label feature selection methods are roughly divided into three modes, namely packaging mode, filtering mode and embedding mode. Compared with the first two modes, the embedding mode requires lower computational cost because it can optimize the model and learn the feature matrix at the same time. In recent years, the embedding mode has attracted the attention of researchers for its unique advantages. The embedding method usually selects the most informative features by exploring the three structural relationships of multi-label data, namely, the relationship between features and labels, between labels, and between features. Therefore, the present invention focuses on an embedded multi-label feature selection method, which selects significant features to characterize the original multi-topic text data, thereby improving the classification performance of the multi-topic text data. Summary of the invention

[0004] In order to solve the above problems, the present invention proposes a method for selecting features of multi-topic text data with relaxed labeling.

[0005] The specific plan is as follows:

[0006] A method for selecting features from multi-topic text data with relaxed labeling, comprising:

[0007] S1, given multi-topic text data consisting of feature space and tag space;

[0008] S2, based on multi-topic text data, uses a non-negative label relaxation matrix to obtain the relaxed label space;

[0009] S3, construct a feature selection matrix and decompose it to obtain a structured subspace and a factor matrix; based on the structured subspace and the factor matrix, mine the structured relationships between features and between tags in multi-topic text data;

[0010] S4, constructing a loss function, applying a norm constraint to the constructed loss function; performing norm sparse regularization on the feature selection matrix based on the loss function; and constructing a total objective function based on the feature selection matrix;

[0011] S5, solving the total objective function based on the relaxed label space and the structured relationship between features and labels; iterating the solution of the total objective function to obtain the final feature selection matrix;

[0012] S6, features are selected through the final feature selection matrix to represent the original text data.

[0013] Furthermore, in S1, the multi-topic text data includes:

[0014] A feature space with d-dimensional features and a label space with l categories of topics ; Where n represents the number of texts in the multi-topic text data, represents the feature description of the i-th text; if the feature description of the i-th text is associated with the jth topic, then ,on the contrary, .

[0015] Furthermore, the S2 specifically includes:

[0016] S21, design a directional traction matrix To ensure the relaxation direction, Defined as:

[0017] ;

[0018] Among them, each element Represents the relaxation direction, a value of "+1" indicates the positive direction of relaxation, and "-1" indicates the negative direction of relaxation;

[0019] S22, randomly initialized non-negative labeled relaxation matrix ;in, Representation Matrix The value of the i-th row and j-th column of ;

[0020] S23, construct the relaxed labeling space , the calculation formula is as follows:

[0021] .

[0022] Furthermore, the S3 specifically includes:

[0023] S31, decompose the constructed feature selection matrix W into structured subspaces and the factor matrix ,have to ;in, represents the norm;

[0024] S32, impose orthogonal constraints on the structured subspace Q ;in, express The identity matrix of

[0025] S33, obtaining the relationship between features in the text, that is, the Laplacian graph representing the feature space; wherein, if the structured subspace middle and is closely related, and the following regularization term is obtained:

[0026] ;

[0027] in, Represents a structured subspace The ath row of is the structured subspace The ath feature position in , Represents a structured subspace The bth row of represents a d*l matrix, d represents the feature dimension, represents the Laplacian graph of the feature space, represents a diagonal matrix with diagonal elements , represents an affinity matrix based on the feature space, which is constructed by the heat kernel function as follows:

[0028] ;

[0029] in, represents the bandwidth of the heat kernel function, Indicated by Constructed k nearest neighbor set;

[0030] S34, obtaining the relationship between topic tags in the text, that is, the Laplacian graph of the tag space; wherein, if the structured subspace Column c and column d is closely related, and the following regularization term is obtained:

[0031] ;

[0032] in, The Laplacian graph representing the labeling space; represents a diagonal matrix whose diagonal elements are ;matrix The heat kernel function is constructed as follows:

[0033] ;

[0034] in, and represents the cth label vector and the dth label vector in the label space, Indicated by The k nearest neighbor set constructed, Indicates the telescopic scale.

[0035] Furthermore, the S4 specifically includes:

[0036] S41, construct the loss function and apply Norm constraint, the calculation formula is as follows:

[0037] ;

[0038] in, express norm;

[0039] S42, feature selection matrix conduct Norm sparse regularization, we get ;

[0040] S43, constitutes the overall objective function, and the calculation formula is as follows:

[0041] ;

[0042] in, , , and Both represent trade-off parameters.

[0043] Furthermore, the S5 specifically includes:

[0044] S51, based on the overall objective function, fix other variables to update the factor matrix ,have to:

[0045] ;

[0046] S52, setting and , fix other variables and update the feature selection matrix W, and get:

[0047] ;

[0048] in, They represent the 1st, 2nd, ..., dth rows of the feature selection matrix W respectively; Respectively represent matrices The 1st, 2nd, ..., nth rows of ; represents a diagonal matrix whose diagonal elements are ; represents a diagonal matrix whose diagonal elements are ;right In Taking the derivative we get:

[0049] ;

[0050] in, , , express * The identity matrix of

[0051] S53, fix other variables and update the structured subspace Q, the formula is as follows:

[0052] ;

[0053] According to the Sherman-Morrison-Woodbury formula, ,have to:

[0054] ;

[0055] in, ,Will The derivative of is set to zero, and we get:

[0056] ;

[0057] Then solve Q through Matlab's Lyapunov function;

[0058] S54, Assumption , fix other variables and solve S, the formula is as follows:

[0059] ;

[0060] in, As the intermediate variable, we can calculate The optimal solution is:

[0061] ;

[0062] S55, based on the calculation of the factor matrix , feature selection matrix W, structured subspace and the non-negative labeled relaxation matrix , iteratively execute S51-S55 to obtain the final feature selection matrix.

[0063] Furthermore, the S6 specifically includes:

[0064] S61, according to Select the top 20% of the features in descending order as the feature subset; where i represents the final feature selection matrix The i-th row of , d represents the number of features;

[0065] S62, using MLKNN to test the selected feature subset, and recording the average performance of the multi-topic text dataset Arts on Hamming loss and Micro-F1 indicators through five-fold cross validation.

[0066] The present invention adopts the above technical solution and has the following beneficial effects:

[0067] (1) The present invention uses a non-negative label relaxation matrix to convert the original label space into a relaxation variable matrix, so that an accurate feature selection matrix can be obtained at the beginning of the fitting process;

[0068] (2) The feature selection matrix of the present invention is decomposed into structured subspaces, which can simultaneously explore the structured relationships between features and between labels;

[0069] (3) The present invention constructs an objective function for feature selection and proposes an effective optimization method to solve the overall objective function, thereby obtaining a more robust feature selection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 A method for selecting features of multi-topic text data with relaxed marking according to an embodiment of the present invention;

[0071] Figure 2 Flow chart of a multi-topic text data feature selection algorithm according to an embodiment of the present invention. DETAILED DESCRIPTION

[0072] The present invention is further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0073] like Figure 1 and Figure 2 As shown, the present invention provides a method for selecting features of multi-topic text data with relaxed labeling, comprising:

[0074] S1, given multi-topic text data consisting of feature space and tag space.

[0075] Specifically, the multi-topic text data includes:

[0076] A feature space with d-dimensional features and a label space with l categories of topics ; Where n represents the number of texts in the multi-topic text data, represents the feature description of the i-th text; if the feature description of the i-th text is associated with the jth topic, then ,on the contrary, ;

[0077] Given any matrix , the trace of U is expressed as , the transpose of U is expressed as , the Frobenius norm of U is defined as , U The norm is defined as ;in represents the element in the i-th row and j-th column of the matrix U, is a positive definite matrix with k variables.

[0078] S2, based on multi-topic text data, uses a non-negative label relaxation matrix to obtain the relaxed label space.

[0079] The non-negative label relaxation matrix improves the flexibility of mapping between features and labels. Specifically, S2 specifically includes:

[0080] S21, design a directional traction matrix To ensure the relaxation direction, Defined as:

[0081] ; (1)

[0082] Among them, each element Represents the relaxation direction, a value of "+1" indicates the positive direction of relaxation, and "-1" indicates the negative direction of relaxation;

[0083] S22, randomly initialized non-negative labeled relaxation matrix ; Wherein, Sij represents the value of the i-th row and j-th column of the matrix S;

[0084] S23, construct the relaxed labeling space , the calculation formula is as follows:

[0085] ; (2)

[0086] To illustrate this operation with an example, suppose:

[0087] ; (3)

[0088] ; (4)

[0089] So, we can get:

[0090] ; (5).

[0091] S3, constructs a feature selection matrix and decomposes it to obtain a structured subspace and a factor matrix; based on the structured subspace and the factor matrix, it mines the structured relationships between features and between tags in multi-topic text data.

[0092] Specifically, the S3 specifically includes:

[0093] S31, decompose the feature selection matrix W into the structured subspace Q and the factor matrix A, and get:

[0094] ; (6)

[0095] in, represents the norm;

[0096] S32, impose orthogonal constraints on the structured subspace Q , which is convenient for model optimization; among them, express The identity matrix of

[0097] S33, obtaining the relationship between features in the text, that is, the Laplacian graph representing the feature space; wherein, if the structured subspace middle and is closely related, and the following regularization term is obtained:

[0098] ; (7)

[0099] in, Represents a structured subspace The ath row of is the structured subspace The ath feature position in , Represents a structured subspace The bth row of represents a d*l matrix, d represents the feature dimension, represents the Laplacian graph of the feature space, represents a diagonal matrix with diagonal elements , represents an affinity matrix based on the feature space, which is constructed by the heat kernel function as follows:

[0100] ; (8)

[0101] in, represents the bandwidth of the heat kernel function, Indicated by Constructed k nearest neighbor set;

[0102] S34, obtaining the relationship between topic tags in the text, that is, the Laplacian graph of the tag space; wherein, if the structured subspace Column c and column d is closely related, and the following regularization term is obtained:

[0103] ; (9)

[0104] in, The Laplacian graph representing the labeling space; represents a diagonal matrix whose diagonal elements are ;matrix The heat kernel function is constructed as follows:

[0105] ; (10)

[0106] in, and represents the cth label vector and the dth label vector in the label space, Indicated by The k nearest neighbor set constructed, Indicates the telescopic scale.

[0107] S4, construct a loss function, impose a norm constraint on the constructed loss function; perform norm sparse regularization on the feature selection matrix based on the loss function; and construct an overall objective function based on the feature selection matrix.

[0108] The S4 specifically includes:

[0109] S41, construct the loss function and apply Norm constraint, the calculation formula is as follows:

[0110] ; (11)

[0111] in, express norm;

[0112] S42, feature selection matrix The regularization is performed The norm is sparse, so we get ;

[0113] S43, integrating formulas (6), (7), (9), (11) and , forming the overall objective function, the calculation formula is as follows:

[0114] ; (12)

[0115] in, , , and Both represent trade-off parameters.

[0116] S5, solving the overall objective function based on the relaxed label space and the structured relationship between features and labels; iterating the solution of the overall objective function to obtain the final feature selection matrix.

[0117] Specifically, the S5 specifically includes:

[0118] S51, based on And the final objective function, by formula (12) The derivative of is set to zero, and A is updated after fixing other variables, and we get:

[0119] ; (13)

[0120] S52, setting and , substitute formula (13) into formula (12), fix other variables and update W, and simplify irrelevant terms, the formula is as follows:

[0121] ; (14)

[0122] in, They represent the 1st, 2nd, ..., dth rows of the feature selection matrix W respectively; Respectively represent matrices The 1st, 2nd, ..., nth rows of ; represents a diagonal matrix whose diagonal elements are ; represents a diagonal matrix whose diagonal elements are ;right In Taking the derivative we get:

[0123] ; (15)

[0124] in, , , is the intermediate variable, express * The identity matrix of

[0125] S53, substitute formula (13) and formula (15) into formula (12), simplify irrelevant terms, fix other variables and update Q. The formula is as follows:

[0126] ; (16)

[0127] According to the Sherman-Morrison-Woodbury formula, ,have to:

[0128] ; (17)

[0129] in, , setting the derivative of Q to zero, we get:

[0130] ; (18)

[0131] The form of formula (18) satisfies , , and , and finally solve Q through Matlab's Lyapunov function;

[0132] S54, Assumption , and simplify the irrelevant terms in formula (11), fix other variables and solve S, the formula is as follows:

[0133] ; (19)

[0134] Among them, K is the intermediate variable, B represents the directional traction matrix, and the optimal solution of S is calculated as:

[0135] ; (20);

[0136] S55, based on the calculation of the factor matrix , feature selection matrix W, structured subspace and the non-negative labeled relaxation matrix , iteratively execute S51-S55 to obtain the final feature selection matrix.

[0137] S6, features are selected through the final feature selection matrix to represent the original text data, and the quality of the selected features is tested through the MLKNN classifier.

[0138] Specifically, the S6 specifically includes:

[0139] S61, according to Select the top 20% of the features in descending order as the feature subset; where i represents the final feature selection matrix The i-th row of , d represents the number of features;

[0140] S62, using MLKNN to test the selected feature subset, and recording the average performance of the multi-topic text dataset Arts on Hamming loss and Micro-F1 indicators through five-fold cross validation.

[0141] As shown in Table 1, taking the multi-emotional music dataset as an example, as shown in Table 2, the performance of MFSLL under the MLKNN classifier is the same as the five comparison methods on two indicators, among which the lower the value of Hamming loss, the better the performance, and the higher the value of Micro-F1, the better the performance, and the method LRMFS of the present invention has the best effect.

[0142] Table 1 Detailed characteristics of the multi-emotional music dataset;

[0143]

[0144] Table 2 Performance of MFSLL with 5 comparison methods on two indicators under MLKNN classifier;

[0145]

[0146] Although the present invention has been specifically shown and described in conjunction with the preferred embodiments, it should be understood by those skilled in the art that various changes may be made to the present invention in form and details without departing from the spirit and scope of the present invention as defined by the appended claims, all of which are within the scope of protection of the present invention.

Claims

1. A method for selecting features from multi-topic text data with loose labeling, characterized in that: include: S1, given multi-topic text data consisting of feature space and tag space; S2, based on multi-topic text data, uses a non-negative label relaxation matrix to obtain the relaxed label space; S3, construct the feature selection matrix and decompose it to obtain the structured subspace and factor matrix; Based on structured subspace and factor matrix, we mine the structured relationships between features and tags in multi-topic text data. S4, constructing a loss function, applying a norm constraint to the constructed loss function; performing norm sparse regularization on the feature selection matrix based on the loss function; Construct the overall objective function based on the feature selection matrix; S5, solving the overall objective function based on the relaxed label space and the structured relationships between features and labels; Iterate the solution of the total objective function to obtain the final feature selection matrix; S6, features are selected through the final feature selection matrix to represent the original text data.

2. The method for selecting features of multi-topic text data with label relaxation according to claim 1, characterized in that: In S1, the multi-topic text data includes: A feature space with d-dimensional features and a label space with l categories of topics ; Where n represents the number of texts in the multi-topic text data, represents the feature description of the i-th text; if the feature description of the i-th text is associated with the jth topic, then ,on the contrary, .

3. A method for selecting features of multi-topic text data with loose labeling according to claim 2, characterized in that: The S2 specifically includes: S21, design a directional traction matrix To ensure the relaxation direction, Defined as: ； Among them, each element Represents the relaxation direction, a value of "+1" indicates a positive relaxation direction, and a value of "-1" indicates a negative relaxation direction; S22, randomly initialized non-negative labeled relaxation matrix ;in, Representation Matrix The i-th text of is associated with the j-th topic, and its position in the non-negative labeled relaxation matrix S is the i-th row and j-th column; S23, construct the relaxed labeling space , the calculation formula is as follows: 。 4. The method for selecting features of multi-topic text data with relaxed labeling according to claim 3, characterized in that: The S3 specifically includes: S31, decompose the constructed feature selection matrix W into structured subspaces and the factor matrix ,have to ;in, represents the norm; S32, impose orthogonal constraints on the structured subspace Q ;in, express The identity matrix of S33, obtaining the relationship between features in the text, that is, the Laplacian graph representing the feature space; wherein, if the structured subspace middle and is closely related, and the following regularization term is obtained: ； in, Represents a structured subspace The ath row of is the structured subspace The a-th feature position in , Represents a structured subspace The bth row of represents a d*l matrix, d represents the feature dimension, represents the Laplacian graph of the feature space, represents a diagonal matrix with diagonal elements , represents an affinity matrix based on the feature space, which is constructed by the heat kernel function as follows: ； in, represents the bandwidth of the heat kernel function, Indicated by Constructed k nearest neighbor set; S34, obtaining the relationship between topic tags in the text, that is, the Laplacian graph of the tag space; wherein, if the structured subspace Column c and column d is closely related, and the following regularization term is obtained: ； in, The Laplacian graph representing the labeling space; represents a diagonal matrix whose diagonal elements are ;matrix The heat kernel function is constructed as follows: ； in, and represents the cth label vector and the dth label vector in the label space, Indicated by The k nearest neighbor set constructed, Indicates the telescopic scale.

5. The method for selecting features of multi-topic text data with label relaxation according to claim 4, characterized in that: The S4 specifically includes: S41, construct the loss function and apply Norm constraint, the calculation formula is as follows: ； in, express norm; S42, feature selection matrix conduct Norm sparse regularization, we get ; S43, constitutes the overall objective function, and the calculation formula is as follows: ； in, , , and Both represent trade-off parameters.

6. The method for selecting features of multi-topic text data with label relaxation according to claim 5, characterized in that: The S5 specifically includes: S51, based on the overall objective function, fix other variables to update the factor matrix ,have to: ； S52, setting and , fix other variables and update the feature selection matrix W, and get: ； in, They represent the 1st, 2nd, ..., dth rows of the feature selection matrix W respectively; Respectively represent matrices The 1st, 2nd, ..., nth rows of ; represents a diagonal matrix whose diagonal elements are ; represents a diagonal matrix whose diagonal elements are ;right In Taking the derivative we get: ； in, , , express * The identity matrix of S53, fix other variables and update the structured subspace Q, the formula is as follows: ； According to the Sherman-Morrison-Woodbury formula, ,have to: ； in, ,Will The derivative of is set to zero, and we get: ； Then solve Q through Matlab's Lyapunov function; S54, Assumption , fix other variables and solve S, the formula is as follows: ； in, As the intermediate variable, we can calculate The optimal solution is: ； S55, based on the calculation of the factor matrix , feature selection matrix W, structured subspace and the non-negative labeled relaxation matrix , iteratively execute S51-S55 to obtain the final feature selection matrix.

7. The method for selecting features of multi-topic text data with relaxed labeling according to claim 6, characterized in that: The S6 specifically includes: S61, according to Select the top 20% of the features in descending order as the feature subset; where i represents the final feature selection matrix The i-th row of , d represents the number of features; S62, using MLKNN to test the selected feature subset, and recording the average performance of the multi-topic text dataset Arts on Hamming loss and Micro-F1 indicators through five-fold cross validation.

Images