A multi-label text feature selection method based on improved ant colony optimization algorithm

Abstract

The present invention discloses a multi-label text feature selection method based on an improved ant colony optimization algorithm, comprising the following steps: step 1, data preparation, pre-processing of the text data set; step 2, environment preparation, extracting feature-label correlations from the converted text vectors and label vectors using ridge regression; step 3, ant colony optimization; step 4, feature selection and output of iterated pheromones. The present invention uses an ant colony to roam in a supervised and unsupervised weighted graph, searching for features with low redundancy under both supervised paradigms, while adaptively adjusting the number of features that the ant colony needs to roam during the iteration process on different data sets through node centrality, improving the search capability of the ant colony through parameter adaptation, and balancing the capabilities of exploration and development. The above two strategies enhance the optimization capability of the ant colony, enabling the ant colony to capture discriminative features and obtain the optimal feature subset.

Description

Technical Field

[0001] The present invention relates to the fields of natural language processing and machine learning, and in particular to a multi-label text feature selection method based on an improved ant colony optimization algorithm. Background Art

[0002] Since the beginning of the new century, information technology for data collection, transmission, and storage has continued to advance. The emergence of internet applications and social networks has led to a massive flow of information. Capturing this massive amount of data requires machine learning algorithms to learn from it.

[0003] For multi-label text, the set of words representing features obtained after preprocessing is often very large, and directly using them for classification can reduce accuracy. This is primarily due to the strong correlation between multiple labels, which affects the classification of multi-label text, and the features that affect it only occupy a portion or even a small part of the feature set, making the entire feature set filled with redundant and irrelevant features. When the redundancy or correlation between two features is particularly high, only one feature can describe the information contained in this part of the data. The additional features do not contribute any additional label information and can be regarded as noise for classification.

[0004] Feature selection is an important method to reduce data dimensionality. It directly removes redundant and irrelevant features from the original feature set, thereby reducing the feature set. The advantage is that it retains the original features, helps to understand the data and improve classification performance.

[0005] The Improved Ant Colony Optimization (IACO) algorithm is a swarm intelligence optimization algorithm that improves upon the ACO algorithm to adapt it to specific domains. When optimizing multi-label feature selection, the ACO algorithm loses most of the feature-label correlation during initialization. Furthermore, some of its built-in parameters are fixed and cannot be adaptively adjusted during the optimization process. Currently, there are no publicly published papers or patents in China related to methods for multi-label text feature selection using the Improved ACO algorithm. Summary of the Invention

[0006] In view of the shortcomings of the existing technology, the purpose of the present invention is to provide a multi-label text feature selection method based on an improved ant colony optimization algorithm to solve the problems raised in the above background technology. The present invention reduces the dimension by reducing the number of feature items and improves the accuracy of text classification.

[0007] To achieve the above object, the present invention is implemented through the following technical solution: a multi-label text feature selection method based on an improved ant colony optimization algorithm, comprising the following steps:

[0008] Step 1: Data preparation: preprocess the text dataset, convert it into text vectors that can be directly processed by computers after word segmentation and stop word removal, and divide the entire dataset into training and test sets;

[0009] Step 2: Prepare the environment. Use ridge regression to extract the feature-label correlation between the converted text vectors and label vectors, and convert them into a supervised feature graph. Use the Pearson coefficient to obtain the correlation between text vectors to measure the redundancy between features. Convert the obtained feature correlation into an unsupervised feature graph. Calculate the eigenvector centrality of the supervised feature graph and use it as the initial pheromone of the ant colony. Simultaneously calculate the eigenvector centrality of the unsupervised feature graph. Combine the two node centralities to determine the number of features that the ant colony needs to roam on a certain text data, so as to achieve the adaptive pheromone volatilization rate.

[0010] Step 3: Ant colony optimization. The improved ant colony optimization algorithm is used to process the text data. The ants randomly wander on the features and accumulate pheromones at the nodes corresponding to the features.

[0011] Furthermore, the specific implementation of step 1 includes the following sub-steps:

[0012] Step 1.1: Remove non-textual content from the text dataset. Word segmentation divides the text into smaller language units. Stop words (meaningless words that appear repeatedly in the text) are removed. Case conversion converts the text vocabulary to a uniform case, removing low-frequency and high-frequency words.

[0013] Step 1.2: Calculate the term frequency TF. For each document, calculate the frequency of each word in the document. The term frequency (TF) can be expressed using formula (1):

[0014] TF=t / d (16)

[0015] Among them, t represents a word and d represents a document;

[0016] Step 1.3: Calculate the Inverse Document Frequency (IDF): Calculate the importance of each word in the entire document set. The Inverse Document Frequency (IDF) can be expressed using formula (2):

[0017] IDF(t)=log(D / T) (17)

[0018] Where D is the total number of documents, and T is the number of documents containing word t;

[0019] Step 1.4: Calculate TF-IDF: Multiply the term frequency (TF) and the inverse document frequency (IDF) to get the TF-IDF value of the term. TF-IDF can be expressed using formula (3):

[0020] TF-IDF(t,d)=TF(t,d)*IDF(t) (18)

[0021] The obtained word vectors are sparsely processed to obtain the preprocessed text dataset. The processed dataset is divided into a training set and a test set.

[0022] Furthermore, the specific implementation of step 2 includes the following sub-steps:

[0023] Step 2.1: X∈R n×m It is a text feature matrix with n instances and m features, Y∈R n×q There are n samples and q different class labels, W∈R m×q It is a coefficient matrix reflecting the correlation relationship between feature labels:

[0024]

[0025] Step 2.2: Construct a supervised weighted graph. In order to obtain the weights between features, Euclidean distance calculation is used:

[0026]

[0027] Where i is the i-th feature and j is the j-th feature. ij is the weight between feature i and feature j;

[0028] Step 2.3: Construct an unsupervised weighted graph. To measure the unsupervised redundancy between features, we use the absolute value of the Pearson correlation coefficient between features:

[0029]

[0030] Where i and j are two n-dimensional (number of instances) feature vectors, and is the average value of the feature value of the entire data set. If they are linearly related, the value of the correlation coefficient will be closer to 1, but if they are independent, the value will be zero.

[0031] Further, step 2.4: For a weighted feature graph, the higher the eigenvector centrality of a feature node, the more important the feature. In this metric, the importance of a feature in eigenvector centrality depends on the importance of its neighboring features. Since the graph is fully connected, with undirected connections between every node, the neighboring feature nodes of a feature node are all the remaining feature nodes in the graph:

[0032] αNC s =FDM*NC s (twenty two)

[0033] α is the eigenvalue of FDM, and the largest eigenvalue α max The corresponding eigenvector is used as the centrality. Similarly, the eigenvector centrality of the unsupervised weighted graph is calculated as:

[0034] αNC us =corr*NC us (twenty three)

[0035] Among them NC us is the unsupervised node centrality.

[0036] Further, step 2.5: use the eigenvector centrality based on the supervised weighted graph as the initial pheromone, retaining all feature label correlations:

[0037] τ0=NC s (twenty four)

[0038] Where τ0 is the initialization pheromone. Combining the two node centralities, we determine the number of features NF that the ant colony needs to roam on a certain text data:

[0039]

[0040]

[0041] Among them NC s and NC us are the node centrality of the supervised weighted graph and the node centrality of the unsupervised weighted graph respectively; m is the feature dimension of the dataset; θ is the average probability of the feature being selected. i The probability Pr(F i ) > θ, indicating that the probability of the feature being selected exceeds the average probability. We record all features exceeding θ and obtain the NF. This issue is of particular concern when applying the ant colony optimization algorithm to large-scale problems. Using traditional evaporation rate algorithms can reduce the search capability of the ant colony optimization algorithm and lead to premature convergence. To address this, a dynamic pheromone evaporation strategy is adopted:

[0042]

[0043] t is the current iteration number, T is the maximum iteration number. ρ min It is the minimum boundary value of the volatilization rate, and its value has a certain impact on the convergence speed of the algorithm and the quality of the optimal solution.

[0044] Furthermore, the specific implementation of step 3 includes the following sub-steps:

[0045] Step 3.1: Randomly place ants on the feature map, using these random features as starting positions for the iterative optimization process. Set the maximum number of iterations and the size of the ant colony. Each ant uses a greedy and probabilistic state transition rule to visit different features in the search space to construct a solution. The state transition rule balances exploration and exploitation by applying heuristic information and pheromone information.

[0046]

[0047]

[0048] Furthermore, in the formula (14), is the set of feasible features that ant k can roam around but has not yet visited; τ i (t) is the value of the pheromone associated with feature i in generation t; η1(F i ,F j ) and η2(F i ,F j ) are the features F in the supervised weighted graph and the unsupervised weighted graph respectively. i and feature F j The redundancy between them is equal to FD ij and 1 / corr(i,j). Parameters α and β are between [0,1] and control the trade-off between pheromone value and heuristic information. Parameter g is a random variable uniformly distributed in [0,1], with g0∈[0,1] being an adaptive value as the algorithm runs.

[0049] Furthermore, in step 3.2, each ant can perform a global update of the pheromone. When each ant visits a feature, the pheromone is updated by determining an m-dimensional vector called AntFeatureCount. When each ant visits a feature, the AFC is incremented. After all ants have completed their walk within a generation, the global pheromone update rule (Equation (15)) is applied to update the pheromone value of each feature i:

[0050]

[0051] Further, step 3.3: repeat the above steps 3.1 and 3.2 until the maximum number of iterations is reached, and a certain amount of pheromones are continuously accumulated while volatilizing.

[0052] Furthermore, after completing step 3, feature selection is performed and the iterated pheromones are output. The pheromones are sorted in descending order to obtain a feature sorting result as the result of feature selection.

[0053] Beneficial effects of the present invention:

[0054] 1. This multi-label text feature selection method, based on an improved ant colony optimization algorithm, uses node centrality to initialize pheromones to accelerate ant colony convergence. By navigating supervised and unsupervised weighted graphs, it searches for features with low redundancy under both supervised and unsupervised paradigms. Furthermore, node centrality is used to adaptively adjust the number of features the ant colony must navigate during its iterations across different datasets. The pheromone evaporation rate is also adaptively adjusted as the search progresses.

[0055] 2. This multi-label text feature selection method based on an improved ant colony optimization algorithm improves the ant colony's search capabilities through parameter adaptation, balancing exploration and exploitation. By enhancing the ant colony's optimization capabilities through these two strategies, the ant colony can capture discriminative features and obtain the optimal feature subset. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a basic flow chart of text classification according to an embodiment of the present invention;

[0057] Figure 2 Flowchart of an embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to make the technical means, creative features, objectives and effects achieved by the present invention easier to understand, the present invention is further described below in conjunction with specific implementation methods.

[0059] See also Figures 1 to 2 The present invention provides a technical solution: a multi-label text feature selection method based on an improved ant colony optimization algorithm, comprising the following steps:

[0060] Step 1: Remove the non-text parts of the text dataset. Word segmentation divides the text into smaller language units and removes stop words, which are words that appear repeatedly in the text without any actual meaning. Case conversion converts the text vocabulary to a uniform case and removes low-frequency and high-frequency words. Calculate the term frequency (TF). For each document, calculate the frequency of each word in the document. The term frequency (TF) can be expressed using formula (1):

[0061] TF=t / d (31)

[0062] Where t represents a word and d represents a document. Calculate the inverse document frequency (IDF): Calculate the importance of each word in the entire document set. The inverse document frequency (IDF) can be expressed using formula (2):

[0063] IDF(t)=log(D / T) (32)

[0064] In this example, D is the total number of documents and T is the number of documents containing word t. Calculate TF-IDF: Multiply the term frequency (TF) and the inverse document frequency (IDF) to get the TF-IDF value of the term. TF-IDF can be expressed using formula (3):

[0065] TF-IDF(t,d)=TF(t,d)*IDF(t) (33)

[0066] The obtained word vectors are sparsely processed to obtain the preprocessed text dataset. The processed dataset is divided into a training set and a test set.

[0067] Step 2: X∈R n×m It is a text feature matrix with n instances and m features, Y∈R n×q There are n samples and q different class labels, W∈R m×q It is a coefficient matrix reflecting the correlation relationship between feature labels:

[0068]

[0069] Construct a supervised weighted graph. In order to obtain the weights between features, Euclidean distance calculation is used:

[0070]

[0071] Where i is the i-th feature and j is the j-th feature. ij is the weight between feature i and feature j. Construct an unsupervised weighted graph. To measure the unsupervised redundancy between features, we use the absolute value of the Pearson correlation coefficient between features:

[0072]

[0073] In this embodiment, i and j are two n-dimensional (number of instances) feature vectors, and is the average value of the feature quantity of the entire dataset. If they are linearly related, the value of the correlation coefficient will be closer to 1, but if they are independent, the value will be zero. For weighted feature graphs, the higher the eigenvector centrality of a feature node, the more important the feature is. In this measure, the importance of a feature in the eigenvector centrality depends on the importance of its neighboring features. Since the graph is fully connected and there are undirected connections between each node, the neighboring feature nodes of a feature node are all the remaining feature nodes in the graph:

[0074] αNC s =FDM*NC s (37)

[0075] α is the eigenvalue of FDM, and the largest eigenvalue α max The corresponding eigenvector is used as the centrality. Similarly, the eigenvector centrality of the unsupervised weighted graph is calculated as:

[0076] αNC us =corr*NC us (38)

[0077] Among them NC us is the unsupervised node centrality. The eigenvector centrality based on the supervised weighted graph is used as the initial pheromone, retaining all feature label correlations:

[0078] τ0=NC s (39)

[0079] Where τ0 is the initialization pheromone. Combining the two node centralities, we determine the number of features NF that the ant colony needs to roam on a certain text data:

[0080]

[0081]

[0082] In this embodiment, NC s and NC us are the node centrality of the supervised weighted graph and the node centrality of the unsupervised weighted graph respectively; m is the feature dimension of the dataset; θ is the average probability of the feature being selected. i The probability Pr(F i ) > θ, indicating that the probability of the feature being selected exceeds the average probability. We record all features exceeding θ and obtain the NF. This issue is of particular concern when applying the ant colony optimization algorithm to large-scale problems. Using traditional evaporation rate algorithms can reduce the search capability of the ant colony optimization algorithm and lead to premature convergence. To address this, a dynamic pheromone evaporation strategy is adopted:

[0083]

[0084] t is the current iteration number, T is the maximum iteration number. ρ min It is the minimum boundary value of the volatilization rate, and its value has a certain impact on the convergence speed of the algorithm and the quality of the optimal solution.

[0085] Step 3: Randomly place ants on the feature map, using these random features as starting positions for the iterative optimization process. Set the maximum number of iterations and the size of the ant colony. Each ant uses a greedy and probabilistic state transition rule to visit different features in the search space to construct a solution. The state transition rule balances exploration and exploitation by applying heuristic information and pheromone information.

[0086]

[0087]

[0088] in is the set of feasible features that ant k can roam around but has not yet visited; τ i (t) is the value of the pheromone associated with feature i in generation t; η1(F i ,F j ) and η2(F i ,F j ) are the features F in the supervised weighted graph and the unsupervised weighted graph respectively. i and feature F j The redundancy between them is equal to FD ij and 1 / corr(i,j). Parameters α and β are between [0,1] and control the trade-off between pheromone value and heuristic information. Parameter g is a random variable uniformly distributed in [0,1], with g0∈[0,1] being an adaptive value as the algorithm runs.

[0089] Each ant can perform a global update of its pheromone. When each ant visits a feature, it updates the pheromone by determining an m-dimensional vector called AntFeatureCount. When each ant visits a feature, the AFC is incremented. After all ants have completed their walk within a generation, the global pheromone update rule (Equation (15)) is applied to update the pheromone value of each feature i:

[0090]

[0091] Repeat steps 3.1 and 3.2 above until the maximum number of iterations is reached, accumulating pheromones while evaporating a certain amount.

[0092] Step 4: Feature selection and output of iterated pheromones. Sort the pheromones in descending order to obtain the feature sorting result as the result of feature selection.

[0093] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, from all points of view, the embodiments should be regarded as illustrative and non-restrictive. The scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes that fall within the meaning and range of equivalents of the claims are included in the present invention. Any reference signs in the claims should not be construed as limiting the claim to which they relate.

[0094] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A multi-label text feature selection method based on an improved ant colony optimization algorithm, characterized by: The following steps are involved: Step 1: Data preparation: preprocess the text dataset, convert it into text vectors that can be directly processed by computers after word segmentation and stop word removal, and divide the entire dataset into training and test sets; Step 2: Prepare the environment. Use ridge regression to extract the feature-label correlation between the converted text vectors and label vectors, and convert them into a supervised feature graph. Use the Pearson coefficient to obtain the correlation between text vectors to measure the redundancy between features. Convert the obtained feature correlation into an unsupervised feature graph. Calculate the eigenvector centrality of the supervised feature graph and use it as the initial pheromone of the ant colony. Simultaneously calculate the eigenvector centrality of the unsupervised feature graph. Combine the two node centralities to determine the number of features that the ant colony needs to roam on a certain text data, so as to achieve the adaptive pheromone volatilization rate. Step 3: Ant Colony Optimization: Using the improved ant colony optimization algorithm to process the text data, the ant colony randomly wanders on the features and accumulates pheromones at the nodes corresponding to the features. The specific implementation of Step 3 includes the following sub-steps: Step 3.1: Randomly place ants on the feature map and use these random features as the starting positions in the ant iterative optimization process. Set the maximum number of iterations and the size of the ant colony. Each ant uses greedy and probabilistic state transition rules to access different features in the search space to construct a solution. The state transition rule balances exploration and exploitation by applying heuristic information and pheromone information: ； ； In the formula (14), It's an ant The set of feasible features that have not yet been visited by the walk; Is and characteristics Related The value of the pheromone of the generation; and The features in supervised weighted graph and unsupervised weighted graph are respectively and features The redundancy between them is equal to and ,parameter and exist The trade-off between controlling pheromone value and heuristic information is controlled by the parameter is a random variable, Evenly distributed in is a value that adapts as the algorithm runs. Step 3.2: Each ant can globally update the pheromone. When each ant visits a feature, it determines a value called the ant feature counter. dimensional vector to update the pheromone. When each ant visits a feature, AFC will increase. After all ants have completed their walk within one generation, the global pheromone update rule (15) is applied to update each feature. Pheromone Value: 。 2. The multi-label text feature selection method based on the improved ant colony optimization algorithm according to claim 1, characterized in that: The specific implementation of step 1 includes the following sub-steps: Step 1.1: Remove non-textual content from the text dataset. Word segmentation divides the text into smaller language units. Stop words (meaningless words that appear repeatedly in the text) are removed. Case conversion converts the text vocabulary to a uniform case, removing low-frequency and high-frequency words. Step 1.2: Calculate the term frequency TF. For each document, calculate the frequency of each word in the document and use formula (1) to express the term frequency (TF): ； Among them, t represents a word and d represents a document; Step 1.3: Calculate the Inverse Document Frequency (IDF): Calculate the importance of each word in the entire document set and use formula (2) to express the Inverse Document Frequency (IDF): ； Where D is the total number of documents, and T is the number of documents containing word t; Step 1.4: Calculate TF-IDF: Multiply the term frequency (TF) and the inverse document frequency (IDF) to get the TF-IDF value of the term. Use formula (3) to express TF-IDF: ； The obtained word vectors are sparsely processed to obtain the preprocessed text dataset, and the processed dataset is divided into a training set and a test set.

3. The multi-label text feature selection method based on the improved ant colony optimization algorithm according to claim 1, characterized in that: The specific implementation of step 2 includes the following sub-steps: Step 2.1: Yes instances and The text feature matrix of features, Yes samples and class labels of different class labels, It is a coefficient matrix reflecting the correlation relationship between feature labels: ； Step 2.2: Construct a supervised weighted graph. In order to obtain the weights between features, Euclidean distance calculation is used: ； Among them, i is the i-th feature, j is the j-th feature, is the weight between feature i and feature j; Step 2.3: Construct an unsupervised weighted graph and use the absolute value of the Pearson correlation coefficient between features to measure the unsupervised redundancy between features: ； Where i and j are two n-dimensional (number of instances) feature vectors, and The correlation coefficient is the average value of the feature quantity of the entire data set. If they are linearly related, the value of the correlation coefficient will be closer to 1, but if they are independent, the value will be zero.

4. The multi-label text feature selection method based on the improved ant colony optimization algorithm according to claim 3, characterized in that: Step 2.4: The adjacent feature nodes of a feature node are all the remaining feature nodes in the graph: ； yes The eigenvalue, the largest eigenvalue The corresponding eigenvector is used as the centrality, and the eigenvector centrality of the unsupervised weighted graph is calculated as follows: ； in is the unsupervised node centrality.

5. The multi-label text feature selection method based on the improved ant colony optimization algorithm according to claim 4 is characterized in that: Step 2.5: Use the supervised weighted graph-based feature vector centrality as the initial pheromone, preserving all feature label correlations: ； in It is the initialization pheromone, which combines the two node centralities to determine the number of features NF that the ant colony needs to roam on a certain text data: ； ； in and They are the node centrality of supervised weighted graph and the node centrality of unsupervised weighted graph respectively; is the characteristic dimension of the dataset; is the average probability of the feature being selected, if the feature Probability , then it means that the probability of the feature being selected exceeds the average probability, record all the The characteristics of NF are obtained by using a dynamic pheromone evaporation strategy: ； is the current iteration number, is the maximum number of iterations, is the minimum boundary value of the volatilization rate.

6. The multi-label text feature selection method based on the improved ant colony optimization algorithm according to claim 1, characterized in that: Step 3.3: Repeat steps 3.1 and 3.2 above until the maximum number of iterations is reached, continuously accumulating pheromones while evaporating a certain amount.

7. The multi-label text feature selection method based on the improved ant colony optimization algorithm according to claim 1, characterized in that: After completing step 3, feature selection is performed and the iterated pheromones are output. The pheromones are sorted in descending order to obtain the feature sorting result as the result of feature selection.

Images