A text sentiment classification method based on genetic algorithm
By combining genetic algorithms with machine learning techniques, and dynamically optimizing feature selection, the problems of feature selection and model rigidity in traditional text classification methods are solved, achieving more efficient sentiment classification results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING UNIV OF TECH
- Filing Date
- 2023-11-27
- Publication Date
- 2026-07-10
AI Technical Summary
Traditional text classification methods suffer from fixed feature selection and model rigidity in feature extraction and model building, making it difficult to cope with the diversity and dynamism of text data, resulting in insufficient accuracy and robustness in sentiment classification.
By combining genetic algorithms with machine learning classification techniques, and through the design of fitness functions and evolutionary operations, the optimal feature set is automatically selected and a high-performance text sentiment classification model is built, dynamically optimizing the feature selection process.
It improves the accuracy and robustness of text sentiment classification, better handles complex text data, adapts to diversity and dynamism, and provides a more flexible and efficient sentiment analysis solution.
Smart Images
Figure CN117668225B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of text sentiment classification, and in particular to an innovative method using genetic algorithms. The goal of this method is to extract optimal features from text data and build an efficient classification model to address the complex challenges of large-scale text classification. Background Technology
[0002] Sentiment classification is a core task in natural language processing, aiming to identify the emotional tendencies contained in text, such as positive, negative, or neutral emotions. However, due to the complexity and diversity of text, effectively extracting features and building accurate classification models becomes quite challenging. Traditional text classification methods have certain limitations in feature extraction and model building. Due to their fixed feature selection and model rigidity, they struggle to cope with the diversity and dynamism of text data. Furthermore, when processing large amounts of text data, the feature space can become extremely large, requiring further feature selection and dimensionality reduction to address this issue.
[0003] Genetic algorithms are heuristic optimization methods inspired by biological evolution. They simulate the selection, crossover, and mutation processes in biological evolution to find optimal solutions. In machine learning, genetic algorithms are widely used for feature selection and optimization problems. They can help systems automatically select the best subset of features, thereby improving classification performance. However, effectively integrating genetic algorithms with classification models to extract the optimal feature set remains a challenging problem.
[0004] To address this challenge, this paper proposes an innovative method that integrates genetic algorithms with machine learning classification techniques to solve complex text classification problems. The method aims to automatically extract the optimal feature set, assign optimal parameters to each base classification model, and integrate the base models by allocating different weights to construct a high-performance classification model.
[0005] The goal of this fusion approach is to overcome the limitations of traditional text classification methods and provide a more flexible and efficient solution for complex sentiment classification tasks. This will help address the challenges posed by the diversity and dynamism of text data, improve the robustness and accuracy of the model, and provide more reliable and efficient technical support for applications in fields such as sentiment analysis. Summary of the Invention
[0006] This invention first preprocesses the text data using techniques such as the bag-of-words model, TF-IDF, and word embedding to convert the text into numerical feature vectors. These feature vectors are then converted into a gene representation suitable for genetic algorithm operations. In the gene encoding stage, the genetic algorithm represents the text features as gene combinations.
[0007] The key innovation lies in designing a fitness function to evaluate the performance of each gene combination. This evaluation function, based on the classifier accuracy on a selected subset of features, helps determine the optimal feature combination to better capture text sentiment features. Subsequently, the genetic algorithm uses evolutionary operations such as selection, crossover, and mutation to optimize the gene combinations, continuously refining and improving the feature selection process. After multiple iterations of optimization, the genetic algorithm evolves into the optimal feature subset, which more accurately represents the sentiment information in the text data. Finally, the obtained optimal feature subset is combined with the corresponding classifier, such as Support Vector Machine or Naive Bayes, to construct an efficient text sentiment classification model.
[0008] This genetic algorithm-based approach offers an innovative and efficient solution for text sentiment classification. It overcomes the limitations of fixed feature selection and model rigidity in traditional methods, improving the accuracy and robustness of sentiment classification through dynamic feature selection optimization. This method has broad application prospects in handling sentiment classification problems in text data and provides strong support for processing increasingly complex text information. Attached Figure Description
[0009] Figure 1 Flowchart of a text sentiment classification method based on genetic algorithms
[0010] Figure 2 This is a schematic diagram of an improved genetic algorithm model. Detailed Implementation
[0011] Based on the above description, the following is a specific implementation process, but the scope of protection of this patent is not limited to this implementation process.
[0012] Step 1: Data preparation and preprocessing
[0013] We acquire online text data and corresponding sentiment classification tags, and then perform text preprocessing. First, we remove special symbols and punctuation marks from the text, then perform word segmentation and stop word removal, followed by stemming or lemmatization to convert words into their basic forms and reduce vocabulary diversity. Secondly, we address missing values or blank fields in the data.
[0014] Step 2: Feature Preprocessing
[0015] The text is converted into a feature vector using the TF-IDF method, where each row represents a text sample, each column represents a word, and each element in the matrix represents the TF-IDF value of the corresponding word in the corresponding text. Then, a sentiment score is assigned to each word. A sentiment dictionary is obtained, containing a large number of words and their sentiment polarity labels. Words representing positive sentiment are assigned positive values, and words representing negative sentiment are assigned negative values. Let the sentiment value be C. For each word, a final representation value is assigned based on the TF-IDF value and C, as shown below:
[0016]
[0017] Therefore, a feature matrix is generated for the text file, where each element represents the value of a word. A value greater than 0 and the larger the value, the stronger the positive sentiment, while a value less than 0 and the smaller the value, the stronger the negative sentiment. A sentiment label is then assigned to the text.
[0018] Step 3: Constructing a feature-based classification model using genetic model algorithms
[0019] Step 3.1 Gene encoding and population initialization
[0020] Gene Encoding: In the method of this invention, we need to encode features for use in genetic algorithms. Features are encoded using binary encoding, where 0 represents inclusion of the feature and 1 represents exclusion. The main purpose of this step is to create a population for the feature selection problem, where each individual represents a feature selection scheme. For the feature set obtained in step 2, we represent each feature as a binary number, where 1 indicates selection of the feature and 0 indicates exclusion.
[0021] Population initialization: In the initial stage of the genetic algorithm, we generate a population containing multiple individuals. Each individual represents a text feature selection scheme, that is, the selection state of a set of text features.
[0022] Step 3.2 Calculate the fitness function
[0023] Step 3.2.1 Calculate the number of features to be included.
[0024] Design the fitness function f(x) i ), where x i Representing different feature selection schemes, i.e., different individuals, this fitness function includes three key aspects. First, we aim to minimize the number of selected features. Let N be the number of features included, and N be the number of features included for different individuals. i Fewer features help reduce data dimensionality and computational complexity. Therefore, the first step of the fitness function is to select fewer features to improve the efficiency of the model.
[0025] Step 3.2.2 Building a prediction model based on machine learning
[0026] We use Logistic Regression (LR), Decision Tree (DT), Random Forest (RF), Support Vector Machine (SVM), K-Nearest Neighbors (KNN), and Naive Bayes (NB) as base models. In each iteration, we perform 10-fold cross-validation and grid search on each model, allowing for fine-tuning of model parameter configurations to maximize performance and prediction accuracy, fully leveraging the strengths of each model. We use the weighted average accuracy of all models as the second part of the fitness function, assigning different weights based on the performance of different base models. Let the weighted average accuracy be...
[0027] Step 3.2.3 Calculate the variance of the accuracy of each base model.
[0028] To ensure the stability of the selected features across all models, the variance of the accuracy of each base model is calculated. This measures the consistency of performance among different base models; a smaller variance indicates more stable performance across models. Therefore, this metric is the third part of the fitness function. Let the variance of the accuracy of each model be...
[0029] Step 3.2.4 Create the fitness function
[0030] Accuracy weighted average based on the number of features included N. And the variance of accuracy is Create the following fitness function:
[0031]
[0032] Where N i Ni represents the number of features included for each individual. Since the number of included features is greater than the weighted average of the model, and we want the number of included features to be as small as possible, we perform a logarithmic transformation with base a and take the opposite number. V represents the weighted average accuracy of different base models for each individual. i This represents the variance of the base model accuracy for each individual. We want the variance to be as small as possible, so we assign it a negative weight. To ensure a decent overall model performance, for For individuals with high fitness, we set their fitness to 0, meaning that we eliminate such individuals in each genetic iteration and retain those with higher fitness, i.e., those with higher overall accuracy.
[0033] Step 3.3 Selecting an operator
[0034] The linear sorting selection operator is used to determine which individuals are chosen to reproduce the next generation. This method considers not only fitness values but also the relative order among individuals. First, individuals are sorted from highest to lowest fitness value, and each individual is assigned a sequence number i. Assuming there are N individuals, the first-ranked individual with the highest fitness has i = N, the second-ranked individual has i = N-1, and so on, with the last individual having i = 1. The formula is:
[0035]
[0036] Let P max =0.9, P min =0.1, but different values can be set according to the actual application. Therefore, the probability that the individual with the highest fitness ranking is selected is P. max The probability of the last person being selected is P. min .
[0037] Step 3.4 Crossover Mutation
[0038] We use the concept of population adaptation to set the rules for crossover and mutation. First, we preset a crossover probability and a mutation probability. To find the optimal solution, in each iteration, we calculate the weighted average of the accuracy of each individual. We perform calculations and sort individuals from high to low. Based on the crossover probability x and the number of individuals N, we select individuals ranked N×x at the bottom for pairwise crossover to generate a new generation. The remaining individuals are directly copied to the next generation.
[0039] Normally, the mutation probability is a very small value. Similarly, for the mutation probability y and the number of individuals N, we select the individuals ranked N×y last for random point mutation. In this study, we convert the code to 0 and 1 to represent the selection of different feature combinations.
[0040] Step 3.5 Population Iteration
[0041] After crossover and mutation operations, new individuals are generated, which will replace some of the individuals in the previous generation. The new population will be used for fitness calculation and further iterations in the next generation. This process will be repeated for multiple generations, gradually evolving and optimizing the population. After iterating a specified number of times, the iteration will stop, and finally, the optimal feature selection scheme and weight allocation will be found, thereby building a high-performance text classification model. Detailed implementation method:
[0043] The specific embodiments of the present invention will be further described below with reference to specific examples and accompanying drawings.
[0044] Taking text classification as an example, suppose we have a dataset containing user reviews of products, where each review contains text and an associated sentiment label (positive, negative, or neutral). We need to classify the review data.
[0045] Step 1: Data Preparation and Preprocessing
[0046] First, user comment data, including comment text and sentiment tags, is obtained from social media platforms. Then, data cleaning is performed, missing values are handled, and text preprocessing is carried out, such as word segmentation and stop word removal.
[0047] Step 2 Feature Preprocessing
[0048] The TF-IDF method is used to convert user comment data into feature vectors, where each row represents a text sample, each column represents a word, and each element in the matrix represents the TF-IDF value of the corresponding word in the corresponding text. Then, a sentiment score is assigned to each word. A sentiment dictionary is obtained, containing a large number of words and their sentiment polarity annotations. Words representing positive sentiment are assigned positive values, and words representing negative sentiment are assigned negative values. Let the sentiment value be C. For each word, based on its TF-IDF value and C, a final representation value is assigned, as shown below:
[0049]
[0050] Therefore, a feature matrix is generated for user comment data, where each element represents the value of a word. A value greater than 0 and the larger the value, the stronger the positive sentiment, while a value less than 0 and the smaller the value, the stronger the negative sentiment. Sentiment labels are then assigned to these elements.
[0051] Step 3: Constructing a feature-based classification model using genetic model algorithms
[0052] Step 3.1 Gene encoding and population initialization
[0053] Gene encoding: Encode candidate text features into binary form, where 0 indicates that the feature is selected and 1 indicates that it is not selected.
[0054] Population initialization: In the initial stage of the genetic algorithm, we generate a population containing multiple individuals. Each individual represents a feature selection scheme, that is, the selection state of a set of features.
[0055] Step 3.2 Calculate the fitness function
[0056] Step 3.2.1 Calculate the number of features to be included.
[0057] First, consider the number of features included. We aim to select fewer features to improve model efficiency. Therefore, the fitness of an individual is negatively influenced by the number of selected features. Let the number of included features be N; this is the first part of the fitness function.
[0058] Step 3.2.2 Building a prediction model based on machine learning
[0059] A sentiment analysis model is constructed using multiple base models, such as Naive Bayes, Support Vector Machines, Decision Trees, and Random Forests, and their accuracy-weighted averages are calculated. At each iteration, 10-fold cross-validation and grid search are performed on each model, allowing for fine-tuning of model parameters to maximize performance and prediction accuracy. The weighted average accuracy of all models is used as the second part of the fitness function, assigning different weights based on the performance of different base models. Let the weighted average accuracy be...
[0060] Step 3.2.3 Calculate the variance of the accuracy of each base model.
[0061] To ensure the stability of the selected features across all models, the variance of the accuracy of each base model is calculated. This measures the consistency of performance among different base models; a smaller variance indicates more stable performance across models. Therefore, this metric is the third part of the fitness function. Let the variance of the accuracy of each model be...
[0062] Step 3.2.4 Create the fitness function
[0063] fitness function f(x) i ), where x i Representing different feature selection schemes, i.e., different individuals, this fitness function includes three key aspects. The accuracy-weighted average is calculated based on the number of features included, N. And the variance of accuracy is Create the following fitness function:
[0064]
[0065] Where N i This represents the number of features included for each individual. Since the number of included features is greater than the weighted average of the model, and we want the number of included features to be as small as possible, therefore, for N... i Perform a logarithmic transformation with base a and then take the opposite number. V represents the weighted average accuracy of different base models for each individual. i This represents the variance of the base model accuracy for each individual. We want the variance to be as small as possible, so we assign it a negative weight. To ensure a decent overall model performance, for For individuals with high fitness, we set their fitness to 0, meaning that we eliminate such individuals in each genetic iteration and retain those with higher fitness, i.e., those with higher overall accuracy.
[0066] Step 3.3 Selecting an operator
[0067] The linear sorting selection operator is used to determine which individuals are chosen to reproduce the next generation. This method considers not only fitness values but also the relative order among individuals. First, individuals are sorted from highest to lowest fitness value, and each individual is assigned a sequence number i. Assuming there are N individuals, the first-ranked individual with the highest fitness has i = N, the second-ranked individual has i = N-1, and so on, with the last individual having i = 1. The formula is:
[0068]
[0069] Let P max =0.9, P min =0.1, but different values can be set according to the actual application. Therefore, the probability that the individual with the highest fitness ranking is selected is P. max The probability of the last person being selected is P. min .
[0070] Step 3.4 Crossover Mutation
[0071] We use the concept of population adaptation to set the rules for crossover and mutation. First, we preset a crossover probability and a mutation probability. To find the optimal solution, in each iteration, we calculate the weighted average of the accuracy of each individual. We perform calculations and sort individuals from high to low. Based on the crossover probability x and the number of individuals N, we select individuals ranked N×x at the bottom for pairwise crossover to generate a new generation. The remaining individuals are directly copied to the next generation.
[0072] The mutation probability is a very small value. Similarly, for the mutation probability y and the number of individuals N, we select the individuals ranked N×y last to undergo random point mutation. In this study, we convert the code to 0 and 1 to represent the selection of different feature combinations.
[0073] Step 3.5 Population Iteration
[0074] Through multiple iterations, the optimal combination of textual features and weight allocation will be found to build a high-performance sentiment classification model and obtain user review classification results. This result will help analysts better understand users' sentiments towards products, thereby improving product quality or marketing strategies.
Claims
1. A text sentiment classification method based on genetic algorithms, characterized in that: Step 1: Data preparation and preprocessing; obtain online text data and corresponding sentiment classification tags, and then perform text preprocessing on the text data; Step 2, Feature Preprocessing: Perform text vectorization, converting the text into numerical feature vectors so that the model can process and understand it, improving the quality of text data and ensuring that the model can efficiently process and analyze text information; Step 3: Construct a model based on a genetic model algorithm; This step uses a genetic algorithm to encode features in binary form and initialize the population. Calculate the fitness function, including the number of features included, the performance of multiple machine learning models, and the variance of the accuracy of each model; use a linear ranking selection operator to determine which individual is selected for reproduction. Using the concept of population adaptation, individuals ranked lower by the weighted average of accuracy are selected based on preset crossover and mutation probabilities, and crossover and mutation operations are performed to generate the next generation of individuals; after iterating to a specified number of times, the final result is obtained. Step 3: Constructing a feature-based classification model using genetic model algorithms Step 3.1 Gene encoding and population initialization Gene encoding: Features are encoded for use in genetic algorithms; features are encoded using binary encoding, where 0 represents inclusion of the feature and 1 represents exclusion; the main purpose of this step is to create a population for the feature selection problem, where each individual represents a feature selection scheme; for the feature set obtained in step 2, each feature is represented as a binary number, where 1 represents selection of the feature and 0 represents exclusion. Population initialization: In the initial stage of the genetic algorithm, a population containing multiple individuals is generated; each individual represents a feature selection scheme, that is, the selection state of a set of features. Step 3.2 Calculate the fitness function Step 3.2.1 Calculate the number of features to be included. Design a fitness function ,in Representing different feature selection schemes, i.e., different individuals, this fitness function includes three key aspects; first, it aims to minimize the number of selected features, let the number of included features be... The number of different individual inclusion features is Fewer features help reduce data dimensionality and computational complexity; Step 3.2.2 Building a prediction model based on machine learning Logistic Regression (LR), Decision Tree (DT), Random Forest (RF), Support Vector Machine (SVM), K-Nearest Neighbors (KNN), and Naive Bayes (NB) were used as base models. At each iteration, 10-fold cross-validation and grid search were performed on each model. The weighted average accuracy of all models was used as the second part of the fitness function, with different weights assigned based on the performance of different base models. The weighted average accuracy was set to... ; Step 3.2.3 Calculate the variance of the accuracy of each base model. This is achieved by calculating the variance of the accuracy of each base model, which measures the consistency of performance among different base models. The smaller the variance, the more stable the performance of each model. Therefore, this index is the third part of the fitness function. Let the variance of the accuracy of each model be... ; Step 3.2.4 Create the fitness function Accuracy weighted average based on the number of features included N. And the variance of accuracy is Create the following fitness function: ;in This represents the number of features included for each individual. Since the number of included features is greater than the weighted average of the model, and we want the number of included features to be as small as possible, therefore... Perform a logarithmic transformation with base a and then take its opposite. This represents the weighted average accuracy of different base models for each individual. This represents the variance of the baseline model accuracy for each individual. We want the variance to be as small as possible, so it's assigned a negative weight. To ensure a decent overall model performance, for... Individuals with a fitness of ≤0.6 are set to 0, meaning that they are eliminated in each genetic iteration, while individuals with higher fitness are retained, i.e., individuals with higher overall accuracy. Step 3.3 Selecting an operator The linear sorting selection operator is used to determine which individual is selected to reproduce the next generation of new individuals. First, individuals are sorted from high to low according to their fitness values, and each individual is assigned a serial number i. Assuming there are N individuals in total, the first-ranked individual with the highest fitness has i=N, the second-ranked individual has i=N-1, and so on, with the last individual having i=1. The formula is: ； set up = 0.9, = 0.1; therefore, the probability of the individual with the highest fitness ranking being selected is . The probability of the last person being selected is ; Step 3.4 Crossover Mutation The rules for crossover and mutation are set using the concept of population adaptation. First, a crossover probability and a mutation probability are preset. In each iteration, the weighted average of the accuracy of each individual is calculated. Perform calculations and sort from high to low based on crossover probability. Given the number of individuals N, select the ones ranked last. Individuals cross over each other to produce a new generation, while the remaining individuals directly replicate into the next generation; For the probability of mutation Given the number of individuals N, select the ones ranked last. Randomly mutating an individual involves converting the code to 0 and 1, representing the selection of different feature combinations. Step 3.5 Population Iteration After crossover and mutation operations, new individuals are generated, which will replace some of the individuals in the previous generation; the new population will be used for fitness calculations and further iterations in the next generation. This process will be repeated for multiple generations, gradually evolving and optimizing the population. After iterating a specified number of times, the iteration will stop, and finally the optimal feature selection scheme and weight allocation will be found.
2. The method according to claim 1, characterized in that: Step 1: Data preparation and preprocessing We acquire online text data and corresponding sentiment classification tags, and then perform text preprocessing. First, we remove special symbols and punctuation marks from the text, segment the text and remove stop words, and then perform stemming or word form restoration to convert words into their basic forms and reduce vocabulary diversity. Second, we process data with missing values or blank fields.
3. The method according to claim 1, characterized in that: Step 2: Feature Preprocessing The text is converted into a feature vector using the TF-IDF method, where each row represents a text sample, each column represents a word, and each element in the matrix represents the TF-IDF value of the corresponding word in the corresponding text. Then, a sentiment score is assigned to each word, and a sentiment dictionary is obtained containing a large number of words and their sentiment polarity labels. Words representing positive sentiment are assigned positive values, and words representing negative sentiment are assigned negative values. Let the sentiment value be C. For each word, a final representation value is assigned based on the TF-IDF value and C, as shown below: ； Therefore, a feature matrix is generated for the text file, where each element represents the value of a word. Values greater than 0 and larger values indicate a stronger tendency towards positive emotions, while values less than 0 and smaller values indicate a stronger tendency towards negative emotions. The text is then assigned a sentiment label.