Text sentiment analysis method based on particle swarm optimization regularized extreme learning machine

Through the particle swarm algorithm, the regularized extreme learning machine is optimized, and the problem of overfitting and hyperparameters are difficult to determine in Chinese short text sentiment analysis is solved, and high-precision and efficient text sentiment classification is achieved.

CN114880465BActive Publication Date: 2024-11-05XIAN UNIV OF POSTS & TELECOMM
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202210313143.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-28
Publication Date
2024-11-05
Estimated Expiration
2042-03-28

AI Technical Summary

Technical Problem

In the previous technology, in the sentiment analysis of short text in Chinese, the extreme learning machine has overfitting problems, the number of hidden layer nodes is difficult to determine, and the selection of activation function affects the generalization ability, resulting in insufficient classification accuracy and operation efficiency.

Method used

The particle swarm algorithm is used to optimize the regularization limit learning machine, and the parameter optimization is optimized by initializing random particles, determining the optimal hyperparameter, and constructing the particle swarm algorithm optimized regularization limit learning machine model for text emotion classification.

Benefits of technology

It improves the accuracy and operation efficiency of text emotion classification, avoids overfitting, and ensures good generalization performance and compact network structure of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114880465B_ABST
    Figure CN114880465B_ABST
Patent Text Reader

Abstract

The present invention discloses a text sentiment analysis method based on a particle swarm algorithm-optimized regularized extreme learning machine. The method comprises the following steps: (1) obtaining text information of data to be analyzed, and preprocessing the text information to obtain text vocabulary; (2) training the text vocabulary with a skip-gram model based on a corpus of data sets in the same field to obtain a text word vector; (3) initializing the parameters of the regularized extreme learning machine to a group of random particles using a particle swarm algorithm, and then determining the optimal parameters through iteration to obtain a particle swarm algorithm-optimized regularized extreme learning machine; (4) inputting the text word vector into the particle swarm algorithm-optimized regularized extreme learning machine to perform sentiment classification on the text information. The present invention proposes to use a particle swarm algorithm to optimize three hyperparameters of the regularized extreme learning machine within a certain range. While improving the calculation accuracy, the network structure is made more compact. In addition, the good generalization performance of the model is guaranteed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of sentiment analysis in natural language processing, and in particular to a text sentiment analysis method based on a particle swarm algorithm-optimized regularized extreme learning machine. Background Art

[0002] With the development of the Internet, many people have become accustomed to expressing their opinions on something or a product with emotional comments. However, the massive amount of comment data makes it more difficult to manually judge the emotional tendency of comments. Therefore, how to automatically and quickly perform sentiment classification on the comment text input by the user to obtain valuable output has become a hot topic. The comment text not only has high feature dimensions, irregular expressions, but also large scale. Therefore, it is also very important to reduce the time and computation cost while improving the classification accuracy of the text sentiment classification model.

[0003] In existing research, sentiment lexicon, machine learning or neural network has been widely considered to be an effective method for sentiment classification. As an important basis for judging the sentiment polarity of comments, sentiment lexicon needs to establish relevant sentiment lexicon for specific fields to improve the accuracy of classification; machine learning algorithm needs to select sentiment words as feature words and classify the text after matrixing, and the processing speed is faster than neural network. Neural network processes information by simulating the thinking mode of human brain. Applying it to sentiment classification can greatly improve the classification performance. With the application of convolutional neural network (CNN) to sentence classification, many researchers began to explore the application of deep neural network to sentiment classification of text and achieved good results. At the same time, in order to improve classification accuracy and operation efficiency, many researchers have focused on extreme learning machine. The biggest feature of extreme learning machine is that the threshold between hidden layer nodes and the weight between different layers only need to be calculated by least squares method once, without back propagation for update. Applying extreme learning machine to short text sentiment analysis can not only greatly shorten the running time, but also be more suitable for unevenly distributed text data than other methods.

[0004] In the study of sentiment analysis of Chinese short texts, the method based on sentiment dictionary needs to establish different sentiment dictionary libraries for different contexts and different semantics. The process of establishing sentiment dictionary requires a lot of manpower, which is often time-consuming and very inconvenient. Based on the traditional machine learning method, its classification performance is too dependent on the data scale and is suitable for processing small data sets. For the classification of large data sets, machine learning often has low classification accuracy and high computational cost. For deep neural networks such as convolutional neural networks (CNN) and long short-term memory networks (LSTM), it is often a waste of time and has high complexity. Training requires many parameters and the cost of parameter adjustment is high. At the same time, there are problems of gradient disappearance and gradient explosion. Compared with the problems of the above methods, the extreme learning machine not only has a fast processing speed, but also has a good classification effect. However, the extreme learning machine does not fully consider the structured risk, which may lead to overfitting problems. The regularized extreme learning machine model introduces a regularization term on the basis of the standard quadratic form of the extreme learning machine, which improves the generalization ability of the extreme learning machine while avoiding overfitting. However, the direct use of regularized extreme learning machines has the following problems: 1) The network structure of the regularized extreme learning machine increases with the increase in the number of Internet users, so the scale of text data in the network is increasing day by day. Therefore, it is an important consideration for the comprehensive performance evaluation of an algorithm to be able to capture and mine useful information from a large amount of text data in real time. From the perspective of model time performance, the present invention adopts an important factor affecting its generalization ability. Its hidden layer nodes are obtained by random initialization, and the network structure is trained by solving the output layer weights using the least squares method. Although the problem of slow convergence speed and easy to fall into the local optimal value is overcome to a certain extent, the number of hidden layer nodes still needs to be determined manually, which is not only cumbersome, but also cannot guarantee the optimal and suboptimal structures. 2) As one of the important hyperparameters in the regularized extreme learning machine network, the activation function can not only help the neural network activate useful information, but also suppress irrelevant data points. The selection of the activation function is related to the calculation accuracy and execution efficiency of the entire neural network. Different activation functions will show their own advantages and disadvantages under different input data and different neural networks. 3) If the number of hidden layers is too large and the input parameters are random, the generalization ability will be greatly reduced. Therefore, it is generally necessary to select an optimal regularization coefficient so that the model can achieve better results on different data sets. Summary of the invention

[0005] The purpose of the present invention is to overcome the above-mentioned deficiencies in the prior art and provide a text sentiment analysis method based on particle swarm algorithm optimization regularized extreme learning machine. It is a model for Chinese text sentiment analysis with high classification accuracy and fast execution speed.

[0006] In order to achieve the above-mentioned object of the invention, the present invention provides the following technical solutions:

[0007] The text sentiment analysis method based on particle swarm algorithm optimization regularized extreme learning machine includes the following steps:

[0008] Acquire text information of data to be analyzed, and preprocess the text information to obtain text vocabulary;

[0009] The text vocabulary is trained with a skip-gram model based on a corpus of data sets in the same field to obtain a text word vector;

[0010] The particle swarm algorithm is used to initialize the parameters of the regularized extreme learning machine into a group of random particles, and then the optimal parameters are determined through iteration to obtain the particle swarm algorithm optimized regularized extreme learning machine;

[0011] The text word vector is input into a particle swarm algorithm-optimized regularized extreme learning machine to perform sentiment classification on the text information.

[0012] In the technical solution of the present invention, preprocessing and vectorization are first performed, and a corpus of data sets in the same field is used for text vectorization; secondly, in the classification stage, the existing technology ignores the execution speed while improving the classification accuracy, and the overfitting problem of the extreme learning machine when performing short text sentiment classification. The present invention introduces a regularization term based on the standard quadratic form of the extreme learning machine, and uses the regularized extreme learning machine as the main model for Chinese text sentiment classification. Finally, in view of the difficulty in determining the number of iterations in the process of Chinese short text sentiment analysis directly using the regularized extreme learning machine, the easy fall into the local optimum, and the strong dependence on the initial value when optimizing the weights and thresholds using the particle swarm optimization algorithm, the present invention proposes three hyperparameter optimization methods for the regularized extreme learning machine using the particle swarm algorithm. The present invention sets the search range of three hyperparameters and searches randomly. While improving the calculation accuracy, the network structure is made more compact, ensuring the good generalization performance of the model.

[0013] Furthermore, the preprocessing includes deduplication, denoising, word segmentation, and word removal.

[0014] Furthermore, after the preprocessing, a frequency analysis is performed on the text vocabulary to evaluate the importance of a word to a data set.

[0015] Furthermore, the specific steps for establishing a corpus of data sets in the same field are:

[0016] S21, classifying the text words according to parts of speech to obtain a segmented word set;

[0017] S21, using the skip-gram model to train word vectors for each word in the word segmentation set;

[0018] S23. Select the high-frequency words in the word vector after processing in step S22, and construct a corpus of data sets in the same field.

[0019] Furthermore, for a training set containing N different samples where x i =[x i1 ,x i2 ,...,x in ] T ∈R n , k i =[k i1 ,k i2 ,...,k im ]∈R m , the objective function of the regularized extreme learning machine is:

[0020]

[0021] Where L is the number of neurons in the hidden layer of the regularized extreme learning machine, β is the output weight, C is the regularization coefficient, N is the number of samples in the training set, H is the hidden layer output matrix, and T is the transposed symbol.

[0022] Furthermore, according to the KKT theorem (Karush–Kuhn–Tucker conditions), two expressions of β can be obtained:

[0023]

[0024] Where I is the unit matrix, C is the regularization coefficient, H is the hidden layer output matrix, and N is the number of training set samples.

[0025] Furthermore, the specific steps of using particle swarm optimization to optimize the regularized extreme learning machine are:

[0026] S31, setting the range of the number of hidden layer nodes, the reciprocal range of the regularization coefficient and the activation function, and determining the maximum number of iterations;

[0027] S32, using uniform distribution to initialize a series of particles to form a population and obtain initial hyperparameters;

[0028] S33. A regularized extreme learning machine model is established according to the initial hyperparameters, and the particles are trained using a five-fold cross validation method and the auc value is calculated according to the fitness function, and the auc value is used as the fitness value of each particle; the fitness function is: maxB = ω × auc, ω∈[0,1];

[0029] S34, judging whether the termination condition is met, the termination condition being the maximum auc value or reaching the maximum number of iterations;

[0030] If the termination condition is not met, the optimal value p of the particle is determined best and the population optimal value g best , update the particle position according to the position and velocity formula of particle i in the d-dimensional space, obtain the updated hyperparameters, and continue step S33 with the updated hyperparameters, and update the position and velocity formula of the particle position as follows:

[0031] ν i (t+1)=ω·ν i (t)+c1rand()·(p best -x i (t))+c2rand()·(g best -x i (t))

[0032] x i (t+1)=x i (t)+ν i (t)

[0033] Where ν i and x i are the velocity and position of particle i at time t, c1 and c2 are learning factors, rand() is a random number between 0 and 1, and ω is the inertia weight;

[0034] If the termination condition is met, the optimal particle position is output;

[0035] S35. A particle swarm algorithm is used to optimize a regularized extreme learning machine based on the hyperparameters corresponding to the output optimal particle position.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] 1. In the text vectorization stage, the present invention uses the same-field data set to establish a corpus to address the semantic gap problem between different fields in the commonly used corpus and the sparsity problem of short texts, which improves the classification accuracy of Chinese text sentiment analysis.

[0038] 2. In the classification stage, the present invention aims to solve the problem that the existing technology ignores the execution speed while improving the classification accuracy and the overfitting problem of the extreme learning machine when performing short text sentiment classification. The present invention introduces a regularization term based on the standard quadratic form of the extreme learning machine and adopts the regularized extreme learning machine as the main model for text sentiment classification, which greatly improves the operating efficiency of the classification model.

[0039] 3. The present invention aims at the problems that the number of iterations is difficult to determine, the dependence on the initial value is strong, and the problem of being easily trapped in the local optimum when directly using the regularized extreme learning machine to perform sentiment analysis on Chinese short texts. The existing technology optimizes the threshold between the hidden layer nodes and the weights between different layers of the regularized extreme learning machine when optimizing the regularized extreme learning machine. The present invention proposes to use the particle swarm algorithm to optimize the three hyperparameters of the regularized extreme learning machine within a certain range. While improving the calculation accuracy, the network structure is made more compact. In addition, the good generalization performance of the model is guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A flowchart of a text sentiment analysis method based on particle swarm optimization regularized extreme learning machine;

[0041] Figure 2 This is the word cloud analysis diagram of the waimai_10k dataset;

[0042] Figure 3 A process diagram for building a corpus of datasets in the same field;

[0043] Figure 4 This is a comparison chart of the classification effects of different corpora;

[0044] Figure 5 A confusion matrix heat map of a data set classified using the analysis method of the present invention;

[0045] Figure 6 This is a comparison chart of CPU time for different models. DETAILED DESCRIPTION

[0046] The present invention is further described in detail below in conjunction with test examples and specific implementation methods. However, this should not be understood as the scope of the above subject matter of the present invention being limited to the following embodiments, and all technologies realized based on the content of the present invention belong to the scope of the present invention.

[0047] Example 1

[0048] The text sentiment analysis method based on particle swarm algorithm optimization regularized extreme learning machine includes the following steps:

[0049] S1. Obtain text information of data to be analyzed, and preprocess the text information to obtain text vocabulary.

[0050] The data set used in this embodiment is a public data set downloaded from github. Two data sets are analyzed, namely the ChnSentiCorp_htl_all hotel field data set and the waimai_10k food delivery platform data set. First, 10,000 files of each data set are integrated into one file. Then, the label of negative comments is changed from -1 to 0 to obtain the text information of the data to be analyzed. Then, text preprocessing is performed. Text preprocessing is to further process the original data to make the data an object that can be operated. The preprocessing includes deduplication, denoising, word segmentation, and word removal. The specific steps of text preprocessing are as follows:

[0051] S11. Traverse and remove duplicate words in the Chinese comment text.

[0052] S12. De-noise some garbled codes and special characters to reduce complexity.

[0053] S13. Use the pkuseg (open source word segmentation tool of Peking University) word segmentation tool in the Python language environment to perform word segmentation, that is, to separate the evaluations in the text information according to the parts of speech.

[0054] S14, remove words, that is, remove modal particles, adverbs, prepositions, conjunctions, etc. in the text.

[0055] The data after preliminary processing is shown in Table 1.

[0056] Table 1 Data after preliminary processing

[0057] Dataset name Number of positive comments (label 1) Number of negative comments (label 0) ChnSentiCorp_htl_all 5322 2444 Waimai_10k 4000 8000

[0058] In some embodiments, after the text is preprocessed, the importance of a word to a data set is evaluated by performing a frequency analysis on the words that appear most frequently in the text. Figure 2 As shown in the figure, they are the statistical word clouds of negative and positive texts in the waimai_10k dataset. Among them, the most common words in the positive text are "good", "send", "delicious", and "good", with 1330, 1138, 943, and 906 appearances respectively. The most common words in the negative text are "send", "no", "hour", and "slow", with 3802, 1487, 1299, and 1259 appearances respectively. This shows that the main reason for the positive sentiment tendency of takeaway review texts is its taste and delivery speed, which provides a direction for improvement for takeaway platforms and merchants, and also provides guidance for buyers when choosing takeaway platforms. This is also the significance of Chinese text sentiment analysis.

[0059] S2. The text vocabulary is trained with a skip-gram model based on a corpus of data sets in the same field to obtain a text word vector.

[0060] Before the computer uses various algorithms to perform text sentiment classification, the text must be converted into a vector form. The word vector model is the most commonly used model in text representation. The basic idea of ​​the word vector model is to convert words into multi-dimensional real vectors, so that similar words will have similar vectors, which makes it much easier to mine the features between words and sentences. Most of the existing studies use word2vec to represent text. It has two implementation methods, among which skip-gram is more suitable for representing unevenly distributed text data, so the present invention chooses this method to train word vectors. When training word vectors, most scholars use public large corpora to train word vectors. Some studies have analyzed and compared the effects of pre-training word vectors of large commonly used corpora such as "Wikipedia" and "Today's Headlines" on the results of text sentiment classification, and found that the word vector model pre-trained using Wikipedia has the best general effect. However, the word vector model trained using the Wikipedia corpus has the problem of semantic gap between different fields and the sparsity problem of short text. Therefore, the present invention directly uses the same domain data set to establish a corpus and calls the gensim package in python to train its own word vector corpus. The specific steps for establishing the same domain data set corpus are:

[0061] S21, classifying the text words according to parts of speech to obtain a segmented word set;

[0062] S21, using the skip-gram model to train word vectors for each word in the word segmentation set;

[0063] S23. Select the high-frequency words in the word vector after processing in step S22, and construct a corpus of data sets in the same field.

[0064] In step S23, after vectorization, the word vectors of the selected high-frequency words are saved in a txt file to construct their own corpus, that is, a corpus of data sets in the same field is obtained, such as Figure 3 The figure shows the process of using the ChnSentiCorp_htl_all hotel domain dataset to build a corpus of the same domain dataset. The constructed corpus is more targeted at the sentiment classification problem of the two review data, and it handles and solves the semantic gap problem between different domains and the sparsity problem of short texts that exist in the existing Chinese corpus. After the corpus is constructed, the similarity coefficients between words can be further observed. As shown in Table 2, the top five words most related to "delicious" in the waimia_10k dataset and their similarity coefficients are listed.

[0065] Table 2 Similarity coefficients of similar words

[0066] Similar words Similarity coefficient Yes 0.7378 good 0.7054 recommend 0.6633 Great 0.6436 beautiful 0.6342

[0067] S3. Using the particle swarm algorithm, the parameters of the regularized extreme learning machine are initialized to a group of random particles, and then the optimal parameters are determined through iteration to obtain the regularized extreme learning machine optimized by the particle swarm algorithm.

[0068] The present invention uses a regularized extreme learning machine to complete the classification of Chinese text sentiment analysis. The regularized extreme learning machine model used in the present invention introduces a regularization term on the basis of the standard quadratic form of the extreme learning machine, which improves the generalization ability of the extreme learning machine while avoiding overfitting. For a training set containing N different samples where x i =[x i1 ,x i2 ,...,x in ] T ∈R n , k i =[k i1 ,k i2 ,...,k im ]∈R m , the objective function of the regularized extreme learning machine is:

[0069]

[0070] Constraints: sth(x i )β=k i T -ξ i T (2)

[0071] Where L is the number of neurons in the hidden layer of the regularized extreme learning machine, β is the output weight, C is the regularization coefficient, N is the number of samples in the training set, and ξ i is the training error of the i-th sample, h(x i ) is the hidden layer relative to x i The output vector of is the hidden layer output matrix, and T is the transposed symbol.

[0072] Substituting (2) into (1), we can obtain the following equivalent optimization problem:

[0073]

[0074] According to the KKT theorem, two expressions of β can be obtained:

[0075]

[0076] Where I is the unit matrix, C is the regularization coefficient, H is the hidden layer output matrix, and N is the number of training set samples.

[0077] However, there are the following problems with directly using regularized extreme learning machines: 1) Although it overcomes the problem of slow convergence and easy to fall into local optimal values ​​to a certain extent, it still requires manual determination of the number of hidden layer nodes, which is not only cumbersome, but also cannot guarantee the optimal and suboptimal structures. 2) The selection of the activation function is related to the calculation accuracy and execution efficiency of the entire neural network. Different activation functions will show their own advantages and disadvantages under different input data and different neural networks. 3) If the number of hidden layers is too large and the input parameters are random, the generalization ability will be greatly reduced.

[0078] Therefore, it is necessary to select an optimal regularization coefficient so that the regularized extreme learning machine model can achieve better results on different data sets. The present invention uses a particle swarm algorithm to perform hyperparameter optimization on the regularized extreme learning machine. Hyperparameters refer to parameters that need to be set in advance during model definition and training, which determine whether the model performance training can be successful or optimal. Finding the most suitable hyperparameter combination for different models is a complex task. The particle swarm optimization algorithm is a parameter combination that optimizes the model performance. Each parameter in the model has a range that needs to be searched. The particle swarm algorithm starts with multiple possible parameter combinations, randomly initializes and iterates multiple times. In each iteration, each particle is decoded into a set of parameters. When the particle swarm algorithm terminates, the best solution, that is, the model parameter combination with the best performance, will be returned. When no improvement is observed in a set of consecutive iterations, the optimization process stops. The process of hyperparameter optimization of the regularized extreme learning machine using the particle swarm algorithm is as follows Figure 5 shown.

[0079] The specific steps of using particle swarm optimization to optimize the regularized extreme learning machine are:

[0080] S31, setting the range of the number of hidden layer nodes, the reciprocal range of the regularization coefficient and the activation function, and determining the maximum number of iterations;

[0081] S32, using uniform distribution to initialize a series of particles to form a population and obtain initial hyperparameters;

[0082] S33. A regularized extreme learning machine model is established according to the initial hyperparameters, and the particles are trained using a five-fold cross validation method and the auc value is calculated according to the fitness function, and the auc value is used as the fitness value of each particle; the fitness function is: maxB = ω × auc, ω∈[0,1];

[0083] S34, judging whether the termination condition is met, the termination condition being the maximum auc value or reaching the maximum number of iterations;

[0084] If the termination condition is not met, the optimal value P of the particle is determined best and the population optimal value g best , update the particle position according to the position and velocity formula of particle i in the d-dimensional space, obtain the updated hyperparameters, and continue step S33 with the updated hyperparameters, and update the position and velocity formula of the particle position as follows:

[0085] ν i (t+1)=ω·ν i (t)+c1rand()·(p best -x i (t))+c2rand()·(g best -x i (t))

[0086] x i (t+1)=x i (t)+ν i (t)

[0087] Where ν i and x i are the velocity and position of particle i at time t, c1 and c2 are learning factors, rand() is a random number between 0 and 1, and ω is the inertia weight;

[0088] If the termination condition is met, the optimal particle position is output;

[0089] S35. A particle swarm algorithm is used to optimize a regularized extreme learning machine based on the hyperparameters corresponding to the output optimal particle position.

[0090] This embodiment uses the optunity package containing the particle swarm algorithm in Python to define the parameter fine-tuning problem of the regularized extreme learning machine as an optimization problem under constraints, with the goal of finding the optimal hyperparameter value to maximize the AUC value. The parameter settings in the optimization process are shown in Table 3 below.

[0091] Table 3 Parameter settings of regularized extreme learning machine optimized by particle swarm algorithm

[0092] parameter Value Activation function type Sigmoid,tanh,relu,softplus Regularization coefficient inverse range [1,10] Hidden layer node number range [1,300] Word vector dimension 300 Window length 5 Maximum number of iterations 150 Number of cross validations 5

[0093] S4. Input the text word vector into a particle swarm algorithm-optimized regularized extreme learning machine to perform sentiment classification on the text information.

[0094] The method of sentiment classification of the two data sets by the present invention is represented by M10. In order to verify the classification performance of the model in this paper, the text information of the two data sets is classified by the analysis methods of M1-M9 respectively. The specific analysis methods of M1-M9 are as follows:

[0095] M1: The text information is processed using steps S1 and S2 of the present invention, and sentiment classification is performed using a decision tree algorithm. The decision tree algorithm starts from a root node, tests the characteristic attributes in the item to be classified, and selects branches according to the values ​​until a leaf node, and the category stored on the leaf node is used as the final classification result.

[0096] M2: The text information is processed using steps S1 and S2 of the present invention, and the Gaussian Naive Bayes algorithm is used. Bayes) for sentiment classification. The Gaussian Naive Bayes algorithm mainly completes the text sentiment classification work based on probability calculation.

[0097] M3: The text information is processed by steps S1 and S2 of the present invention, and sentiment classification is performed by random forest. Random forest is a classifier that includes many decision trees. Multiple trees are used to train samples, and then the classification is determined based on the majority of the output categories of individual trees.

[0098] M4: The text information is processed using steps S1 and S2 of the present invention, and sentiment classification is performed using a regularized extreme learning machine (RELM).

[0099] M5: The text information is processed by steps S1 and S2 of the present invention, and sentiment classification is performed by support vector machine (SVM). The support vector machine is a generalized linear classifier that performs binary classification on data in a supervised learning manner, and its decision boundary is the maximum margin hyperplane solved for the learning samples.

[0100] M6: The text information is processed by steps S1 and S2 of the present invention, and SVM+emotional dictionary+NB is used for sentiment classification. During classification, the naive Bayesian classification method based on the emotional dictionary is used for sentiment polarity classification. Finally, the method based on the emotional dictionary and integrated learning is combined.

[0101] M7: The text information is processed using steps S1 and S2 of the present invention, and sentiment classification is performed using a one-dimensional convolutional neural time series (Conv1D). The optimizer is Adam, the learning rate is 0.0001, the loss function uses cross entropy, and the number of epochs is set to 50.

[0102] M8: The text information is processed using steps S1 and S2 of the present invention, and sentiment classification is performed using a long short-term memory network (LSTM).

[0103] M9: The text information is processed by step S1 of the present invention, the skip-gram model is used to train word vectors based on the Wikipedia corpus, and the particle swarm algorithm is used to perform sentiment classification on the regularized extreme learning machine.

[0104] The Chinese text sentiment classification results of word vectors trained with the commonly used Wikipedia corpus and the self-built corpus of the same domain dataset are compared, where C1 is the Wikipedia corpus and C2 is the corpus of the same domain dataset, such as Figure 4 As shown. By comparing the classification effects of word vectors trained with different corpora, it can be found that the classification effect of the word vector model pre-trained with the same-domain dataset corpus is better than that of the word vector trained with the commonly used Wikipedia corpus. If you want to perform sentiment classification on text in a specific field, using a dataset related to the field or directly using the dataset to build a corpus of the same-domain dataset and train word vectors will be more effective and more targeted.

[0105] In order to more intuitively and effectively demonstrate the effect of applying the particle swarm optimization regularized extreme learning machine model to Chinese text sentiment analysis, the present invention uses the python seaborn library in the experimental part to visualize the results on two data sets. Table 4 and Table 5 show the classification effects of different analysis methods on the two data sets. The analysis method of the present invention has a higher AUC value. The classification accuracy P, recall rate R, and F1 value in the table are all macro means. TP (True Positive) means that the positive class is predicted correctly; FP (False Positive) means that the positive class is predicted incorrectly; FN (False Negative) means that the negative class is predicted incorrectly; TN (True Positive) means that the negative class is predicted correctly.

[0106] Table 4 Classification effects of different analysis methods on ChnSentiCorp_htl_all

[0107]

[0108]

[0109] Table 5 Classification effects of different analysis methods on waimai_10k

[0110] Model P R F1 auc value M1 66.34 67.01 66.67 68.98 M2 76.35 76.52 76.43 66.77 M3 73.31 80.01 76.51 74.31 M4 79.83 81.05 80.43 83.39 M5 81.15 79.32 80.22 82.12 M6 - 79.45 78.59 80.12 M7 80.45 81.49 80.93 81.23 M8 80.00 82.01 81.23 82.11 M9 83.56 81.96 83.02 87.53 M10 86.58 85.87 86.72 90.39

[0111] Confusion matrix thermal analysis after particle swarm optimization regularization on the ChnSentiCorp_htl_all dataset Figure 5As shown in the left figure, there are 1234 positive comments that are correctly classified and 198 negative comments that are incorrectly classified; there are 394 negative comments that are correctly classified and 115 positive comments that are incorrectly classified. Among them, the classification accuracy of category 0 (i.e. negative comments) is 0.77, the recall rate is 0.67, and the F1 value is 0.72; the classification accuracy of category 1 (i.e. positive comments) is 0.86, the recall rate is 0.92, and the F1 value is 0.89. On the waimai_10k dataset, the confusion matrix heat map after the particle swarm optimization regularized extreme learning machine is as follows Figure 5 As shown in the right figure of . There are 769 correctly classified positive comments, 158 incorrectly classified negative comments, 1833 correctly classified negative comments, and 233 incorrectly classified positive comments. Among them, the classification accuracy of category 0 (i.e. negative comments) is 0.89, the recall rate is 0.92, and the F1 value is 0.90; the classification accuracy of category 1 (i.e. positive comments) is 0.83, the recall rate is 0.77, and the F1 value is 0.80.

[0112] When using a model for classification, the time it takes for the CPU to complete the classification or prediction task at full speed often determines the time performance of the model. The longer the CPU time of the model, the worse the time performance of the model. Therefore, in order to compare and analyze the time performance of each model, the present invention records the CPU time of them performing classification tasks on the same data set, such as Figure 6 shown.

[0113] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A text sentiment analysis method based on particle swarm algorithm optimization regularized extreme learning machine, characterized in that: The following steps are involved: Acquire text information of data to be analyzed, and preprocess the text information to obtain text vocabulary; The text vocabulary is trained with the skip-gram model based on the same domain data set corpus to obtain the text word vector; the specific steps of establishing the same domain data set corpus are: S21, classifying the text words according to parts of speech to obtain a segmented word set; S22, using the skip-gram model to perform word vector training on each word in the word segmentation word set; S23, select the high-frequency words of the word vector after processing in step S22, and construct a corpus of data sets in the same field; The particle swarm algorithm is used to initialize the parameters of the regularized extreme learning machine to a group of random particles, and then the optimal parameters are determined through iteration to obtain the particle swarm algorithm to optimize the regularized extreme learning machine; the specific steps of using the particle swarm algorithm to optimize the regularized extreme learning machine are: S31, setting the range of the number of hidden layer nodes, the reciprocal range of the regularization coefficient and the activation function, and determining the maximum number of iterations; S32, using uniform distribution to initialize a series of particles to form a population and obtain initial hyperparameters; S33, establish a regularized extreme learning machine model according to the initial hyperparameters, use the five-fold cross validation method to train the particles and calculate the auc value according to the fitness function, and use the auc value as the fitness value of each particle; the fitness function is: ; S34, judging whether the termination condition is met, the termination condition being the maximum auc value or reaching the maximum number of iterations; If the termination condition is not met, the optimal value p of the particle is determined best and the population optimal value g best , update the particle position according to the position and velocity formula of particle i in the d-dimensional space, obtain the updated hyperparameters, and continue step S33 with the updated hyperparameters, and update the position and velocity formula of the particle position as follows: In the formula and are the velocity and position of particle i at time t, and is the learning factor, rand() is a random number between 0 and 1, is the inertia weight; If the termination condition is met, the optimal particle position is output; S35, constructing a particle swarm algorithm optimized regularized extreme learning machine according to the hyperparameters corresponding to the output optimal particle position; The text word vector is input into a particle swarm algorithm-optimized regularized extreme learning machine to perform sentiment classification on the text information.

2. The text sentiment analysis method based on particle swarm algorithm optimization regularized extreme learning machine according to claim 1, characterized in that: The preprocessing includes deduplication, denoising, word segmentation and word removal.

3. The text sentiment analysis method based on particle swarm algorithm optimization regularized extreme learning machine according to claim 1, characterized in that: For a training set containing N different samples ,in , , the objective function of the regularized extreme learning machine is: Where L is the number of neurons in the hidden layer of the regularized extreme learning machine, is the output weight, C is the regularization coefficient, N is the number of training set samples, H is the hidden layer output matrix, and T is the transposed symbol.

4. The text sentiment analysis method based on particle swarm algorithm optimization regularized extreme learning machine according to claim 3 is characterized in that: According to the KKT theorem, we can get Two expressions of: Where I is the unit matrix.

Citation Information

Patent Citations

  • Method and system for classifying electrocardiosignals based on PSO-ELM algorithm

    CN110755065A

  • Hyper-parameter optimization device and method

    CN111160459A

  • Intelligent building air conditioner cooler system operation optimization method based on PSO-ELM algorithm

    CN113268913A

  • Extended question recommendation method and device, computer equipment and storage medium

    CN113688636A