Aspect-level opinion mining method for water environment text based on multi-feature fusion

Abstract

The present invention discloses an aspect-level opinion mining method for water environment text based on multi-feature fusion, which performs feature extraction on collected water environment text. First, a multi-feature fusion model is constructed by combining the attention mechanism and the convolutional neural network to fully and accurately extract aspect words and sentiment features. On this basis, an aspect-level opinion mining model based on a gated linear unit and a bidirectional long short-term memory network is constructed. Through iterative training of the water environment text, the model parameters are optimized, and the trained opinion mining model is used to judge the sentiment polarity of different aspects of the water environment text. Finally, the classification results are compared with the test data set, and the aspect-level opinion mining model is subjected to error analysis and performance evaluation. The present invention comprehensively considers the relationship between the multiple features of the water environment text, the word order features of the context, and the local sentiment features, and effectively improves the accuracy of sentiment classification of each aspect of the word.

Description

Technical Field

[0001] The present invention relates to the field of aspect-level opinion mining in water environment texts, and in particular to a method for aspect-level opinion mining in water environment texts based on multi-feature fusion. Background Art

[0002] As my country's modernization continues to accelerate, technology continues to develop and advance, and the pace of technological innovation is accelerating, generating massive amounts of text data daily. With the popularity of Weibo and other social networking sites, a vast amount of emotional textual information is generated on social networks. Weibo is one of China's largest social networks. By the end of 2021, 550 million active users were sharing and disseminating information on Weibo. Extracting the emotional information contained in these texts can help us better capture users' emotions and opinions. Traditional analysis methods are no longer suitable for data with large volumes, complex structures, and varying data volumes. The real world generates a constant stream of data, which is largely unstructured, dynamic, and interconnected, and takes the form of natural language. Deep learning technology has demonstrated excellent performance in image and text processing and can be applied to text mining. Deep learning has two distinct advantages in text processing. First, it can automatically extract features from unstructured text data, effectively avoiding the tedious manual construction of features. Second, it has a strong ability to express and fit more complex functions. At present, we are in the era of a new industrial revolution, an era centered on artificial intelligence technology and big data. Its main feature is to regard all available data as a unique means of production. Through precise intelligent data mining and artificial intelligence technologies such as deep learning, these data can create intelligence and generate huge value.

[0003] Text data, as a unique form of data, has a wide range of applications. First, if we consider humans as subjective sensors, then text data can be considered the product of these sensors. Humans express their observations and perceptions of the objective world through text data. Second, because human expression is inherently subjective, text data contains information about people's opinions, needs, and preferences. Therefore, it is particularly helpful for mining various human attributes, enabling water environment management departments to better understand people's needs. Water resources are essential to our lives. With the continuous acceleration of urbanization, my country's water environment faces serious problems such as water pollution. The water environment encompasses many aspects, and how to extract important information from this vast amount of complex data is crucial. Because different people have different concerns about various aspects of the water environment, aspect-level opinion mining of news texts on the water environment can capture more granular information about different people's sentiments towards various aspects of the water environment. This can provide government departments with more comprehensive, authentic, and objective public feedback, helping to understand people's needs for the water environment, improve water quality, and promptly adjust water environment management plans. Therefore, aspect-level opinion mining of water environment texts is of great significance.

[0004] For a long time, research on aspect-level opinion mining in water environment texts has mainly focused on using single word vectors as text representation. This paper proposes a multi-granularity feature fusion text representation method. To address the shortcomings of traditional BiLSTM, a multi-feature fusion hybrid neural network is proposed from the aspects of multi-feature fusion and multi-model fusion. This model introduces word, part-of-speech, and character features to comprehensively identify text semantic features, integrates GLU and BiLSTM, and combines the Attention mechanism and CNN model to judge the sentiment polarity of different aspect words, thereby improving the accuracy of aspect-level opinion mining in water environment texts. Summary of the Invention

[0005] In response to the deficiencies of the existing technology, the present invention provides a method for aspect-level opinion mining of water environment texts based on multi-feature fusion. The method comprehensively considers the contextual semantic features and long-distance dependency features of the water environment texts, as well as the local sentiment feature relationships, thereby realizing aspect-level opinion mining of water environment texts.

[0006] According to one aspect of the present invention, a multi-feature fusion method is considered, that is, a feature vector fused with phrase features, part-of-speech features, category word features and character features is used to construct a word vector matrix; on this basis, a two-layer LSTM network is used to capture the contextual semantic features; at the same time, the influence of local text relations is taken into account, and the contextual semantic features and long-distance dependency features of water environment texts, as well as local sentiment feature relationships, are captured through CNN network processing.

[0007] According to another aspect of the present invention, according to the above aspect of the present invention, an Attention mechanism is introduced to dynamically describe the different importance of words in different aspects, assign attention weights to the feature vectors of the feature fusion layer, obtain a weighted feature vector matrix, and identify the emotional tendency of each aspect word, so as to improve the accuracy of opinion mining.

[0008] In summary, a method for aspect-level opinion mining of water environment text based on multi-feature fusion includes the following steps:

[0009] Step 1: Extract features from the collected water environment texts and establish a feature vector of multi-feature fusion;

[0010] Step 2: Construct a water environment text aspect-level opinion mining method model based on multi-feature fusion;

[0011] Step 3: Iteratively train the aspect-level opinion mining model to obtain the final model;

[0012] Step 4: Use the final model trained in step 3 to judge the sentiment polarity of water environment text in different aspects;

[0013] Step 5: Based on the classification results obtained in step 4, perform error analysis and performance evaluation on the model.

[0014] The step 1 considers extracting features from the collected water environment texts and establishing a feature vector of multi-feature fusion. The specific process is as follows:

[0015] First, the segmented text is used as input, and the word embedding layer is used to map words, parts of speech, and characters into a low-dimensional vector space. First, according to the correspondence between words and indexes in the dictionary, the word sequence in the text is converted into an index sequence, and the index sequence and the trained word vector matrix V are embedded through the Embedding_Lookup function to obtain the word embedding matrix. in, The word vector represents the i-th word in the comment. Through the two-layer BiLSTM learning network model, the more network layers there are, the deeper semantic information can be learned. The input of BiLSTM is the embedding matrix obtained above. The state of the hidden layer at time t is determined by the input x at time t. t and the previous moment forward hidden layer state and the backward hidden layer state at the next moment Decision. The output of the forward hidden layer at time t is obtained through the forward LSTM

[0016]

[0017] Among them, t∈[1,n], LSTM is a nonlinear activation function.

[0018] The output of the backward hidden layer at time t is obtained through the backward LSTM

[0019]

[0020] Finally, the hidden layer state at time t is forwarded and the backward hidden layer state Horizontal splicing to obtain the final output of the hidden layer at time t

[0021]

[0022] Among them, h wp is the number of neurons in the BiLSTM hidden layer.

[0023] Furthermore, the step 2 constructs a water environment text aspect-level opinion mining method model based on multi-feature fusion, and the specific process is as follows:

[0024] The model of the water environment text aspect-level opinion mining method combines GLU and Attention models for iterative training. The obtained feature vector matrix is input into BiLSTM to capture the contextual relationship. The first step is to use the Sigmoid unit of the LSTM forget gate to determine which information needs to be discarded in the cell state. It determines the cell state C at the previous moment by looking at the output value at the previous moment t-1 and the input value processed by LSTM at the current moment t. t-1 How many are retained to the current moment C t , the specific formula is:

[0025] f t =σ(W f ·[h t-1 ,f(X t ,A)]+b f ).

[0026] Among them, f t It is the forget gate, W f is the weight matrix of the forget gate, h t-1 is the network output value at time t-1, f(X t ,A) is the output value processed by step 2, [h t-1 ,f(X t ,A)] means connecting two vectors into a longer vector, b f is the bias term of the forget gate, and σ(·) is the Softmax activation function.

[0027] The next step is to determine whether the new information is stored in the cell state. This consists of two parts: the first is the Sigmoid layer, which is the input gate layer, which determines how much of the network input is stored in the cell state at the current moment; the second is the Tanh layer, which creates a new candidate value vector Add to the state. The specific update formula is:

[0028] i t =σ(W i ·[h t-1 ,f(X t ,A)]+b i ),

[0029]

[0030] Among them, i t is the input gate, Describes the cell state of the current input at time t.

[0031] Further, the old cell state C will be updated t-1 , updated to the new cell state C at the current moment t , the update rule is to choose to forget part of the old cell state through the forget gate, and to input the gate i t Select Add Candidate Cell State The specific update formula is as follows:

[0032]

[0033] Among them, C t Represents the cell state at the current time t.

[0034] Furthermore, after updating the cell state, the state features of the output cell are determined based on the input. The input is passed through a Sigmoid layer called the output gate to obtain the judgment condition. The cell state is then passed through the Tanh layer to obtain a new vector. This vector is multiplied by the judgment condition obtained by the output gate to obtain the result. The specific formula is as follows:

[0035] o t =σ(W o ·[h t-1 ,f(X t ,A)]+b o ),

[0036] h t =o t *Tanh(C t ).

[0037] Among them, t is the output gate, h t is the output value at time t.

[0038] Finally, the model uses the Attention mechanism and aspect words to assign attention weights to the feature vectors of the feature fusion layer, obtaining a weighted feature vector matrix and identifying the sentiment tendency of each aspect word, thereby improving the accuracy of opinion mining. The calculation process is to calculate the similarity between the context features and the aspect words, that is, the contribution of the word to the aspect word, and normalize the obtained contribution to obtain the attention weight. The larger the weight, the more dependent the current input label is on the current input. The context feature vector and the attention weight are weighted and summed to obtain the context semantic encoding vector of each aspect word. The model input is [X t-n ,…,X t-2 ,X t-1 ], after being processed by the LSTM network, the output is H t =[h t-n+1 ,…,h t-1 ,h t ], introduce the Attention mechanism to dynamically solve the importance of information at each historical moment. The formula is as follows:

[0039] a=Softmax(Tanh(H t W a +b a )u a ),

[0040] O=(H t ) T a.

[0041] Among them, a is the importance of historical moment information, Softmax is the normalized exponential function, O is the output result, and Tanh is the activation function.

[0042] To integrate the word features, part-of-speech features, and character features of the text, the context feature vector matrix extracted above is used as the input matrix of the decoding layer. Utilizing the multiple information of the text facilitates the subsequent BiLSTM to learn richer features. The output of the hidden layer at each moment is used as the input of the fully connected layer. After nonlinear transformation, it is input into the Softmax classification to output the sentiment polarity corresponding to each aspect of the word. The formula is as follows:

[0043] Y=[y1,y2,…,y m ]∈R m*1 .

[0044] Among them, y j represents the sentiment polarity of the jth aspect word, and m is the number of aspect word labels.

[0045] p j =Softmax(w o Owpca +b o ),

[0046] y j =Argmax(p j ).

[0047] Among them, p j represents the probability of the predicted category of the j-th aspect word, O wpca is the output of the fully connected layer, w o is the weight matrix from the fully connected layer to the Softmax layer, b o is the bias of the Softmax layer.

[0048] In iterative model training, the model's hyperparameter selection mainly includes: learning rate, batch size, training time and number of hidden units. In iterative training, the normalized dataset is divided into training and test datasets. For iterative model training, the loss function formula is defined as follows:

[0049]

[0050] Among them, the first term is used to minimize the actual data Y t With model training value The error between the two, the second term L reg The regularization term helps to avoid overfitting problems, and λ is a model hyperparameter.

[0051] The beneficial effects of the present invention are as follows:

[0052] The present invention provides an aspect-level opinion mining method for water environment text based on multi-feature fusion. The method comprehensively considers the contextual semantic features and long-distance dependency features of the water environment text, as well as the local sentiment feature relationship, constructs a multi-feature fusion hybrid neural network, introduces an Attention mechanism to describe the different importance levels of aspect words, and assigns attention weights to the feature vectors of the feature fusion layer, thereby realizing aspect-level opinion mining of water environment text and improving the accuracy of opinion mining. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The specific embodiments of the present invention are further described in detail below with reference to the accompanying drawings:

[0054] Figure 1 This is a flowchart of the aspect-level opinion mining method for water environment text in this patent;

[0055] Figure 2 This is a diagram of the water environment text aspect-level opinion mining model based on multi-feature fusion;

[0056] Figure 3 This is the BiLSTM network structure diagram. DETAILED DESCRIPTION

[0057] In order to more clearly illustrate the present invention, the present invention is further described below in conjunction with preferred embodiments and the accompanying drawings. Similar components in the accompanying drawings are represented by the same reference numerals. Those skilled in the art should understand that the content described below is illustrative and not restrictive and should not be used to limit the scope of protection of the present invention. The present invention is in no way limited to any specific configuration and algorithm proposed below, but covers any modification, replacement and improvement of the relevant elements, components and algorithms without departing from the spirit of the present invention.

[0058] The present invention proposes a method for mining aspect-level opinions of water environment text based on multi-feature fusion. By using the method of multi-feature fusion, that is, using the feature vector of phrase features, part-of-speech features, category word features and character features to construct a word vector matrix; on this basis, a two-layer LSTM network is used to capture the contextual semantic features; at the same time, considering the influence of local text relations, the CNN network is used to capture the contextual semantic features and long-distance dependency features of water environment text, as well as the local sentiment feature relationship. Figure 1 The following describes a method for mining aspect-level opinions in water environment text based on multi-feature fusion implemented in accordance with the present invention. The specific steps are as follows:

[0059] The first step is to consider feature extraction of the collected water environment text and establish a feature vector of multi-feature fusion

[0060] First, Jieba is used to segment the text to obtain the word sequence S = [w1, w2, ..., w N ], where n is the number of words. Use the Word2vec tool to train word vectors and represent the text in the vector space to obtain the word feature vector representation V = [v1, v2, ..., v N ]∈R n*d , d is the dimension of the word vector. The vector matrix is input into the convolutional layer of the CNN. Different N-gram feature maps are extracted using convolution kernels of different sizes. Finally, multiple feature maps are combined to form the feature vector matrix of the text. The segmented text is used as input, and the word embedding layer maps words, parts of speech, and characters into a low-dimensional vector space.

[0061] The second step is to build a water environment text aspect-level opinion mining method model based on multi-feature fusion

[0062] like Figure 2 As shown, first, according to the correspondence between words and indexes in the dictionary, the word sequence in the text is converted into an index sequence, and the index sequence and the trained word vector matrix V are embedded through the Embedding_Lookup function to obtain the word embedding matrix:

[0063]

[0064] in, Represents the word vector of the i-th word in the review.

[0065] Through the two-layer BiLSTM learning network model, the more layers of the network, the deeper the semantic information can be learned. The input of the BiLSTM model is the embedding matrix obtained above, such as Figure 3 As shown. The state of the hidden layer at time t is determined by the input x at time t. t and the state of the forward hidden layer at the previous moment and the state of the backward hidden layer at the next moment Decision. The output of the forward hidden layer at time t is obtained through the forward LSTM

[0066]

[0067] Among them, t∈[1,n], LSTM is a nonlinear activation function.

[0068] The output of the backward hidden layer at time t is obtained through the backward LSTM

[0069]

[0070] Finally, the hidden layer state at time t is forwarded and the backward hidden layer state Horizontal splicing to obtain the final output of the hidden layer at time t

[0071]

[0072] Among them, h wp is the number of neurons in the BiLSTM hidden layer.

[0073] The third step is to iteratively train the aspect-level opinion mining model to obtain the final model.

[0074] The model of the water environment text aspect-level opinion mining method combines GLU and Attention models for iterative training. The obtained feature vector matrix is input into BiLSTM to capture the contextual relationship. The first step is to use the Sigmoid unit of the LSTM forget gate to determine which information needs to be discarded in the cell state. It determines the cell state C at the previous moment by looking at the output value at the previous moment t-1 and the input value processed by LSTM at the current moment t. t-1 How many are retained to the current moment C t , the specific formula is:

[0075] ft =σ(W f ·[h t-1 ,f(X t ,A)]+b f ).

[0076] Among them, f t It is the forget gate, W f is the weight matrix of the forget gate, h t-1 is the network output value at time t-1, f(X t ,A) is the output value after the second step processing, [h t-1 ,f(X t ,A)] means connecting two vectors into a longer vector, b f is the bias term of the forget gate, and σ(·) is the Softmax activation function.

[0077] The next step is to determine whether the new information is stored in the cell state. This consists of two parts: the first is the Sigmoid layer, which is the input gate layer, which determines how much of the network input is stored in the cell state at the current moment; the second is the Tanh layer, which creates a new candidate value vector Add to the state. The specific update formula is:

[0078] i t =σ(W i ·[h t-1 ,f(X t ,A)]+b i ),

[0079]

[0080] Among them, i t is the input gate, Describes the cell state of the current input at time t.

[0081] Next, the old cell state C will be updated t-1 , updated to the new cell state C at the current moment t , the update rule is to choose to forget part of the old cell state through the forget gate, and to input the gate i t Select Add Candidate Cell State The specific update formula is as follows:

[0082]

[0083] Among them, C t Represents the cell state at the current time t.

[0084] Furthermore, after updating the cell state, the state features of the output cell are determined based on the input. The input is passed through a Sigmoid layer called the output gate to obtain the judgment condition. The cell state is then passed through the Tanh layer to obtain a new vector. This vector is multiplied by the judgment condition obtained by the output gate to obtain the result. The specific formula is as follows:

[0085] o t =σ(W o ·[h t-1 ,f(X t ,A)]+b o ),

[0086] h t =o t *Tanh(C t ).

[0087] Among them, t is the output gate, h t is the output value at time t.

[0088] Finally, the model uses the Attention mechanism and aspect words to assign attention weights to the feature vectors of the feature fusion layer, obtaining a weighted feature vector matrix and identifying the sentiment tendency of each aspect word, thereby improving the accuracy of opinion mining. The calculation process is to calculate the similarity between the context features and the aspect words, that is, the contribution of the word to the aspect word, and normalize the obtained contribution to obtain the attention weight. The larger the weight, the more dependent the current input label is on the current input. The context feature vector and the attention weight are weighted and summed to obtain the context semantic encoding vector of each aspect word. The model input is [X t-n ,…,X t-2 ,X t-1 ], after being processed by the LSTM network, the output is H t =[h t-n+1 ,…,h t-1 ,h t ], introduce the Attention mechanism to dynamically solve the importance of information at each historical moment. The formula is as follows:

[0089] a=Softmax(Tanh(H t W a +b a )u a ),

[0090] O=(H t ) T a.

[0091] Among them, a is the importance of historical moment information, Softmax is the normalized exponential function, O is the output result, and Tanh is the activation function.

[0092] The fourth step is to use the final model trained in the third step to judge the sentiment polarity of water environment text in different aspects.

[0093] To integrate the word features, part-of-speech features, and character features of the text, the context feature vector matrix extracted above is used as the input matrix of the decoding layer. Utilizing the multiple information of the text facilitates the subsequent BiLSTM to learn richer features. The output of the hidden layer at each moment is used as the input of the fully connected layer. After nonlinear transformation, it is input into the Softmax classification to output the sentiment polarity corresponding to each aspect of the word. The formula is as follows:

[0094] Y=[y1,y2,…,y m ]∈R m*1 .

[0095] Among them, y j represents the sentiment polarity of the jth aspect word, and m is the number of aspect word labels.

[0096] p j =Softmax(w o O wpca +b o ),

[0097] y j =Argmax(p j ).

[0098] Among them, p j represents the probability of the predicted category of the j-th aspect word, O wpca is the output of the fully connected layer, w o is the weight matrix from the fully connected layer to the Softmax layer, b o is the bias of the Softmax layer.

[0099] Step 5: Perform error analysis and performance evaluation on the classification results obtained in step 4

[0100] In iterative model training, the model's hyperparameter selection mainly includes: learning rate, batch size, training time and number of hidden units. In iterative training, the normalized dataset is divided into training and test datasets. For iterative model training, the loss function formula is defined as follows:

[0101]

[0102] Among them, the first term is used to minimize the actual data Y t With model training value The error between the two, the second term L reg The regularization term helps to avoid overfitting problems, and λ is a model hyperparameter.

[0103] The above-described specific embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to be limiting of the embodiments of the present invention. Persons skilled in the art will appreciate that various other variations or modifications may be made based on the above-described embodiments of the present invention. It is not possible to enumerate all embodiments here. Any obvious variations or modifications derived from the technical solution of the present invention remain within the scope of protection of the present invention.

Claims

1. A method for aspect-level opinion mining of water environment text based on multi-feature fusion. Aspect-level mining refers to mining the aspects involved in a sentence and the emotions expressed for each aspect. A water environment text contains different aspects of the water environment. The mining model is used to study different aspects of the water environment. The characteristics are: The method comprises the following steps: Step 1: Extract features from the collected water environment texts and establish a feature vector of multi-feature fusion; Step 2: Use the multi-feature fusion method, that is, use the feature vector of phrase features, part-of-speech features, category word features and character features to build a water environment text aspect-level opinion mining model based on multi-feature fusion, combine BiLSTM and CNN to learn water environment text and mine text opinions, respectively capturing the context order features and long-distance dependency features, as well as local sentiment feature relationships; first, use the segmented text as input, use the word embedding layer to map words, parts of speech and characters to a low-dimensional vector space, and according to the correspondence between words and indexes in the dictionary, convert the word sequence in the text into an index sequence, and embed the index sequence and the trained word vector matrix V through the Embedding_Lookup function to obtain the word embedding matrix in, The word vector representing the i-th word in the review. A two-layer BiLSTM learning network model is used. The more layers a network has, the deeper semantic information it can learn. The input to the BiLSTM model is the embedding matrix obtained previously. Step 3: An aspect-level opinion mining model that integrates a bidirectional long short-term memory (BiLSTM) network and a convolutional neural network (CNN) with gated linear units (GLU) and an attention mechanism is iteratively trained to optimize model parameters and obtain the final model. The feature vectors trained with Word2vec are used to input the embedding matrix into the BiLSTM model, which is then iteratively trained with the GLU and Attention models. The Attention mechanism and aspect words are used to assign attention weights to the feature vectors of the feature fusion layer to obtain a weighted feature vector matrix. The sentiment tendency of each aspect word is then identified, thereby improving the accuracy of opinion mining. Step 4: Use the final model trained in step 3 to judge the sentiment polarity of different aspects of water environment text; Step 5: Based on the classification results obtained in step 4, perform error analysis and performance evaluation on the model.

2. The method for mining aspect-level opinions of water environment text based on multi-feature fusion according to claim 1 is characterized in that: Extract features from the collected water environment texts and establish a feature vector of multi-feature fusion, specifically: Use Jieba to segment the text and get the word sequence S = [w1,w2,…,w N ], where n is the number of words; use the Word2vec tool to train word vectors, represent text in vector space, and get the word feature vector representation of the text V = [v1, v2, ..., v N ]∈R n*d , d is the dimension of the word vector; the vector matrix is input into the convolution layer of CNN, and different N-gram feature maps are extracted using convolution kernels of different sizes. Finally, multiple feature maps are combined as the text feature vector matrix.

3. The water environment text aspect-level opinion mining method based on multi-feature fusion according to claim 1, wherein the training process calculates the similarity between the context features and the aspect words, that is, the contribution degree of the words to the aspect words, normalizes the obtained contribution degree, and obtains the attention weight. The larger the weight, the more dependent the current input label is on the current input. The feature vector of the context and the attention weight are weighted and summed to obtain the context semantic encoding vector of each aspect word; in order to fuse the word features, part-of-speech features and character features of the text, the context feature vector matrix extracted above is used as the input matrix of the decoding layer, and the multiple information of the text is used to facilitate the subsequent BiLSTM to learn richer features; the output of the hidden layer at each moment is used as the input of the fully connected layer, and is input into the Softmax classification after nonlinear transformation, and the sentiment polarity corresponding to each aspect word is output. The formula is as follows: Y=[y1,y2,…,y m ]∈R m*1 ; in, y j represents the sentiment polarity of the jth aspect word, and m is the number of aspect word labels.

4. The method for mining aspect-level opinions of water environment text based on multi-feature fusion according to claim 1 is characterized in that: The final model after training is used to perform aspect-level sentiment classification on the Weibo water environment text dataset using test data, and the sentiment index of the water environment text is analyzed according to different aspects of the water environment.

5. The method for mining aspect-level opinions of water environment text based on multi-feature fusion according to claim 1 is characterized in that: The experimental results are compared and the model is subjected to error analysis and performance evaluation.

Images