Text sentiment analysis method and system integrating long short-term memory network and word embedding

By integrating long and short-term memory networks and word embedding technology, a text sentiment analysis model is established, and the accuracy of text sentiment analysis and long-sequence data processing problems in the existing technology are solved, achieving higher analysis accuracy and context understanding.

CN118468848BActive Publication Date: 2024-11-12BEIJING CORAL REEF INTELLIGENT TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410937607.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-12
Publication Date
2024-11-12
Estimated Expiration
2044-07-12

AI Technical Summary

Technical Problem

The prior art is difficult to capture the relationship between text context features in text sentiment analysis, resulting in low accuracy of sentiment analysis, and the problems of information loss and gradient disappearance when processing long-sequence data. Long-term short-term memory networks are adjusted in parameter and computing resources. Consumption is limited, and word embeddings are difficult to maintain the integrity of context information in complex sequence models.

Method used

Integrate long and short-term memory networks and word embeddings, and create text sentiment analysis models, including word embedding units, first long and short-term memory network units, and second long and short-term memory network units, adjust network parameters using training samples, generate word vector sequences and hide The state sequence, combined with the attention mechanism to identify the emotions of the text, optimizes network parameters through backpropagation algorithm.

Benefits of technology

It effectively solves the problems of information loss and gradient vanishing in long-sequence data processing, improves the accuracy of text sentiment analysis, and enhances the ability to understand text semantics and context.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118468848B_ABST
    Figure CN118468848B_ABST
Patent Text Reader

Abstract

The present invention provides a text sentiment analysis method and system integrating long short-term memory network and word embedding, and relates to the field of data processing. The method comprises: establishing a text sentiment analysis model, comprising a word embedding unit, a first long short-term memory network unit and a second long short-term memory network unit; setting initial parameters of the first long short-term memory network unit and initial parameters of the second long short-term memory network unit, and using training samples to adjust the initial parameters of the first long short-term memory network unit and the initial parameters of the second long short-term memory network unit; using the word embedding unit to generate a word vector sequence corresponding to a text to be recognized; using the optimized first long short-term memory network unit to generate a hidden state sequence of the text to be recognized based on the word vector sequence corresponding to the text to be recognized; using the optimized second long short-term memory network unit to recognize the sentiment of the text to be recognized based on the hidden state sequence of the text to be recognized, and has the advantage of improving the accuracy of text sentiment analysis.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a text sentiment analysis method and system integrating long short-term memory network and word embedding. Background Art

[0002] Sentiment analysis technology is currently widely used in Internet applications such as online translation, user evaluation analysis and opinion mining. Especially for various emerging online social platforms and shopping websites, quickly obtaining the sentiment tendency of user comments can provide great convenience for businesses in advertising and hot topic push. From the perspective of current market demand and the level of technological development, research and innovation on sentiment analysis technology has great value and room for improvement.

[0003] Existing technologies mainly focus on sentiment analysis based on sentiment dictionaries or traditional machine learning methods such as support vector machines (SVMs), naive Bayesian sentiment analysis methods, or methods based on single neural network models, such as CNNs (Convolutional Neural Networks), RNNs (Recurrent Neural Networks), GNNs (Graph Neural Networks), etc. They cannot capture the relationship between the contextual features of the text, nor can they learn the text semantics, resulting in low accuracy of sentiment analysis and poor classification results. In addition, in the field of natural language processing (NLP), when processing sequence data, there are often problems of information loss and gradient disappearance, especially when processing long sequence data. Existing technologies such as recurrent neural networks (RNNs) have difficulties in capturing long-distance dependencies, and although long short-term memory networks (LSTMs) have improved this problem, they are still limited by their parameter adjustment and consumption of computing resources. At the same time, although word embedding technology can effectively capture the semantic relationship between words, when it is integrated into a complex sequence model, how to effectively maintain the integrity of contextual information is still a challenge, resulting in inaccurate text sentiment analysis.

[0004] Therefore, it is necessary to provide a text sentiment analysis method and system that integrates long short-term memory network and word embedding to improve the accuracy of text sentiment analysis. Summary of the invention

[0005] The present invention provides a text sentiment analysis method integrating long short-term memory network and word embedding, comprising: establishing a text sentiment analysis model, wherein the text sentiment analysis model at least includes a word embedding unit, a first long short-term memory network unit and a second long short-term memory network unit; setting parameters of the word embedding unit, wherein the parameters of the word embedding unit include a vector representation of each word in a vocabulary; setting initial parameters of the first long short-term memory network unit and initial parameters of the second long short-term memory network unit; based on the word embedding unit, using training samples, adjusting the initial parameters of the first long short-term memory network unit and the initial parameters of the second long short-term memory network unit to obtain an optimized first long short-term memory network unit and an optimized second long short-term memory network unit; using the word embedding unit to generate a word vector sequence corresponding to the text to be recognized; using the optimized first long short-term memory network unit to generate a hidden state sequence of the text to be recognized based on the word vector sequence corresponding to the text to be recognized; using the optimized second long short-term memory network unit to recognize the sentiment of the text to be recognized based on the hidden state sequence of the text to be recognized.

[0006] Further, based on the word embedding unit, using the training sample, adjusting the initial parameters of the first long short-term memory network unit and the initial parameters of the second long short-term memory network unit, obtaining the optimized first long short-term memory network unit and the optimized second long short-term memory network unit, including: S1, using the word embedding unit to generate a sample word vector sequence corresponding to the sample text of the training sample, and executing S2; S2, executing forward propagation, using the first long short-term memory network unit based on the sample word vector sequence corresponding to the sample text of the training sample, extracting the sample hidden state sequence corresponding to the sample text of the training sample, and executing S3; S3, using Use the second long short-term memory network unit to identify the emotion of the training sample based on the sample hidden state sequence corresponding to the sample text of the training sample, and execute S4; S4, calculate the loss function according to the emotion of the training sample identified by the second long short-term memory network unit and the label of the training sample, and execute S5; S5, adjust the initial parameters of the first long short-term memory network unit and the initial parameters of the second long short-term memory network unit through the back propagation algorithm, and execute S6; S6, repeat S1-S5 until the preset conditions are met to obtain the optimized first long short-term memory network unit and the optimized second long short-term memory network unit.

[0007] Furthermore, the loss function is: , where N is the number of tags, is the i-th label of the training sample, is the predicted output of the second LSTM network unit corresponding to the i-th label of the training sample.

[0008] Further, based on the following parameter adjustment function, the initial parameters of the first long short-term memory network unit and the initial parameters of the second long short-term memory network unit are adjusted by the back propagation algorithm: ,in, is the parameter at time step 𝑡+1, are the parameters at time step 𝑡, is the learning rate, is the first-order moment estimate at time step 𝑡, is the second-order moment estimate at time step 𝑡, is a constant.

[0009] Furthermore, a word embedding unit is used to generate a word vector sequence corresponding to the text to be recognized, including: decomposing the text to be recognized into multiple words to generate a word sequence corresponding to the text to be recognized; using a word embedding model to map each word included in the word sequence corresponding to the text to be recognized into a corresponding vector to generate a word vector sequence corresponding to the text to be recognized.

[0010] Furthermore, the optimized first long short-term memory network unit is used to generate a hidden state sequence of the text to be recognized based on the word vector sequence corresponding to the text to be recognized, including: passing the word vector sequence corresponding to the text to be recognized into the optimized first long short-term memory network unit time step by time step; at each time step, the optimized first long short-term memory network unit calculates the activation value of the gate control according to the input of the current time step and the hidden state of the previous time step, and updates the internal state and the hidden state; the updated internal state and the hidden state are passed to the next time step until each vector of the word vector sequence corresponding to the text to be recognized is processed, and the hidden state sequence of the text to be recognized is obtained.

[0011] Furthermore, the optimized second long short-term memory network unit is used to identify the emotion of the text to be recognized based on the hidden state sequence of the text to be recognized, including: calculating the attention score of each hidden state in the hidden state sequence of the text to be recognized; based on the attention score of each hidden state in the hidden state sequence of the text to be recognized, performing weighted summation on each hidden state in the hidden state sequence of the text to be recognized to generate a context vector; based on the context vector, identifying the emotion of the text to be recognized.

[0012] Furthermore, the attention score of each hidden state in the hidden state sequence of the text to be recognized is calculated based on the following formula: ,in, is the attention score of the jth hidden state, W is the weight matrix of the attention layer, is the jth hidden state, is the normalization function.

[0013] Furthermore, the context vector is generated based on the following formula: ,in, is the context vector, and J is the total number of hidden states included in the hidden state sequence of the text to be recognized.

[0014] The present invention provides a text sentiment analysis system integrating long short-term memory network and word embedding, comprising: a model building module, used to build a text sentiment analysis model, wherein the text sentiment analysis model at least includes a word embedding unit, a first long short-term memory network unit and a second long short-term memory network unit; a parameter setting module, used to set the parameters of the word embedding unit, wherein the parameters of the word embedding unit include the vector representation of each word in the vocabulary; the parameter setting module is also used to set the initial parameters of the first long short-term memory network unit and the initial parameters of the second long short-term memory network unit; a parameter optimization module, used to adjust the parameters of the word embedding based on the parameters of the word embedding, using training samples The initial parameters of the first long short-term memory network unit and the initial parameters of the second long short-term memory network unit are used to obtain the optimized first long short-term memory network unit and the optimized second long short-term memory network unit; a vector generation module is used to use the word embedding unit to generate a word vector sequence corresponding to the text to be recognized; a sequence extraction module is used to use the optimized first long short-term memory network unit to generate a hidden state sequence of the text to be recognized based on the word vector sequence corresponding to the text to be recognized; an emotion recognition module is used to use the optimized second long short-term memory network unit to identify the emotion of the text to be recognized based on the hidden state sequence of the text to be recognized.

[0015] Compared with the prior art, the text sentiment analysis method and system integrating long short-term memory network and word embedding provided by the present invention have at least the following beneficial effects:

[0016] By integrating long short-term memory networks and word embedding technology, the problems of information loss and gradient vanishing in long sequence data processing can be effectively solved, and the accuracy of text sentiment analysis can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0018] Figure 1 It is a flowchart of a text sentiment analysis method integrating long short-term memory network and word embedding according to some embodiments of this specification;

[0019] Figure 2 is a schematic diagram of a process of generating a hidden state sequence of a text to be recognized according to some embodiments of this specification;

[0020] Figure 3 It is a schematic diagram of a process of adjusting initial parameters of a first long short-term memory network unit and initial parameters of a second long short-term memory network unit according to some embodiments of this specification;

[0021] Figure 4 It is a module diagram of a text sentiment analysis system integrating long short-term memory network and word embedding according to some embodiments of this specification. DETAILED DESCRIPTION

[0022] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of this specification. For ordinary technicians in this field, this specification can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.

[0023] Figure 1 is a flowchart of a text sentiment analysis method integrating long short-term memory network and word embedding according to some embodiments of this specification, such as Figure 1 As shown, the text sentiment analysis method integrating long short-term memory network and word embedding can include the following steps.

[0024] Step 110: Establish a text sentiment analysis model.

[0025] Among them, the text sentiment analysis model includes at least a word embedding unit, a first long short-term memory network unit and a second long short-term memory network unit.

[0026] Step 120, setting parameters of the word embedding unit.

[0027] The parameters of the word embedding unit include the vector representation of each word in the vocabulary.

[0028] Step 130, setting initial parameters of the first long short-term memory network unit and initial parameters of the second long short-term memory network unit.

[0029] As the core architecture, the Long Short-Term Memory Network (LSTM) is a special type of Recurrent Neural Network (RNN) designed to solve the information loss and gradient vanishing problems encountered by traditional RNNs when processing long sequence data. The key to the LSTM network lies in its internal gating mechanism, including the input gate, forget gate, and output gate. These gates control the flow of information between units, allowing the network to introduce new information when necessary, discard irrelevant information, or retain current information for future use.

[0030] The initial parameters of the first long short-term memory network unit may include the weights and biases of the input gate, forget gate, output gate, and unit state of the first long short-term memory network unit. The initial parameters of the second long short-term memory network unit may include the weights and biases of the input gate, forget gate, output gate, and unit state of the second long short-term memory network unit.

[0031] Step 140, based on the word embedding unit, using the training sample, adjust the initial parameters of the first long short-term memory network unit and the initial parameters of the second long short-term memory network unit to obtain an optimized first long short-term memory network unit and an optimized second long short-term memory network unit.

[0032] Figure 3 is a flow chart of adjusting the initial parameters of the first long short-term memory network unit and the initial parameters of the second long short-term memory network unit according to some embodiments of the present specification, such as Figure 3 As shown, in some embodiments, based on the word embedding unit, using the training sample, adjusting the initial parameters of the first long short-term memory network unit and the initial parameters of the second long short-term memory network unit to obtain the optimized first long short-term memory network unit and the optimized second long short-term memory network unit, including:

[0033] S1. Use the word embedding unit to generate a sample word vector sequence corresponding to the sample text of the training sample;

[0034] S2. Perform forward propagation, using the first long short-term memory network unit to extract the sample hidden state sequence corresponding to the sample text of the training sample based on the sample word vector sequence corresponding to the sample text of the training sample. Specifically, this step involves converting the sample word vector sequence into a vector representation through a word embedding layer, and then inputting it into the first long short-term memory network unit. The first long short-term memory network unit takes each element in the sample word vector sequence as input and outputs the hidden state of the sample word vector sequence;

[0035] S3, using the second long short-term memory network unit to identify the sentiment of the training sample based on the sample hidden state sequence corresponding to the sample text of the training sample. Specifically, the importance of each state of the sample hidden state sequence is calculated through the attention layer, and finally the information is integrated into a context vector, which contains the key sentiment information of the entire sample hidden state sequence, and is used to judge the overall sentiment tendency;

[0036] S4, calculating a loss function according to the sentiment of the training sample identified by the second long short-term memory network unit and the label of the training sample;

[0037] S5. Adjusting the initial parameters of the first long short-term memory network unit and the initial parameters of the second long short-term memory network unit through a back propagation algorithm;

[0038] S6. Repeat S1-S5 until a preset condition is met, wherein the preset condition may be that the performance of the text sentiment analysis model no longer improves or reaches a predetermined number of iterations, thereby obtaining an optimized first long short-term memory network unit and an optimized second long short-term memory network unit.

[0039] In some embodiments, the loss function is:

[0040]

[0041] Where N is the number of tags, is the i-th label of the training sample, is the predicted output of the second LSTM network unit corresponding to the i-th label of the training sample.

[0042] In some embodiments, the initial parameters of the first LSTM network unit and the initial parameters of the second LSTM network unit are adjusted by a back propagation algorithm based on the following parameter adjustment function:

[0043]

[0044] in, is the parameter at time step 𝑡+1, are the parameters at time step 𝑡, is the learning rate, is the first-order moment estimate at time step 𝑡, is the second-order moment estimate at time step 𝑡, is a constant.

[0045] Step 150: Use the word embedding unit to generate a word vector sequence corresponding to the text to be recognized.

[0046] Figure 2is a flow chart of generating a hidden state sequence of a text to be recognized according to some embodiments of this specification, such as Figure 2 As shown, in some embodiments, using a word embedding unit to generate a word vector sequence corresponding to the text to be recognized includes:

[0047] Decompose the text to be recognized into multiple words and generate a word sequence corresponding to the text to be recognized;

[0048] Use the word embedding model to map each word in the word sequence corresponding to the text to be recognized into a corresponding vector, and generate a word vector sequence corresponding to the text to be recognized.

[0049] Word embedding technology is achieved by converting words into vectors in a high-dimensional space. These vectors capture the semantic and syntactic relationships between words. Specifically, word embedding is usually achieved by training a neural network model to learn dense vector representations of each word on a large amount of text data. These vectors can represent multiple attributes and contextual information of words. The advantage of using word embedding is that compared to traditional word representation methods (such as one-hot encoding), word vectors can express richer information, such as the similarity between words. Each word in the text to be recognized is first converted to a corresponding vector through a pre-trained word embedding model, such as Word2Vec or GloVe. These vectors are then used as input to the LSTM network, further enhancing the text sentiment analysis model's ability to understand word-level relationships in long sequence data.

[0050] Just as an example, in a sentiment analysis task, given a review text containing multiple sentences, the review is first segmented into words and these words are converted into vectors. These vectors are input into the LSTM network and processed step by step, and the network captures the sentiment tendency of the entire review. Finally, the representation of the network output can be used to determine whether the review is positive or negative. In this way, the text sentiment analysis model can use word embedding technology to provide more accurate and fine-grained sequence data processing.

[0051] Step 160, using the optimized first long short-term memory network unit to generate a hidden state sequence of the text to be recognized based on the word vector sequence corresponding to the text to be recognized.

[0052] Specifically include:

[0053] The word vector sequence corresponding to the text to be recognized is passed into the optimized first long short-term memory network unit time step by time step;

[0054] At each time step, the optimized first LSTM network unit calculates the activation value of the gate control according to the input of the current time step and the hidden state of the previous time step, and updates the internal state and hidden state;

[0055] The updated internal state and hidden state are passed to the next time step until each vector of the word vector sequence corresponding to the text to be recognized is processed and the hidden state sequence of the text to be recognized is obtained.

[0056] The state update inside the first LSTM network unit can be described by the following mathematical formula:

[0057]

[0058] in, , and Represent the activation values ​​of the forget gate, input gate, and output gate respectively, is the hidden state at the previous time step, is the input of the current time step, is the internal state of the current unit, is the candidate state, and are network parameters, is the sigmoid activation function, represents the element-wise product, is the hyperbolic tangent function.

[0059] Just as an example, in the text sentiment analysis task, given a text sequence, it is first converted into a sequence of word vectors. Then, these word vectors are input into the first long short-term memory network unit one by one. The first long short-term memory network unit gradually builds up the contextual representation of the entire sequence through a gating mechanism, and outputs the final hidden state at the last time step of the sequence. This hidden state carries the information of the entire text sequence and can be used in the second long short-term memory network unit to predict the category of the text. In this way, the text sentiment analysis model can effectively process long sequence data and improve the accuracy in the text sentiment analysis task.

[0060] Step 170: Use the optimized second long short-term memory network unit to identify the emotion of the text to be recognized based on the hidden state sequence of the text to be recognized.

[0061] Specifically include:

[0062] Calculate the attention score of each hidden state in the hidden state sequence of the text to be recognized;

[0063] Based on the attention score of each hidden state in the hidden state sequence of the text to be recognized, a weighted sum is performed on each hidden state in the hidden state sequence of the text to be recognized to generate a context vector;

[0064] Based on the context vector, the sentiment of the text to be recognized is identified.

[0065] In order to further improve the model's understanding of complex sequence data dependencies, an innovative attention mechanism is introduced. This mechanism enables the model to automatically identify and focus on the most critical information in the sequence, thereby optimizing the model's processing capabilities. Specifically, the attention mechanism allows the model to assign different weights when processing input sequences, allowing the model to focus more on the parts that are more important to the prediction task. Mathematically, this can be expressed as a weight assignment process, where the weight of each input element is determined by an attention score, which is calculated through a trainable attention layer.

[0066] In some embodiments, the attention score of each hidden state in the hidden state sequence of the text to be recognized is calculated based on the following formula:

[0067]

[0068] in, is the attention score of the jth hidden state, W is the weight matrix of the attention layer, is the jth hidden state, is the normalization function.

[0069] In some embodiments, the context vector is generated based on the following formula:

[0070]

[0071] in, is the context vector, and J is the total number of hidden states included in the hidden state sequence of the text to be recognized.

[0072] Specifically, for sentiment analysis, the input sequence is a series of word vectors, which are processed by the first long short-term memory network unit to obtain a hidden state sequence, and then the importance of each state is calculated through the attention layer. Finally, this information is integrated into a context vector, which contains the key sentiment information of the entire sequence and is used to judge the overall sentiment tendency. Through such a mechanism, the text sentiment analysis model can not only capture long-distance dependencies, but also adaptively focus on the most informative sequence part, greatly improving the accuracy and efficiency of sequence data processing.

[0073] For example, when a sentence (such as "This movie is great") is input into a text sentiment analysis model, the sentence is first converted into a word vector, then features are extracted through the text sentiment analysis model, and the network parameters are adjusted through an optimization algorithm to accurately classify sentiment tendencies.

[0074] In order to ensure the high performance of the text sentiment analysis model when processing sequence data of different lengths, especially long text tasks, a comprehensive training and validation process is adopted. This process details how to train and evaluate the model through a large amount of sequence data to ensure that it has excellent processing capabilities in various situations.

[0075] First, the text sentiment analysis model will be trained by inputting large-scale sequence data sets in batches. These data sets contain sequences of various lengths, especially a large amount of long text data. During the training process, the parameters of the text sentiment analysis model will be continuously adjusted according to the feedback of the loss function to optimize the model's ability to process sequence data.

[0076] Secondly, to verify the performance of the text sentiment analysis model, a set of independent validation datasets will be used for testing. These datasets also contain sequences of different lengths to ensure the comprehensiveness of the test results. The performance of the text sentiment analysis model will be evaluated based on its performance on these test data, especially its accuracy and stability when processing long text sequences.

[0077] In addition, in order to further improve the performance of the text sentiment analysis model, the text sentiment analysis model will be cross-validated. This means that the data is divided into several parts, each of which is used as a validation data set in turn, and the remaining part is used as training data. This method can more comprehensively evaluate the performance of the model and reduce the risk of overfitting.

[0078] For example, when processing the sentiment analysis task of movie review text, the review text is segmented into sentences and these sentences are input into the text sentiment analysis model as sequence data. The text sentiment analysis model first converts the sentences into vector representations through word embedding, and then processes the sequence through the LSTM network to capture the long-term dependencies between sentences. By comparing the sentiment labels predicted by the text sentiment analysis model with the actual labels, the loss function is calculated, and the parameters of the text sentiment analysis model are adjusted through the gradient descent algorithm. This process is repeated until the text sentiment analysis model shows high accuracy and stability on long text sequences. Through this training and verification method, it can be ensured that the text sentiment analysis model can maintain high accuracy and excellent performance when processing sequence data of varying lengths, especially in long text tasks.

[0079] Figure 4 is a module diagram of a text sentiment analysis system integrating long short-term memory network and word embedding according to some embodiments of this specification, such as Figure 4 As shown, the text sentiment analysis system integrating long short-term memory network and word embedding may include a model building module 410, a parameter setting module 420, a parameter optimization module 430, a vector generation module 440, a sequence extraction module 450 and a sentiment recognition module 460.

[0080] The model building module 410 can be used to build a text sentiment analysis model, wherein the text sentiment analysis model includes at least a word embedding unit, a first long short-term memory network unit, and a second long short-term memory network unit.

[0081] The parameter setting module 420 can be used to set the parameters of the word embedding unit, wherein the parameters of the word embedding unit include the vector representation of each word in the vocabulary.

[0082] The parameter setting module 430 can also be used to set the initial parameters of the first long short-term memory network unit and the initial parameters of the second long short-term memory network unit.

[0083] The parameter optimization module 440 can be used to adjust the initial parameters of the first long short-term memory network unit and the initial parameters of the second long short-term memory network unit based on the parameters of word embedding using training samples, and obtain the optimized first long short-term memory network unit and the optimized second long short-term memory network unit.

[0084] The vector generation module 450 may be configured to generate a word vector sequence corresponding to the text to be recognized using a word embedding unit.

[0085] The sequence extraction module 450 can be used to generate a hidden state sequence of the text to be recognized based on the word vector sequence corresponding to the text to be recognized using the optimized first long short-term memory network unit.

[0086] The emotion recognition module 460 can be used to identify the emotion of the text to be recognized based on the hidden state sequence of the text to be recognized using the optimized second long short-term memory network unit.

[0087] The text sentiment analysis system integrating long short-term memory network and word embedding can be used to execute the text sentiment analysis method integrating long short-term memory network and word embedding. For more descriptions of the text sentiment analysis system integrating long short-term memory network and word embedding, please refer to the relevant descriptions of the text sentiment analysis method integrating long short-term memory network and word embedding, which will not be repeated here.

[0088] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. A text sentiment analysis method integrating long short-term memory network and word embedding, characterized in that: include: Establishing a text sentiment analysis model, wherein the text sentiment analysis model at least includes a word embedding unit, a first long short-term memory network unit, and a second long short-term memory network unit; Setting parameters of the word embedding unit, wherein the parameters of the word embedding unit include a vector representation of each word in the vocabulary; Setting initial parameters of the first long short-term memory network unit and initial parameters of the second long short-term memory network unit, wherein the initial parameters of the first long short-term memory network unit include the weights and biases of the input gate, forget gate, output gate, and unit state of the first long short-term memory network unit, and the initial parameters of the second long short-term memory network unit include the weights and biases of the input gate, forget gate, output gate, and unit state of the second long short-term memory network unit; Based on the word embedding unit, using training samples, adjusting initial parameters of the first long short-term memory network unit and initial parameters of the second long short-term memory network unit to obtain an optimized first long short-term memory network unit and an optimized second long short-term memory network unit; Use the word embedding unit to generate the word vector sequence corresponding to the text to be recognized; Using the optimized first long short-term memory network unit to generate a hidden state sequence of the text to be recognized based on the word vector sequence corresponding to the text to be recognized; Using the optimized second long short-term memory network unit to identify the emotion of the text to be recognized based on the hidden state sequence of the text to be recognized; Among them, based on the word embedding unit, using the training sample, adjusting the initial parameters of the first long short-term memory network unit and the initial parameters of the second long short-term memory network unit to obtain the optimized first long short-term memory network unit and the optimized second long short-term memory network unit, including: S1. Generate a sample word vector sequence corresponding to the sample text of the training sample using a word embedding unit; S2, performing forward propagation, using the first long short-term memory network unit to extract the sample hidden state sequence corresponding to the sample text of the training sample based on the sample word vector sequence corresponding to the sample text of the training sample, specifically, converting the sample word vector sequence into a vector representation through a word embedding layer, and then inputting it into the first long short-term memory network unit, the first long short-term memory network unit takes each element in the sample word vector sequence as input, and outputs the sample hidden state sequence of the sample word vector sequence; S3. Using the second long short-term memory network unit to identify the emotion of the training sample based on the sample hidden state sequence corresponding to the sample text of the training sample, specifically, calculating the attention score of each hidden state of the sample hidden state sequence through the attention layer, generating a context vector based on the sample hidden state sequence of the sample word vector sequence and the attention score of each hidden state, wherein the context vector contains the key emotion information of the entire sample hidden state sequence, and is used to judge the overall emotion tendency. The first long short-term memory network unit gradually builds the context representation of the entire sequence through the gating mechanism, and outputs the final hidden state at the last time step of the sequence. The hidden state carries the information of the entire text sequence and is used by the second long short-term memory network unit to predict the category of the text. The attention mechanism allows the second long short-term memory network unit to assign different weights when processing the input sequence, so that the second long short-term memory network unit can focus more on the part that is more important to the prediction task, so that the second long short-term memory network unit can capture long-distance dependencies and adaptively focus on the most informative sequence part. S4, calculating a loss function according to the emotion of the training sample identified by the second long short-term memory network unit and the label of the training sample; S5. Adjusting the initial parameters of the first long short-term memory network unit and the initial parameters of the second long short-term memory network unit through a back propagation algorithm; S6, repeatedly executing S1-S5 until the preset conditions are met, and obtaining the optimized first long short-term memory network unit and the optimized second long short-term memory network unit; The attention score for each hidden state in the hidden state sequence is calculated based on the following formula: in, is the attention score of the jth hidden state, W is the weight matrix of the attention layer, is the jth hidden state, is the normalization function; The context vector is generated based on the following formula: in, is the context vector, J is the total number of hidden states included in the hidden state sequence; Based on the following parameter adjustment function, the initial parameters of the first long short-term memory network unit and the initial parameters of the second long short-term memory network unit are adjusted by the back propagation algorithm: in, is the parameter at time step t+1, is the parameter at time step t, is the learning rate, is the first-order moment estimate at time step t, is the second-order moment estimate at time step t, is a constant; Using the optimized first long short-term memory network unit to generate a hidden state sequence of the text to be recognized based on the word vector sequence corresponding to the text to be recognized, including: The word vector sequence corresponding to the to-be-recognized text is transferred into the optimized first long short-term memory network unit time step by time step; At each time step, the optimized first long short-term memory network unit calculates the activation value of the gate control according to the input of the current time step and the hidden state of the previous time step, and updates the internal state and the hidden state; The updated internal state and hidden state are passed to the next time step until each vector of the word vector sequence corresponding to the text to be recognized is processed, and the hidden state sequence of the text to be recognized is completed; Using the optimized second long short-term memory network unit to identify the emotion of the text to be recognized based on the hidden state sequence of the text to be recognized, comprising: Calculating the attention score of each hidden state in the hidden state sequence of the text to be recognized; Based on the attention score of each hidden state in the hidden state sequence of the text to be recognized, weighted sum is performed on each hidden state in the hidden state sequence of the text to be recognized to generate a context vector; Based on the context vector, the sentiment of the text to be recognized is recognized.

2. The text sentiment analysis method integrating long short-term memory network and word embedding according to claim 1, characterized in that: The loss function is: Where N is the number of tags, is the i-th label of the training sample, is the predicted output of the second LSTM network unit corresponding to the i-th label of the training sample.

3. The text sentiment analysis method integrating long short-term memory network and word embedding according to claim 1 or 2, characterized in that: Using a word embedding unit to generate a word vector sequence corresponding to the text to be recognized includes: Decomposing the text to be recognized into a plurality of words to generate a word sequence corresponding to the text to be recognized; A word embedding model is used to map each word included in the word sequence corresponding to the text to be recognized into a corresponding vector, so as to generate a word vector sequence corresponding to the text to be recognized.

4. A text sentiment analysis system integrating long short-term memory network and word embedding, characterized in that: A text sentiment analysis method integrating long short-term memory network and word embedding for executing any one of claims 1 to 3, comprising: A model building module, used to build a text sentiment analysis model, wherein the text sentiment analysis model at least includes a word embedding unit, a first long short-term memory network unit, and a second long short-term memory network unit; A parameter setting module, used to set the parameters of the word embedding unit, wherein the parameters of the word embedding unit include the vector representation of each word in the vocabulary; The parameter setting module is also used to set the initial parameters of the first long short-term memory network unit and the initial parameters of the second long short-term memory network unit; A parameter optimization module, configured to adjust the initial parameters of the first long short-term memory network unit and the initial parameters of the second long short-term memory network unit based on the parameters of the word embedding and using training samples to obtain an optimized first long short-term memory network unit and an optimized second long short-term memory network unit; A vector generation module is used to generate a word vector sequence corresponding to the text to be recognized using a word embedding unit; A sequence extraction module, used to generate a hidden state sequence of the text to be recognized based on the word vector sequence corresponding to the text to be recognized using the optimized first long short-term memory network unit; The emotion recognition module is used to use the optimized second long short-term memory network unit to identify the emotion of the text to be recognized based on the hidden state sequence of the text to be recognized.