Emotion classification algorithm combining double attention mechanism and Bi-LSTM

Abstract

The invention discloses a sentiment classification algorithm combining a double attention mechanism and a Bi-LSTM (Bidirectional Long Short Term Memory), which belongs to a text sentiment analysis technology and comprises the following steps of: preprocessing data; constructing a model; model training; and evaluating and optimizing the model. The double attention mechanism of the algorithm can help the model to understand the position information of the words or sentences in the text and quickly focus on the key information in the text, so that the speed and accuracy of positioning the key information are improved. The input sequence is processed through the bidirectional LSTM, the context representation of each word is extracted, then the self-attention mechanism is used for weighting representation, important sentiment vocabularies are highlighted, and therefore the sentiment analysis accuracy is improved. In addition, the model is not only suitable for aspect-based sentiment analysis, but also suitable for natural language processing tasks such as relation extraction and the like, and the wide application potential of the model is shown.

Description

Technical Field

[0001] The present invention relates to a text sentiment analysis technology, and in particular to a sentiment classification algorithm combining a dual attention mechanism with Bi-LSTM. Background Art

[0002] At present, with the vigorous development of social networks, users have posted a large amount of content with clear emotional tendencies on various social networking sites, Weibo, e-commerce platforms, etc. These contents cover all levels of society, and detailed statistics, in-depth analysis and systematic induction of them are indispensable for product market research, capturing online public opinion and analyzing social hot spots.

[0003] Sentiment analysis of text can be divided into three categories: sentiment analysis based on dictionaries and rules, sentiment analysis based on traditional machine learning, and sentiment analysis based on deep learning. In recent years, deep learning technology, especially recurrent neural networks (RNN) and long short-term memory networks (LSTM), has made significant progress in sentiment classification tasks. Summary of the invention

[0004] The main purpose of the present invention is to provide a sentiment classification algorithm that combines a dual attention mechanism with Bi-LSTM to solve the problems existing in existing sentiment classification technologies, such as the subjectivity and limitations of feature extraction, the limited expressive power of shallow models, the sensitivity to data imbalance, the lack of cross-language processing capabilities, the low computational efficiency and the weak model interpretability.

[0005] The technical solution adopted by the present invention is: a sentiment classification algorithm combining a dual attention mechanism and Bi-LSTM, comprising:

[0006] Data preprocessing;

[0007] Model building;

[0008] Model training;

[0009] Model evaluation and optimization.

[0010] Furthermore, the data preprocessing includes:

[0011] Word segmentation and vocabulary building: First, the input text data is merged to form a word list separated by spaces to achieve word segmentation. Then, the word list is traversed to build a vocabulary that maps each word to a unique index.

[0012] Generate training data set: Convert the text information in the segmented text data into word indexes according to the vocabulary, obtain the word index sequence, and define the corresponding sentiment labels.

[0013] Furthermore, the model building includes:

[0014] Use the nn.Embedding function to convert word vectors into fixed-size word vectors;

[0015] Use nn.LSTM function to capture the contextual information of text;

[0016] The Softmax function is used to normalize the attention weights to focus on the key information in the text.

[0017] Furthermore, the model training includes:

[0018] Define the loss function and optimizer: use the nn.CrossEntropyLoss function as the loss function and the optim.Adam parameter as the optimizer;

[0019] Training process: Optimize model parameters through back-propagation algorithm to minimize the loss function;

[0020] Forward propagation: calculate the output and loss of the model;

[0021] Back propagation: calculate gradients and update model parameters;

[0022] Evaluate regularly: Print the loss every 1000 epochs to monitor training progress.

[0023] Furthermore, the model evaluation and optimization includes: evaluating the performance of the model on a validation set, evaluating the model according to common model indicators, and adjusting model parameters based on the results.

[0024] Advantages of the present invention:

[0025] This paper proposes an innovative sentiment classification algorithm which combines a dual attention mechanism with a bidirectional long short-term memory network (Bi-LSTM).

[0026] Compared with other technologies, the dual attention mechanism in this technical solution can help the model understand the position information of words or sentences in the text, quickly focus on the key information in the text, and improve the speed and accuracy of locating key information.

[0027] The input sequence is processed by bidirectional LSTM to extract the contextual representation of each word, and then the self-attention mechanism is used to weight the representation to highlight important sentiment words, thereby improving the accuracy of sentiment analysis. In addition, the model is not only suitable for aspect-based sentiment analysis, but also for natural language processing tasks such as relation extraction, showing its wide application potential.

[0028] The introduction of this algorithm not only solves the problems of existing models in processing long texts, but also brings new research directions and application prospects to the field of sentiment analysis.

[0029] In addition to the above-described purposes, features and advantages, the present invention has other purposes, features and advantages. The present invention will be further described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The drawings constituting a part of this application are used to provide a further understanding of the present invention. The illustrative embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0031] Figure 1 It is a flowchart of the sentiment classification algorithm of the present invention. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0033] The present invention relates to a sentiment classification algorithm combining a dual attention mechanism with Bi-LSTM, aiming to improve the accuracy and efficiency of sentiment classification.

[0034] The calculation process of the attention value is as follows:

[0035]

[0036] Among them, L x Indicates the length of Source, Similarity(Query,Key i ) is the weight coefficient.

[0037] From the above formula, we can see that the calculation of attention value depends on the three tensors of Query, Key and Value. Each item in the source sequence can be regarded as a series of<Key,Value> The key-value pairs are constructed, and an item in the target sequence is regarded as a Query element. By calculating the similarity between the Query and each Key, the scaled dot product method is used to calculate the similarity between the Query and the Key, the Query and Key vectors are aggregated, and the dot product calculation is performed on their d-dimensional vectors (corresponding elements are multiplied and then summed), that is, Get a scalar value, then scale it to get the similarity matrix, and finally perform a Softmax regression operation to get the attention weight.

[0038] This method can effectively control the numerical range, improve the stability of calculation, and greatly improve the efficiency. The model effect is better and can be seamlessly combined with other model components, greatly improving the processing ability of the model, thereby obtaining the attention weight corresponding to each Value. Then the weight is weighted and summed with the corresponding Value to obtain the final attention value.

[0039] The dual attention mechanism can effectively locate the key information in the text content. Compared with single-head attention, the dual attention mechanism can pay more comprehensive and accurate attention to the text information of different dimensions in the text, capture more complex emotional expressions, and identify more subtle and implicit emotional information in the text.

[0040] Bi-LSTM has certain advantages over traditional RNN. It can use input gate function, forget gate function and output gate function to control the memory of the neural network, so as to judge the input information more effectively and save more valuable information. At the same time, compared with the existing use of Bi-LSTM on the market to perform sentiment analysis on text content, the present invention effectively combines the dual attention mechanism with Bi-LSTM to more accurately find the corresponding sentiment information for the text content, and improves the text classification results by combining the text context information, which has a significant improvement in the accuracy and efficiency of sentiment classification. The combination method is described as follows:

[0041] It consists of a model architecture input layer, a dual attention mechanism layer, a Bi-LSTM layer, and an output layer. The dual attention mechanism layer is divided into two modules, one is the channel attention module, and the other is the spatial attention module. It focuses on the importance distribution of input features in the spatial dimension and the importance of different feature channels, and weights the input features through attention weighting. Bi-LSTM consists of a forward LSTM and a backward LSTM. The forward LSTM and the backward LSTM in the Bi-LSTM layer process the input sequence respectively to capture the information of the sequence. A more refined feature representation is obtained by fusing the features of the outputs of the two, and end-to-end back-propagation training is performed on them so that they can collaborate and optimize together.

[0042] The formula of the forget gate is as follows, which uses the hidden state h at the previous moment t-1 and the current input data x t Calculate and map the result using Sigmoid activation function, and the output result is f(t). The value of f(t) is between 0 and 1. When the value of f(t) is close to 0, it means that it is more likely to discard information, thus achieving the purpose of forgetting; when the value of f(t) is close to 1, it means that it is more likely to retain information.

[0043] f(t)=σ(Wf. [h t-1 ,x t ]+b f )

[0044] The function of the input gate is to update the cell state. It determines how much information of the network input at the current moment can be stored in the cell state. The formula of the input gate is as follows. It consists of two parts. The first part uses the Sigmoid activation function, and the output result is i t , the output value of Sigmoid will determine which values ​​can be retained. The second part uses the tanh activation function, and the output result is

[0045] i t =σ(W o .[h t-1 ,x t ]+b i )

[0046]

[0047] The formula of the output gate is as follows, which controls the number of cell states that are filtered. First, the sigmoid activation function is used to calculate and obtain an O value in the range of [0,1]. t , and then multiply it by the cell state processed by the tanh activation function to get the output h of this layer t .

[0048] O t =σ(W o. [h t-1 ,x t ]+b o )

[0049] h t =o t *tanh(C t )

[0050] The technical solution mainly includes the following steps:

[0051] (1) Data preprocessing

[0052] Word segmentation and vocabulary building: First, merge the input text data to form a word list separated by spaces to achieve word segmentation, and then traverse the word list to build a vocabulary with a unique index for each word mapping. Suppose there is a word set W = {ω1,ω2,…,ω n}Let function f:W→{1,2,…,n} be for any word ω i ∈W,f(ω i)=k, where k is a unique positive integer representing the word ω i The corresponding index.

[0053] Generate training data set: Convert the text information in the text data of the segmented text into word index according to the vocabulary, obtain the word index sequence, and define the corresponding sentiment label. First, convert the text to word index, and set the vocabulary V = {v1, v2, ..., v m}, the text after the word segmentation table is T = {t1, t2, ..., t n}, define a function f; V×T→{1,2,…,m}, so that each word t in the text T i , find its position index in the vocabulary table V, that is, I i =f(v j ,t i ), where v j is the same as t in the vocabulary i Matching words, get word index sequence I = {I1, I2, ..., I n}; Then define the emotion label, assuming that the emotion label set is S = {s1, s2, ...s k}, define a mapping function g:T→S, and map the text T to the corresponding sentiment label according to its overall sentiment tendency.

[0054] (2) Model construction

[0055] First, use the nn.Embedding function to convert the word vector into a fixed-size word vector. The working principle of the nn.Embedding function is as follows:

[0056] ① Create a trainable embedding matrix where each row corresponds to the vector representation of a word.

[0057] ②One-hot encoding vector of the input word.

[0058] ③Use one-hot encoding as an index to find the corresponding word vector from the embedding matrix.

[0059] Then, we use the nn.LSTM function to capture the contextual information of the text. The calculation formula is as follows. It can process the input sequence, encode each element in the sequence and save their states to enhance the semantic understanding ability of the model and capture the previous and next dependencies in the input sequence.

[0060] i t =σ(W ii x t +b ii +W hi h t-1 +bhi )

[0061] f t =σ(W if x t +b if +W hf h t-1 +b hf )

[0062] g t =tanh(W ig x t +b ig +W hg h t-1 +b hg )

[0063] o t =σ(W io x t +b io +W ho h t-1 +b ho )

[0064] c t =f t ⊙c t-1 +i t ⊙g t

[0065] h t =o t ⊙tanh(c t )

[0066] Among them, h t represents the hidden state at time t, c t represents the cell state at time t, x t represents the hidden state of the previous layer at time t or the input of the first layer at time t, i t 、f t , g t , o t They represent input gate, forget gate, cell and output gate respectively.

[0067] By designing a dual attention mechanism, the model can automatically identify and weight important sentiment features in the text.

[0068] The first step is attention: the final hidden state of the LSTM layer is calculated by torch.bmm. The dot product between the LSTM output and the final hidden state is used to obtain the attention weight. First, the data is batch processed and expanded, and then the torch.bmm function is used to calculate H and h' final Batch matrix multiplication between .

[0069] Mathematically speaking, for each sample n∈{1,2,…,N}, at time t∈{1,2,…,T}, the formula for calculating the dot product is: After torch.bmm operation, a new tensor S is obtained, whose dimension is R N×T×1 , where each element S in S n,t It is the dot product result of the LSTM output and the final hidden state at the corresponding position. Finally, the Softmax function is used to normalize the attention weights to focus on the key information in the text.

[0070] Second-step attention: Based on the first-step attention, the normalized attention weights are used to perform weighted summation on the LSTM output to obtain the context vector, thereby further focusing on more fine-grained sentiment features.

[0071] Output layer: maps the processed feature vector to the result of sentiment classification.

[0072] torch.bmm is a function in PyTorch that performs batch matrix multiplication operations. It calculates the matrix multiplication of two three-dimensional tensors with the same batch size to simplify calculations and reduce dimensions. Suppose there are two three-dimensional tensors A and B, the dimensions of A are (N, a, b), and the dimensions of B are (N, b, c). torch.bmm will perform matrix multiplication on the matrices of the corresponding batches in these two tensors. That is, for the nth batch, calculate C n =A n *B n , where A n is the nth matrix of size (a*b) in A, B n is the nth matrix of size (b*c) in B. The dimension of the resulting tensor C is (N, a, c). The calculation formula is as follows:

[0073] output[i]=input[i]×mat2[i]

[0074] Softmax regression is an extension of Logistic regression in multi-classification problems. It is used to normalize the classification results to form a probability distribution. In binary classification problems, the Sigmoid function maps a real number to the interval (0,1), indicating the probability that the sample belongs to a certain category. In multi-classification problems, Softmax regression performs exponential operations and normalization on the scores of multiple categories so that the sum of the output probabilities is 1, thereby being able to clearly indicate the probability that the sample belongs to each different category. Its function is similar to the Sigmoid function in binary classification. The specific formula is as follows:

[0075]

[0076] Among them, w c is the weight vector of the cth class, x is the sample, and the decision function of softmax regression can be expressed as:

[0077]

[0078] During the forward propagation of the model, the input is first converted to word vector representations through the embedding layer, and then these word vectors are passed as input to the bidirectional LSTM layer. Next, a custom dual attention mechanism is applied, which calculates attention weights based on the output of the LSTM and the last hidden state. These weights reflect the importance of different positions in the input sequence. By weighted averaging the output of the LSTM using these attention weights, a context vector that comprehensively considers the information of each position in the sequence can be obtained. Finally, this context vector is converted into a sentiment category prediction through the output layer, thus completing the entire sentiment classification process.

[0079] (3) Model training

[0080] Define the loss function and optimizer: Use the nn.CrossEntropyLoss function as the loss function and the optim.Adam parameter as the optimizer.

[0081] The cross entropy loss function is a commonly used loss function, usually used in multi-classification problems. It is calculated by comparing the probability distribution predicted by the model with the probability distribution of the true label, and then calculating the cross entropy between them. This function combines the two functions nn.LogSoftmax() and nn.NLLLOSS(), and can directly process the output of the model without manual softmax operation. The calculation formula is as follows:

[0082] loss(x,y)=L={l1,...,l n ) T ,

[0083]

[0084] Among them, w yn Indicates the weight of each category, the default is 1, x n ,y n Indicates the probability of the corresponding target class.

[0085] Adam is one of the most commonly used optimization algorithms in deep learning. It combines the advantages of momentum and adaptive learning rate to achieve fast convergence, good generalization performance, and robustness to noisy gradients. In addition, it also has built-in bias correction to deal with initialization problems. Its calculation formula is as follows:

[0086]

[0087]

[0088] Among them, m t 、v t represents the moving average of past gradients and squared gradients, and β1 and β2 represent the hyperparameters of momentum and squared gradient.

[0089] Training process: Optimize model parameters through the back propagation algorithm to minimize the loss function. The formula of the back propagation algorithm is:

[0090]

[0091] Among them, each θ i Represents a weight matrix or bias vector, etc., where x is the input data and y is the corresponding true label. The loss function is recorded as α is the learning rate, which determines the step size of parameter update.

[0092] Forward propagation: Calculate the output and loss of the model. First, prepare and input the data. After obtaining the data from the training set, extract its feature vector and the corresponding true label, and then send it to the model layer by layer for calculation. In each layer, calculate h according to the weight W, bias b and activation function f of the layer. l+1 =f(W l h l +b l ), after multiple layers of calculation, the output of the model is obtained, and finally the loss function L is calculated based on the model output and the true label.

[0093] Back propagation: Calculate the gradient and update the model parameters. Starting from the loss function L, calculate the gradient of the loss function to the model output according to the chain rule, and then start from the last layer of the model and back propagate the gradient to each previous layer. For each layer, the gradient is calculated according to the calculation method of the layer and the derivative of the activation function. For a fully connected layer in At the same time, the gradient of loss with respect to bias is calculated Finally, the optimization algorithm is used to update the model parameters based on the calculated gradients.

[0094] Regular evaluation: Print the loss every 1000 epochs to monitor the training progress. After each epoch is completed, add 1 to the counter count. When the count reaches 1000, calculate the average loss of the current model on the training set and print the loss value. After printing the loss, reset the counter to 0 and continue the subsequent training process. In this way, you can regularly check the training progress and performance changes of the model during the training process.

[0095] During the training process, some hyperparameters are first set. Hyperparameters include learning rate, hidden layer size, number of layers, etc. The values ​​of these hyperparameters will affect the training process and performance of the model. Then, the Bi-LSTM model is instantiated and moved to the specified device. In the training cycle, the Adam optimizer is used to update the model parameters, and the loss is printed regularly. When the loss is less than 0.001 for 30 consecutive rounds, the training is terminated early and the model is saved. The formula for updating the model parameters by the Adam optimizer is as follows:

[0096] m t =β1m t-1 +(1-β1)g t

[0097]

[0098]

[0099] Among them, θ represents the model parameters, g t represents the gradient of the parameter, α represents the learning rate, β1 and β2 represent the exponential decay rate, and m t and v t denote the first-order moment estimate and the second-order moment estimate, respectively. and Represents the corrected first-order moment estimate and second-order moment estimate, ∈ represents a very small number (to prevent division by zero).

[0100] (4) Model evaluation and optimization

[0101] Evaluate the performance of the model on the validation set, evaluate the model according to common model indicators, and adjust the model parameters based on the results. The application method is as follows:

[0102] ①Accuracy:

[0103] ②Accuracy:

[0104] ③Recall rate:

[0105] ④F1 value:

[0106] Parameter optimization: Find the optimal model parameters through grid search or other optimization methods.

[0107] The technical problem to be solved by the present invention is to provide a sentiment classification algorithm combining a dual attention mechanism and Bi-LSTM, aiming to solve the following technical problems existing in the existing sentiment classification technology:

[0108] (1) Subjectivity and limitations of feature extraction: Existing technologies rely on manually extracted features, and feature selection relies on domain knowledge, which lacks universal applicability. Existing technologies cannot identify all important features or believe that certain important features are not important in specific tasks, resulting in these features often not being able to fully and accurately reflect the deep semantic information of the text, resulting in limited accuracy of sentiment classification. The current automated feature selection techniques (such as random forests, PCA, etc.) can reduce the impact of subjectivity, but these methods are still limited by the design of the algorithm and the specific structure of the input data, and need to be further optimized.

[0109] (2) Shallow models have limited expressive power [improving text information integration capabilities]: Shallow models usually have low complexity, so their expressive power is limited and they have difficulty processing complex patterns or high-order feature interactions. Traditional sentiment classification models such as SVM and Naive Bayes have relatively weak expressive and learning capabilities when processing complex sentiment expressions, and have difficulty capturing deep semantic relationships in the data.

[0110] (3) Sensitivity to data imbalance [Enhancing the accuracy of sentiment analysis]: In sentiment classification tasks, the data of positive and negative emotions are often unbalanced. The use of traditional methods has alleviated the imbalance to a certain extent, but they often require additional computing resources and may damage the generalization ability of the model. They usually perform poorly on such imbalanced datasets.

[0111] (4) Lack of cross-language processing capabilities: The currently designed models have good performance on single Chinese or English datasets. In order to be applicable to datasets in different languages, a cross-language reading comprehension system needs to be built. Cross-language processing requires the model to be able to understand and convert semantic information between different languages. The current method is to use multilingual training data for end-to-end multilingual training models. Although it has made some progress, there are still problems such as errors in language understanding and translation and low language coverage. It is necessary to further improve the model's ability to understand language, expand the language library, and focus on key information.

[0112] (5) Low computational efficiency and weak model interpretability: Deep neural networks can learn high-level representations of data through multiple layers of nonlinear transformations, thus achieving remarkable results in many fields. However, training models requires a lot of computing resources and time, and intensive computing is also required when forward propagating input data. The decision-making process of deep networks is not transparent, and the interpretability of the model needs to be improved to allow users to have a deeper understanding of computer mechanics.

[0113] The main objects of the present invention are:

[0114] (1) Improve the accuracy of sentiment classification: By introducing a dual attention mechanism and deep learning, the Bi-LSTM network is used to capture the bidirectional semantic information in the text sequence, and the attention mechanism is combined to weight the key information, so that the model can automatically focus on the key sentiment information in the text, thereby more accurately understanding and classifying the sentiment color of the text. And through machine learning, the accuracy of sentiment prediction is further improved.

[0115] (2) Enhance the expressive power of the model: The BERT pre-trained language model based on the Transformer architecture is used to effectively capture language features and context features. At the same time, the Bi-LSTM structure is used to enhance the model's ability to capture text context information, further improving the model's ability to process complex text patterns and improving the model's understanding of complex emotional expressions.

[0116] (3) Improve the data imbalance problem: By designing a reasonable model structure and training strategy, the overfitting of the model to the majority class during the training process can be reduced, so that the model can still maintain good classification performance when facing an unbalanced data set, thereby improving the model's generalization ability to all categories.

[0117] (4) Improve cross-language processing capabilities: In view of the limitation that existing intelligent reading systems can only process a single language, this patent intends to design a universal machine reading comprehension model that can adapt to multiple language environments and process a wider range of language types.

[0118] (5) Enhance computational efficiency and model interpretability: Technologies such as model pruning and quantization will be introduced to reduce model complexity. Visualization tools and techniques will be used to explain the internal working mechanism of the model so that users can more intuitively understand the reasons for the model output results.

[0119] The algorithm of the present invention has the following advantages and effects:

[0120] It has the following advantages and technical effects:

[0121] (1) Improve the accuracy of sentiment classification

[0122] Dual attention mechanism: The dual attention mechanism is widely used in the field of computer vision. It consists of a spatial attention module and a channel attention module. The essence of this mechanism is to allocate different resources to different visual observation objects according to their importance. This resource allocation is mostly manifested in weight allocation. In this application, the spatial attention module mainly focuses on the importance of spatial position, helping the model understand the position information of words or sentences in the text and the spatial relationship between them. It can help the model better capture spatial context information and quickly focus on key information in the text, thereby improving the speed and accuracy of locating key information. The channel attention module focuses on the importance of different dimensions in the text. The channel attention module learns the importance of each channel and weights different feature dimensions, thereby enhancing features that are useful for the task and suppressing irrelevant features, so that attention is focused on model features with heavier weights. For example, if the emotional feature is taken as the feature with the heaviest weight, attention can be focused on the emotional feature, so that attention can be further focused on more fine-grained emotional features.

[0123] Bidirectional LSTM layer: Bidirectional LSTM can obtain information from both the forward and reverse directions of the sequence, thus providing richer contextual information. This bidirectional information flow helps the model better capture the features and patterns in the input sequence. The use of bidirectional LSTM layers provides richer contextual information and enhances the model's representation capabilities, enabling the model to better capture features, patterns, and dependencies when processing sequence data, so that the model can more comprehensively understand the emotional content of the text.

[0124] (2) Improving the efficiency of sentiment classification

[0125] Word embedding layer: Using nn.Embedding to convert word indexes into fixed-size word vectors can better capture semantic information and make similar data closer in the embedding space. At the same time, it maps high-dimensional data to low-dimensional space, speeding up the model training process, reducing computing and memory requirements, and improving the efficiency of sentiment classification.

[0126] Optimizer selection: Use optim.Adam as the optimizer. The Adam optimizer has efficient computing power and fast convergence speed. It has good adaptability to the instability of the objective function and the problem of gradient sparsity, and can handle complex and non-convex optimization problems more efficiently. At the same time, the hyperparameters of the Adam optimizer are well interpretable, which helps users better understand and adjust the parameters. The Adam optimizer can improve the efficiency of sentiment classification while achieving better optimization efficiency.

[0127] (3) Data security

[0128] Word segmentation and vocabulary building: After the continuous text is segmented into meaningful word units, the words are organized and classified according to the word segmentation results to form a vocabulary that can be used by the natural language processing system. Each word is mapped to a unique index to ensure that the data is correctly understood and analyzed, which helps to more accurately identify sensitive information, thereby more effectively encrypting, desensitizing or monitoring data. It can also improve the efficiency of indexing, speed up the retrieval of sensitive information, and improve the response speed of data leaks. Protect the privacy and security of original text data and avoid the leakage of sensitive information.

[0129] (4) Network resource saving

[0130] Model training and evaluation: Through regular evaluation and parameter optimization, the model can evaluate performance on the validation set, detect whether the model has problems such as overfitting or underfitting, and adjust the model in time. Parameter optimization adjusts model parameters and structure to obtain a better model. Improve the efficiency and effect of model development, ensure the generalization ability of the model, reduce unnecessary computing resource consumption, and improve resource utilization efficiency.

[0131] Application and deployment: Use the tested model to perform sentiment classification on new texts. By visualizing the attention weights, analyze the model's attention to different words, improve work efficiency, reduce data processing costs, save network resources, and quickly respond to sentiment classification needs.

[0132] In summary, the technical solution of the present invention mainly captures emotional information and more fine-grained emotional features more accurately through the combination of dual attention mechanism and bidirectional LSTM, thereby improving the accuracy of emotional classification. The word embedding layer and the selection of Adam as the optimizer not only improve the efficiency of emotional classification, but also reduce resource consumption. In addition, this solution also ensures data security and saves network resources.

[0133] Compared with the common traditional machine learning methods (such as Naive Bayes, support vector machine, etc.) or simple neural network models (such as unidirectional LSTM or CNN) in the market, the technical solution of the present invention has the following significant advantages:

[0134] More accurate sentiment capture: By introducing bidirectional LSTM and attention mechanisms, the contextual dependencies of sentiment information in texts can be captured more effectively. Especially when processing long texts, the correlation within and between sentences can be better understood, thereby improving the accuracy of sentiment classification.

[0135] Efficient learning ability: The Adam optimizer is used to converge to the optimal solution faster while reducing the risk of overfitting, so that the model can achieve good performance even on a limited data set.

[0136] Lower resource consumption: By optimizing the model structure and training strategy, this solution can reduce the demand for computing resources while ensuring high accuracy, which is especially important for large-scale deployment and real-time processing.

[0137] Better scalability: The model is designed with the possibility of future expansion in mind, making it easy to integrate into the existing system architecture and facilitating model upgrades and maintenance.

[0138] Higher security: This solution can not only ensure data security, but also effectively protect users' privacy information and avoid sensitive data leakage.

[0139] These technical effects are achieved thanks to the careful design and optimization of key technical solutions, which gives the present invention significant technical advantages and application value in the field of sentiment classification.

[0140] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A sentiment classification algorithm combining dual attention mechanism and Bi-LSTM, characterized in that: include: Data preprocessing; Model building; Model training; Model evaluation and optimization.

2. The sentiment classification algorithm combining dual attention mechanism and Bi-LSTM according to claim 1, characterized in that: The data preprocessing includes: Word segmentation and vocabulary building: First, the input text data is merged to form a word list separated by spaces to achieve word segmentation. Then, the word list is traversed to build a vocabulary with a unique index for each word mapping. Generate training data set: Convert the text information in the segmented text data into word indexes according to the vocabulary, obtain the word index sequence, and define the corresponding sentiment labels.

3. The sentiment classification algorithm combining dual attention mechanism and Bi-LSTM according to claim 1, characterized in that: The model construction includes: Use the nn.Embedding function to convert word vectors into fixed-size word vectors; Use nn.LSTM function to capture the contextual information of text; The Softmax function is used to normalize the attention weights to focus on the key information in the text.

4. The sentiment classification algorithm combining dual attention mechanism and Bi-LSTM according to claim 1, characterized in that: The model training includes: Define the loss function and optimizer: use the nn.CrossEntropyLoss function as the loss function and the optim.Adam parameter as the optimizer; Training process: Optimize model parameters through back-propagation algorithm to minimize the loss function; Forward propagation: calculate the output and loss of the model; Back propagation: calculate gradients and update model parameters; Evaluate regularly: Print the loss every 1000 epochs to monitor training progress.

5. The sentiment classification algorithm combining dual attention mechanism and Bi-LSTM according to claim 1, characterized in that: The model evaluation and optimization includes: evaluating the performance of the model on a validation set, evaluating the model according to common model indicators, and adjusting the model parameters based on the results.

Images