An industrial text sentiment analysis method based on an asymmetric position weighting strategy

Abstract

This invention discloses an industrial text sentiment analysis method based on an asymmetric position-weighted strategy, belonging to the field of natural language processing technology. The method includes: acquiring a text sample dataset; segmenting each sentence to obtain a word vector representation sequence; manually pre-annotating aspect words to obtain a training set; constructing a sentiment analysis model based on an asymmetric position-weighted strategy, including a Bi-GRU layer, a Self-Attention layer, an adaptive weighting strategy, an aspect information fusion layer, and a sentiment analysis layer connected sequentially; training the model using the training set to obtain an optimized sentiment analysis model; collecting actual text data; processing the data using the optimized sentiment analysis model to obtain sentiment analysis results. This invention introduces an adaptive weighting strategy to compensate for the problem of the Self-Attention layer ignoring the positional relationships between context words, further enhancing the expressive power of the sentiment analysis model.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to an industrial text sentiment analysis method based on an asymmetric position-weighted strategy. Background Technology

[0002] Sentiment analysis, also known as opinion mining or sentiment analysis, is a crucial task in natural language processing. Its purpose is to analyze, process, summarize, and infer the subjective sentiment of text. Currently, the main methods of sentiment analysis can be categorized into three types: rule-based and dictionary-based, machine learning, and deep learning. Dictionary-based sentiment analysis requires manually constructing a sentiment dictionary, but the same sentiment word may express different meanings in different domains or contexts, leading to poor cross-domain performance and an inability to understand contextual semantic relationships. Machine learning-based sentiment analysis methods cannot fully utilize the contextual information in the text, resulting in inaccurate classification results. In contrast, deep learning methods can identify and extract latent or hidden contextual semantic information in text, enabling more accurate sentiment polarity predictions.

[0003] In existing deep learning-based sentiment classification models, Tang et al. directly divided a fixed-size window centered on aspect-related words and studied the influence of context words within the window on aspect sentiment polarity. However, this method ignores context words outside the window. When the window size is not set appropriately, it is easy to miss words that have a significant impact on aspect sentiment polarity, thus affecting the model's performance. Li et al.'s TNet uses a distance-based linear decay weight allocation method, which is too uniform for sentences with grammatical structure. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention incorporates an adaptive positional weight transformation function and a multi-sentence-level Bi-GRU into the sentiment analysis model, which has achieved good results in sentiment analysis in the industrial enterprise sector.

[0005] To achieve the above objectives, this invention provides an industrial text sentiment analysis method based on an asymmetric position-weighted strategy, comprising the following steps:

[0006] (1) Obtain a text sample dataset, segment each sentence to obtain the word vector representation sequence of the sentence; and manually pre-annotate aspect words to form a training set;

[0007] (2) Construct a sentiment analysis model based on an asymmetric position weighting strategy, including a Bi-GRU layer, a Self-Attention layer, an adaptive weighting strategy, an aspect information fusion layer, and a sentiment analysis layer connected in sequence; train the model using the training set, calculate the loss function, and obtain an optimized sentiment analysis model based on an asymmetric position weighting strategy after the cutoff condition is met.

[0008] (3) Collect actual text data, process the data using the optimized sentiment analysis model based on asymmetric position weighting strategy, and obtain sentiment analysis results.

[0009] Furthermore, in step (1), the sentence in the input text is segmented to obtain the word vector representation sequence by: putting the sentence into the global word vector GloVe pre-trained model to obtain the word vector representation sequence.

[0010] Furthermore, the sentiment analysis model based on the asymmetric position-weighted strategy is specifically as follows:

[0011] (2.1) Input the word vector representation sequence of the sentence into the Bi-GRU layer to obtain the hidden layer states of the forward GRU and the reverse GRU respectively. After concatenation, the hidden layer state at the current time is obtained.

[0012] (2.2) In the Self-Attention layer, the hidden layer states output by the Bi-GRU layer for each word vector in the sentence are similar to each other through the vector inner product. The corresponding Self-Attention weights are obtained by Softmax normalization, and then the new hidden layer state at the current time is obtained.

[0013] (2.3) For each sentence, an asymmetric context position weight transformation function is introduced. Based on the position of the words marked as aspect words in the sentence, the final word vector representation is obtained by using the asymmetric context position weight transformation function.

[0014] (2.4) A learnable aspect vector Um is constructed in the aspect information fusion layer. The aspect information containing the specified category is obtained through the Global-Attention mechanism. The similarity between each word vector and the aspect vector of the sentence is calculated using the inner product. The Global-Attention weight is obtained by normalization through softmax. Finally, a sentence vector that integrates the specified aspect information is obtained.

[0015] (2.5) In the sentiment analysis layer, after the sentence vector representation is passed through a fully connected layer, the sentiment polarity distribution of the sentence is obtained through the softmax function, and the classification result with the highest probability is selected as the sentiment classification result.

[0016] The hidden layer state at the current moment in step (2.1) is specifically as follows:

[0017]

[0018] in: These are the hidden layer states of the positive GRU for the word vectors. t is the hidden state of the inverse GRU of the word vector, and t is the position of the word vector in the sentence.

[0019] Furthermore, the similarity calculation in step (2.2) specifically involves:

[0020]

[0021] The Self-Attention weights are specifically as follows:

[0022]

[0023] in: These are the hidden layer states corresponding to the i-th and j-th word vectors in the sentence;

[0024] The new hidden layer state is specifically represented as follows:

[0025]

[0026] Where t is the position of the word vector in the sentence, and T is the length of the sentence.

[0027] Furthermore, the asymmetric context position weight transformation function in step (2.3) is specifically as follows:

[0028]

[0029] The final word vector is represented as follows:

[0030]

[0031] in, This represents the normalized value of the word vector at position t in the sentence, where T is the sentence length, Γ is the gamma function, and α is the normalized value. k ,β k This is a hyperparameter, where k represents the index. This represents the new hidden layer state of the word vector;

[0032] The k is calculated in the following way:

[0033] For each set of hyperparameters, calculate the distance:

[0034]

[0035] Compare all d j Find the minimum distance d. min The corresponding index is K;

[0036]

[0037] Where: j is the index value, and These represent the normalized values ​​of the word vectors labeled as aspect words in the sentence, specifically the start position l and end position r. T is the length of the sentence.

[0038] Furthermore, step (2.4) specifically involves:

[0039] Similarity between each final word vector of the sentence and the aspect vector:

[0040] Similarity(u m ,v t )=u m ·v t

[0041] The Global-Attention weights are:

[0042]

[0043] The sentence vector representation of the fused aspect information is as follows:

[0044]

[0045] Where: T is the length of the sentence, v t This is the final word vector for the sentence.

[0046] Furthermore, the emotional polarity distribution of the sentences in step (2.5) is specifically as follows:

[0047]

[0048] Where: y is the emotional polarity, W o and b o These are the parameters learned in the sentiment polarity classification layer. Let y be the probability distribution of emotional polarity.

[0049] Furthermore, the model is trained using the cross-entropy loss function, as follows:

[0050]

[0051] in: The predicted probability for the sentiment polarity classification of the i-th sentence; training stops when the set number of training iterations is reached or the accuracy of the loss no longer improves.

[0052] This invention also provides an industrial text sentiment analysis system based on an asymmetric position-weighted strategy, comprising:

[0053] Data acquisition module: used to acquire text sample datasets;

[0054] The data preprocessing module is used to segment each sentence to obtain the word vector representation sequence of the sentence; and manually pre-annotate aspect words to form a training set;

[0055] Modeling and training module: used to construct a sentiment analysis model based on an asymmetric position weighting strategy, including a Bi-GRU layer, a Self-Attention layer, an adaptive weighting strategy, an aspect information fusion layer, and a sentiment analysis layer connected in sequence; the model is trained using the training set, the loss function is calculated, and an optimized sentiment analysis model based on an asymmetric position weighting strategy is obtained after the cutoff condition is met.

[0056] Analysis module: Used to collect actual text data, process the data using the optimized sentiment analysis model based on asymmetric position weighting strategy, and obtain sentiment analysis results.

[0057] The beneficial effects of this invention are:

[0058] This invention proposes a sentiment analysis method for industrial product evaluation texts based on an asymmetric position weighting strategy. This method compensates for the fact that the vector representation obtained by the Self-Attention layer ignores the positional relationship between context words, thereby further enhancing the expressive power of the sentiment analysis model. Attached Figure Description

[0059] Figure 1 This is a schematic diagram of the process of an industrial text sentiment analysis method based on an asymmetric position weighting strategy according to an embodiment of the present invention. Detailed Implementation

[0060] The present invention will be further explained below with reference to the accompanying drawings and embodiments.

[0061] like Figure 1 As shown, this embodiment of the invention provides a method for industrial text sentiment analysis based on an asymmetric position-weighted strategy, comprising the following steps:

[0062] S101. Obtain the text sample dataset, segment each sentence to obtain the word vector representation sequence of the sentence; and manually pre-annotate aspect words to form a training set;

[0063] The text sample set used in this embodiment of the invention comes from existing industrial product evaluation texts.

[0064] The obtained text sample dataset needs to be preprocessed by putting the sentences in the text into the global word vector GloVe pre-trained model to obtain the word vector representation sequence.

[0065] Then, the words in the sentences are manually pre-annotated with aspect terms. In sentiment analysis, aspect terms refer to words or phrases in the text that represent a specific aspect or object. Aspect terms are typically used to describe different aspects or attributes of the entity, event, product, or service being evaluated. In fine-grained sentiment analysis, we focus on determining the sentiment polarity of each aspect term in the text.

[0066] For example, when conducting sentiment analysis on restaurant reviews, aspect words can be words like "service," "food," and "price," which represent the reviewer's evaluation of different aspects of the restaurant. By analyzing the sentiment of each aspect word, we can understand the reviewer's likes or dissatisfactions regarding each aspect.

[0067] The goal of fine-grained sentiment analysis is to determine the sentiment polarity of specific words in a text, rather than simply classifying the sentiment of the entire text.

[0068] S102. Construct a sentiment analysis model based on an asymmetric position weighting strategy, including a Bi-GRU layer, a Self-Attention layer, an adaptive weighting strategy, an aspect information fusion layer, and a sentiment analysis layer connected in sequence.

[0069] The sentiment analysis model based on the asymmetric position-weighted strategy is as follows:

[0070] (1) Input the word vector representation sequence into the Bi-GRU layer to obtain the hidden layer states of the forward GRU and the reverse GRU respectively. After concatenation, the hidden layer state at the current time is obtained.

[0071] The current hidden layer state is as follows:

[0072]

[0073] in: These are the hidden layer states of the word vectors in a forward GRU algorithm. These are the hidden layer states of the word vector inverse GRU.

[0074] (2) In the Self-Attention layer, the similarity between the hidden layer states output by the Bi-GRU layer for each word vector in the sentence is calculated through the vector inner product. The Self-Attention weights corresponding to the hidden layer states of each word vector are obtained by Softmax normalization, and then the new hidden layer states at the current time are obtained.

[0075]

[0076] The Self-Attention weights are as follows:

[0077]

[0078] The new hidden layer state representation is as follows:

[0079]

[0080] in: Let T be the hidden layer state corresponding to the i-th and j-th word vectors in the sentence;

[0081] (3) For each sentence, an asymmetric context position weight transformation function is introduced. Based on the position of the words marked as aspect words in the sentence, the final word vector representation is obtained by using the asymmetric context position weight transformation function.

[0082] The asymmetric context position weight transformation function is as follows:

[0083]

[0084] The final word vector representation is as follows:

[0085]

[0086] in, This represents the normalized value of the word vector at position t in the sentence, where T is the sentence length, Γ is the gamma function, and α is the normalized value. k ,β k This is a hyperparameter, where k represents the index. This is the new hidden layer state of the word vector.

[0087] The calculation of index k is as follows:

[0088] α and β are predefined hyperparameter sets, containing multiple sets of hyperparameters. For each set of hyperparameters, the distance is calculated:

[0089]

[0090] Iterate through all hyperparameter combinations and compare all d. jFind the minimum distance d. min The corresponding index is K;

[0091]

[0092] The corresponding hyperparameter combination α at this time k ,β k Used to calculate the final word vector.

[0093] in: and These represent the normalized values ​​of the word vectors labeled as aspect words in the sentence, specifically the starting position l and the ending position r. T is the length of the sentence.

[0094] (4) A learnable aspect vector Um is constructed in the aspect information fusion layer. The aspect information containing the specified category is obtained through the Global-Attention mechanism. The similarity between each word vector and the aspect vector of the sentence is calculated using the inner product. The Global-Attention weight is obtained by normalization through softmax. Finally, the sentence vector that integrates the specified aspect information is obtained.

[0095] The aspect information fusion layer introduces a learnable aspect vector Um to represent the vector representation of different aspect words, in order to assist in the sentiment polarity classification task.

[0096] The similarity between the vector representation of a sentence and its corresponding aspect vector is as follows:

[0097] Similarity(u m v t )=u m ·v t

[0098] The global-attention weights are:

[0099]

[0100] The sentence vector representation that integrates information from specific aspects is as follows:

[0101]

[0102] Where: T is the length of the sentence, v t This is the final word vector for the sentence.

[0103] (5) In the sentiment analysis layer, the sentence vector representation is passed through a fully connected layer, and the sentiment polarity distribution of the sentence is obtained through the softmax function. The classification result with the highest probability is selected as the sentiment classification result.

[0104] The specific distribution of the sentiment polarity of the sentence is as follows:

[0105]

[0106] Where: y is the emotional polarity, W o and b o These are the parameters learned in the sentiment polarity classification layer. Let y be the probability distribution of emotional polarity.

[0107] The model is trained using the cross-entropy loss function, calculated as follows:

[0108]

[0109] in: The predicted probability for the sentiment polarity classification of the i-th sentence; training stops when the set number of training iterations is reached or the accuracy of the loss no longer improves.

[0110] S103. Input the word vector representation sequence into the sentiment analysis model based on the asymmetric position weighting strategy to obtain the sentiment distribution of the sentence.

[0111] This invention also provides an industrial text sentiment analysis system based on an asymmetric position-weighted strategy, comprising:

[0112] Data acquisition module: used to acquire text sample datasets;

[0113] The data preprocessing module is used to segment each sentence to obtain the word vector representation sequence of the sentence; and manually pre-annotate aspect words to form a training set;

[0114] Modeling and training module: used to construct a sentiment analysis model based on an asymmetric position weighting strategy, including a Bi-GRU layer, a Self-Attention layer, an adaptive weighting strategy, an aspect information fusion layer, and a sentiment analysis layer connected in sequence; the model is trained using the training set, the loss function is calculated, and an optimized sentiment analysis model based on an asymmetric position weighting strategy is obtained after the cutoff condition is met.

[0115] Analysis module: Used to collect actual text data, process the data using the optimized sentiment analysis model based on asymmetric position weighting strategy, and obtain sentiment analysis results.

[0116] It should be understood that any parts not described in detail in this application belong to the prior art. The above description of the preferred embodiments is quite detailed, but it should not be considered as a limitation on the scope of protection of this application. Those skilled in the art, under the guidance of this invention, can make substitutions or modifications without departing from the scope of protection of the claims of this application, and all such substitutions or modifications fall within the scope of protection of this application. The scope of protection claimed in this application shall be determined by the appended claims.

Claims

1. A sentiment analysis method for industrial text based on an asymmetric position-weighted strategy, characterized in that, Includes the following steps: (1) Obtain the text sample dataset, segment each sentence to obtain the word vector representation sequence of the sentence; and manually pre-annotate aspect words to form a training set; (2) Construct a sentiment analysis model based on an asymmetric position weighting strategy, including a Bi-GRU layer, a Self-Attention layer, an adaptive weighting strategy, an aspect information fusion layer, and a sentiment analysis layer connected in sequence; train the model using the training set, calculate the loss function, and obtain an optimized sentiment analysis model based on an asymmetric position weighting strategy after the cutoff condition is met. (2.1) Input the word vector representation sequence of the sentence into the Bi-GRU layer to obtain the hidden layer states of the forward GRU and the reverse GRU respectively. After concatenation, the hidden layer state at the current time is obtained. (2.2) In the Self-Attention layer, the similarity between the hidden layer states output by the Bi-GRU layer for each word vector in the sentence is calculated by the vector inner product, and the corresponding Self-Attention weights are obtained by Softmax normalization, thereby obtaining the new hidden layer state at the current time. The similarity calculation is as follows: ； The Self-Attention weights are specifically as follows: ； in: 、 For the first in the sentence i , j The hidden layer states corresponding to each word vector; The new hidden layer state is specifically represented as follows: in: t It is the position of the word vector in the sentence. T It is the length of the sentence; (2.3) For each sentence, an asymmetric context position weight transformation function is introduced. Based on the position of the words marked as aspect words in the sentence, the final word vector representation is obtained by using the asymmetric context position weight transformation function. The asymmetric context position weight transformation function is as follows: The final word vector is represented as follows: in, Indicates the position of the word vector in the sentence. The normalized value, T It is the length of the sentence. It is the gamma function. For hyperparameters, Indicates an index. This is the new hidden layer state of the word vector; The It is calculated in the following way: For each set of hyperparameters, calculate the distance: Compare all Find the minimum distance based on the value. The corresponding index is K ; in: j For index value, and These represent the starting positions of the word vectors labeled as aspect words in the sentence. and end position normalized value ; (2.4) A learnable aspect vector U is constructed in the aspect information fusion layer. m The Global-Attention mechanism is used to obtain aspect information containing a specified category; the similarity between each final word vector of the sentence and the aspect vector is calculated using the inner product; the Global-Attention weights are obtained by softmax normalization; and finally, a sentence vector that integrates the specified aspect information is obtained. The similarity between each final word vector of the sentence and the aspect vector: The Global-Attention weights are: The sentence vector representation of the fused aspect information is as follows: in: T The length of the sentence. This is the final word vector for the sentence; (2.5) In the sentiment analysis layer, after the sentence vector representation is passed through a fully connected layer, the sentiment polarity distribution of the sentence is obtained through the softmax function, and the classification result with the highest probability is selected as the sentiment classification result; (3) Collect actual text data, process the data using the optimized sentiment analysis model based on asymmetric position weighting strategy, and obtain sentiment analysis results.

2. The industrial text sentiment analysis method based on an asymmetric position-weighted strategy according to claim 1, characterized in that, In step (1), the sentence in the input text is segmented to obtain the word vector representation sequence: the sentence is put into the global word vector GloVe pre-trained model to obtain the word vector representation sequence.

3. The industrial text sentiment analysis method based on an asymmetric position-weighted strategy according to claim 1, characterized in that, The hidden layer state at the current moment in step (2.1) is specifically as follows: in: These are the hidden layer states of the positive GRU for the word vectors. These are the hidden layer states of the inverse GRU of the word vectors. t It is the position of the word vector in the sentence.

4. The industrial text sentiment analysis method based on an asymmetric position-weighted strategy according to claim 1, characterized in that, The specific distribution of the sentiment polarity of the sentences in step (2.5) is as follows: in: It is emotional polarity. and These are the parameters learned in the sentiment polarity classification layer. For emotional polarity The probability distribution.

5. The industrial text sentiment analysis method based on an asymmetric position-weighted strategy according to claim 4, characterized in that, The model is trained using the cross-entropy loss function, as follows: in: For the first i The predicted probability of sentiment polarity classification for each sentence; training stops when the set number of training iterations is reached or the accuracy of the loss no longer improves.

6. An industrial text sentiment analysis system based on an asymmetric position-weighted strategy, characterized in that, include: Data acquisition module: used to acquire text sample datasets; Data preprocessing module: used to segment each sentence to obtain the word vector representation sequence of the sentence; The training set is formed by manually labeling the terms in advance. Modeling and training module: used to construct a sentiment analysis model based on an asymmetric position weighting strategy, including a Bi-GRU layer, a Self-Attention layer, an adaptive weighting strategy, an aspect information fusion layer, and a sentiment analysis layer connected in sequence; the model is trained using the training set, the loss function is calculated, and an optimized sentiment analysis model based on an asymmetric position weighting strategy is obtained after the cutoff condition is met. (1) Input the word vector representation sequence of the sentence into the Bi-GRU layer to obtain the hidden layer states of the forward GRU and the reverse GRU respectively, and then concatenate them to obtain the hidden layer state at the current time. (2) In the Self-Attention layer, the similarity between the hidden layer states output by the Bi-GRU layer for each word vector in the sentence is calculated by the inner product of the vectors, and the corresponding Self-Attention weights are obtained by Softmax normalization, thereby obtaining the new hidden layer state at the current time. The similarity calculation is as follows: ； The Self-Attention weights are specifically as follows: ； in: 、 For the first in the sentence i , j The hidden layer states corresponding to each word vector; The new hidden layer state is specifically represented as follows: in: t It is the position of the word vector in the sentence. T It is the length of the sentence; (3) For each sentence, an asymmetric context position weight transformation function is introduced, and the final word vector representation is obtained by using the asymmetric context position weight transformation function based on the position of the words marked as aspect words in the sentence. The asymmetric context position weight transformation function is as follows: The final word vector is represented as follows: in, Indicates the position of the word vector in the sentence. The normalized value, T It is the length of the sentence. It is the gamma function. For hyperparameters, Indicates an index. This is the new hidden layer state of the word vector; The It is calculated in the following way: For each set of hyperparameters, calculate the distance: Compare all Find the minimum distance based on the value. The corresponding index is K ; in: j For index value, and These represent the starting positions of the word vectors labeled as aspect words in the sentence. and end position normalized value ; (4) A learnable aspect vector U is constructed in the aspect information fusion layer. m The Global-Attention mechanism is used to obtain aspect information containing a specified category; the similarity between each final word vector of the sentence and the aspect vector is calculated using the inner product; the Global-Attention weights are obtained by softmax normalization; and finally, a sentence vector that integrates the specified aspect information is obtained. The similarity between each final word vector of the sentence and the aspect vector: The Global-Attention weights are: The sentence vector representation of the fused aspect information is as follows: in: T The length of the sentence. This is the final word vector for the sentence; (5) In the sentiment analysis layer, after the sentence vector representation is passed through a fully connected layer, the sentiment polarity distribution of the sentence is obtained through the softmax function, and the classification result with the highest probability is selected as the sentiment classification result; Analysis module: Used to collect actual text data, process the data using the optimized sentiment analysis model based on asymmetric position weighting strategy, and obtain sentiment analysis results.

Images