A context-enhanced new energy vehicle complaint text aspect-opinion extraction method

Abstract

The application discloses a new energy vehicle complaint text aspect-opinion pair extraction method based on context enhancement, and belongs to the field of natural language processing, and comprises the following steps: S1, acquiring new energy vehicle complaint data, and performing part-of-speech tagging and text screening processing on the new energy vehicle complaint data; S2, an entity extraction method based on a conditional random field is used to acquire a text feature vector, and CRF is used for sequence labeling to obtain an entity set; S3, text features and theme features are fused to acquire an enhanced text context representation; and S4, based on a three-affine mechanism, the context representation is used as auxiliary information to enhance the ability of a model to perceive the relationship between aspect words and opinion words, so that suitable aspect-opinion pairs are screened out. According to the application, the theme features and the text features are fused to acquire an enhanced context representation, and the enhanced context representation is used as auxiliary information to improve the ability of a model to perceive the relationship between aspect words and opinion words, so that reliable data reference is provided for new energy vehicle merchants to improve products.

Description

Technical Field

[0001] This invention relates to the field of natural language processing, specifically to a method for extracting aspect-opinion pairs from texts related to new energy vehicle complaints based on context enhancement. Background Technology

[0002] With the rapid development of new energy products, more and more people are choosing new energy vehicles as their mode of transportation. While these new products bring convenience, they also raise many new issues, leading to numerous customer complaints. Customer complaints often expose shortcomings in the product design process. Therefore, analyzing customer opinions on specific aspects of the product within complaint texts can provide reliable data for adjustments and decisions in business operations, thereby helping companies design products that meet the needs of the public.

[0003] Currently, research on aspect-opinion pair extraction from new energy vehicle complaint texts is relatively lacking. Existing research mainly relies on word distance information or syntactic dependencies for relation detection, i.e., target word information. However, due to the characteristics of new energy vehicle complaint texts, such as high entity density and lengthy sentences, the aforementioned ideas and methods only consider word-level information, ignoring the influence of the text's context on word pair matching. This can easily lead to misjudgments in matching aspect words and opinion words. Since the topic information of complaint texts is a high-level summary of the text content and has a low probability of being interfered with by redundant information, this invention considers fusing topic features and text features to obtain an enhanced text context representation. This context representation is then used as auxiliary information to enhance the model's ability to perceive the relationship between aspect words and opinion words, thereby improving the extraction accuracy. Summary of the Invention

[0004] The purpose of this invention is to provide a context-enhanced method for extracting aspect-opinion pairs from new energy vehicle complaint texts. This method integrates topic features and text features as contextual representations, using these representations as auxiliary information to enhance the model's ability to identify relationships between entities. In the entity extraction stage, the entity extraction model captures contextual information to obtain textual features, which are then input into a CRF (Contextual Representation Rendering) to obtain an entity set. In the relationship detection stage, the enhanced contextual representation is obtained by integrating topic features and text features, and a triaffine mechanism is used, with the contextual representation as an aid, to filter out suitable aspect-opinion pairs.

[0005] To achieve its objectives, the present invention employs the following technical solution:

[0006] A context-enhanced method for extracting aspect-opinion pairs from texts related to new energy vehicle complaints, comprising the following steps:

[0007] (1) Acquisition of complaint data and text preprocessing for new energy vehicles;

[0008] We acquire data on complaints about new energy vehicles and perform part-of-speech tagging and text filtering on it.

[0009] (2) An entity extraction method based on conditional random fields is used to obtain the set of aspect words and the set of opinion words in the complaint text;

[0010] Text word vectors are used as input to a bidirectional long short-term memory network. The weights of the feature vectors are updated through a multi-head attention mechanism, and sequence labeling is performed using a CRF to obtain the entity set.

[0011] (3) Integrate text features and thematic features to obtain enhanced text context representation;

[0012] The topic of the complaint text is used as input to the BERT pre-trained model to obtain topic feature representations, and the topic features are fused with text features to obtain an enhanced text context representation.

[0013] (4) Based on the three-affine mechanism, contextual representation is used as auxiliary information to enhance the model's ability to perceive the relationship between aspect words and opinion words;

[0014] Using contextual representation, aspect word representation, and opinion word representation as input to a three-affine mechanism, word pair matching scores are calculated, and suitable aspect-opinion pairs are selected.

[0015] The specific operations for obtaining new energy vehicle complaint data and preprocessing text filtering in step (1) are as follows:

[0016] (1.1) Obtain complaint data from the CheZhi.com platform through web crawling technology.

[0017] (1.2) Remove stop words. Remove meaningless phrases that only have structural functions but do not provide useful information. For example, "I purchased...", "I spent a certain amount of money to buy...", "Let it continue to be used...", etc.

[0018] (1.3) Data filtering. Complaint texts with fewer than 30 words and more than 200 words in the crawled data are removed to ensure the balance of the dataset.

[0019] (1.4) Use the NITK part-of-speech tagging tool to tag the new energy vehicle complaint texts with part-of-speech tags, and then use the BIO tagging method to tag the data, thereby constructing a new energy vehicle complaint text dataset containing 4680 data points.

[0020] In step (2), the specific steps for obtaining the set of aspect terms and the set of opinion terms in the complaint text based on entity relation extraction are as follows:

[0021] (2.1) Use the BERT pre-trained model to obtain the word vector of each word in the text, and input the obtained vector representation into the bidirectional long short-term memory network to obtain the feature representation of the text.

[0022] (2.2) By using a multi-head attention mechanism, by considering multiple attention scores, the weight of aspect words and opinion words is increased to reduce the interference of redundant information in the text of new energy vehicle complaints, while obtaining richer text feature representations.

[0023] The calculation process of the multi-head attention mechanism is as follows:

[0024]

[0025] Where Q is the query vector, K is the key vector, V is the value vector matrix, and d k The word vector dimension Attention(Q, K, V) represents the individual self-attention score, and the head... j Let W represent the calculation result of the j-th attention head, W be the weight matrix, and MHA(Q,K,V) represent the final attention score.

[0026] The feature vector representation of the obtained complaint text is input into the CRF for sequence labeling, which labels the aspect words and opinion words, and obtains the aspect word set and opinion word set.

[0027] By calculating the joint probability distribution of the entire sequence, a globally optimal solution is sought, thus effectively addressing the issue of nested and complex terminology in the field of new energy vehicles. Let the currently observed sequence be x={x1,x2,x3,..,x n The label sequence is y = {y1, y2, y3, ..., y}. n The calculation process is as follows:

[0028]

[0029] Here, W1 and W2 are both trainable weight vectors, and F(y,x) represents the combination of transition feature function and state feature function.

[0030] In step (3), the specific steps for obtaining the enhanced text context representation based on the fusion of text features and topic features are as follows:

[0031] (3.1) A BERT pre-trained model is used as the encoder to obtain the sentence representation of the topic. By adding special markers [CLS] and [SEP] to the beginning and end of the topic sentence respectively as input, the hidden state representation of each position is obtained after passing through the encoding layer. The acquisition process is as follows:

[0032]

[0033] Here, the CLS vector is the hidden state representation at the first position in the input sequence, representing the sentence features of the complaint topic, denoted as h. CLS .

[0034] (3.2) To improve the correlation between the entity recognition and relationship detection modules, pooling is used to select the parts that are important for the extraction task. The new feature representation is then processed using max pooling. The calculation process is as follows:

[0035] (3.2.1) Represent the text features h t Average pooling is performed to obtain a vector with the same dimensions as the topic feature representation. This new text feature vector is then concatenated with the topic feature vector to obtain a new feature representation. The aim is to balance the representation of each feature vector, thus better expressing the semantic information of the entity. The calculation process is as follows:

[0036]

[0037] Among them, V A V represents the average information of the text features, and V represents the contextual representation after concatenating the topic features and text features.

[0038] (3.2.2) Max pooling is performed on the concatenated new feature vector to obtain the final contextual feature representation. The purpose of max pooling is to reduce the model's sensitivity to noise and enhance its ability to represent contextual information. In addition, max pooling reduces the number of training parameters and improves the training speed for the entire model.

[0039] h t =max pooling(V)

[0040] Among them, h t This indicates the enhanced contextual features.

[0041] In step (4), the ability of the model to perceive the relationship between aspect words and opinion words is enhanced by using contextual representation as auxiliary information based on the three-affine mechanism. The process is as follows:

[0042] A three-affine mechanism is used to project the query vector, key vector, and value vector into different low-dimensional spaces, and the relationship between the three is calculated through an additional linear transformation.

[0043] By pairing aspect terms and viewpoint terms one by one, we obtain a set of aspect-viewpoint pairs, and for each target word pair (S a ,S o Assignment Relationship Labels This transforms the task into a binary classification task.

[0044] Then, the aspect vector representation a is obtained through positional encoding. t ={a i ,a i+1 ,...,a i+q} and viewpoint vector representation o t ={o i ,o i+1 ,...,o i+p Finally, the aspect representation, opinion representation, and contextual feature representation are used as input to TriAffine. The specific process is as follows:

[0045]

[0046] Where W represents the learnable parameters, and h represents the aspect representation and viewpoint representation from the entity recognition module, respectively. c Φ represents the contextual feature vector. a,o,t r represents the probability of a word pair matching. a,o This indicates the result of binary classification.

[0047] The aspect-opinion pair extraction method for new energy vehicle complaint texts based on context enhancement provided by this invention has the following advantages:

[0048] (1) This invention obtains text vectors through a long short-term memory network, which can obtain more accurate and comprehensive text feature representations. Then, through a multi-head attention mechanism, it reduces noise interference caused by redundant information in complaint texts, thereby providing technical assistance for subsequent relationship detection.

[0049] (2) This invention utilizes a method that integrates topic features and text features to make the contextual semantics in the obtained complaint text more accurate; at the same time, it uses a three-affine mechanism to use contextual representation as an auxiliary to calculate the relationship between entity words, and obtains the target aspect-opinion pair through the Sigmoid function. Attached Figure Description

[0050] Figure 1 A flowchart of an opinion pair extraction method for context-enhanced new energy vehicle complaint texts;

[0051] Figure 2 This diagram illustrates the process of obtaining an entity word set using a conditional random field-based entity extraction method.

[0052] Figure 3 A schematic diagram illustrating the process of fusing textual and topic features to obtain enhanced contextual representations;

[0053] Figure 4 This is a schematic diagram of the relationship detection method based on the three affine mechanism. Detailed Implementation

[0054] The present invention will be further explained and illustrated below through specific embodiments.

[0055] Example 1: This invention provides a method for extracting aspect-opinion pairs from texts related to new energy vehicle complaints based on context enhancement, such as... Figure 1 As shown. The specific steps are as follows:

[0056] S1. Acquisition and preprocessing of complaint texts related to new energy vehicles;

[0057] S1.1. First, obtain the complaint texts for new energy vehicles. Using web crawling technology, 5,000 complaint data related to new energy vehicles with different clauses were crawled from the Car Complaints Network. At the same time, the topic sentence corresponding to each complaint text was obtained and stored in the form of an Excel file.

[0058] S1.2. Remove stop words. Remove meaningless phrases that only have structural functions but do not provide useful information. For example, "I purchased...", "I spent a certain amount of money to buy...", "Please continue to use...", etc.

[0059] S1.3. Data Filtering. Complaint texts with fewer than 30 characters or more than 200 characters in the crawled data will be removed to ensure the balance of the dataset.

[0060] S1.4. Use the NITK part-of-speech tagging tool to perform part-of-speech tagging on the text of complaints about new energy vehicles, and obtain its vectorized representation V=[v1,v2,...,v n Then, the BIO annotation method was used to annotate the data, thereby constructing a new energy vehicle complaint text dataset containing 4,680 data points.

[0061] S2. An entity extraction method based on conditional random fields is used to obtain the set of aspect terms and opinion terms in the complaint text. Combined with... Figure 2 The following explanation is provided:

[0062] S2.1 Obtain the forward hidden state of the text using a Bidirectional Long Short-Term Memory (BiLSTM) network. With backward hidden state The sequences are then spliced ​​together to obtain a complete sequence, thereby capturing long-term dependencies within the sequence.

[0063] S2.1.1 Let the complaint text data P = {C1, C2, ..., C...} n Let C be a collection of sentences, n ≥ 2, where each sentence contains m words. p This represents the p-th sentence, w p,j This represents the j-th word in the p-th sentence.

[0064] S2.1.2 uses a BERT pre-trained model as an encoder to encode sentences and obtain text word vectors with semantic representations, as shown below:

[0065] W S ={w1,w2,w3,...,w n}, w k This represents the vector representation of the k-th word.

[0066] S2.1.3. By capturing bidirectional information through a bidirectional long short-term memory network, contextual information containing candidate aspect items can be passed to the corresponding view item, and contextual information containing candidate view items can be passed to the corresponding aspect item to improve the interactivity between target entities.

[0067] S2.2 utilizes a multi-head attention mechanism, considering multiple attention scores to increase the weight of aspect words and opinion words, thereby reducing the interference of redundant information in new energy vehicle complaint texts while obtaining richer text feature representations.

[0068] S2.3 seeks the global optimal solution by calculating the joint probability distribution of the entire sequence, thus effectively addressing the issue of nested and complex terminology in the field of new energy vehicles. Let the currently observed sequence be x={x1,x2,x3,..,x n The label sequence is y = {y1, y2, y3, ..., y}. n The calculation process is as follows:

[0069]

[0070] Here, W1 and W2 are both trainable weight vectors, and F(y,x) represents the combination of transition feature function and state feature function.

[0071] S3. Integrate textual features and topic features to obtain an enhanced textual context representation. Combined with... Figure 3 Explanation:

[0072] S3.1 uses a BERT pre-trained model as the encoder to obtain the sentence representation of the topic. By adding special markers [CLS] and [SEP] to the beginning and end of the topic sentence respectively as input, the hidden state representation T = [h] at each position is obtained after passing through the encoding layer. t 1,h t 2,h t 3...,h t n The calculation process is as follows:

[0073]

[0074] Here, the CLS vector is the hidden state representation at the first position in the input sequence, representing the sentence features of the complaint topic, denoted as h. CLS

[0075] S3.2. After obtaining the topic features, the text features are fused with the topic features to obtain an enhanced contextual representation.

[0076] S3.2.1. To improve the correlation between the entity recognition and relationship detection modules, this invention uses pooling operations to select the parts that are crucial for the extraction task. The new feature representation is then processed using max pooling. The calculation process is as follows:

[0077]

[0078] Among them, V A V represents the average information of the text features, and V represents the contextual representation after concatenating the topic features and text features.

[0079] S3.2.2 The newly concatenated feature vector is then subjected to max pooling to obtain the final contextual feature vector. The purpose of max pooling is to reduce the model's sensitivity to noise and enhance its ability to represent contextual information. In addition, max pooling reduces the number of training parameters and improves the training speed for the entire model.

[0080] h t =max pooling(V)

[0081] Among them, h t This represents the enhanced contextual features.

[0082] S4. Based on a three-affine mechanism, contextual representation is used as auxiliary information to enhance the model's ability to perceive the relationship between aspect words and opinion words. Combined with... Figure 4 The following explanation is provided:

[0083] S4.1 employs a three-affine mechanism to project the query vector, key vector, and value vector onto different low-dimensional spaces, and calculates the relationships between them through an additional linear transformation. Then, it obtains the aspect vector representation 'a' through positional encoding. t ={a i ,a i+1 ,...,a i+q} and viewpoint vector representation o t ={o i ,o i+1 ,...,o i+p}

[0084] S4.2 uses aspect representation, opinion representation, and contextual feature representation as TriAffine input. The specific process is as follows:

[0085]

[0086] Where W represents the learnable parameters, and h represents the aspect representation and viewpoint representation from the entity recognition module, respectively. c Φ represents the contextual feature vector. a,o,t r represents the probability of a word pair matching. a,o This indicates the result of binary classification.

[0087] Furthermore, the above embodiments are only used to illustrate specific implementations of the present invention and are not intended to limit it. Those skilled in the art should understand that equivalent substitutions can be made to some of the technologies, and these modifications and substitutions also fall within the protection scope of the present invention.

Claims

1. A method for extracting aspect-opinion pairs from texts related to new energy vehicle complaints based on context enhancement, characterized in that... Includes the following steps: Step 1: Acquisition and text preprocessing of new energy vehicle complaint data; Acquire new energy vehicle complaint data and perform part-of-speech tagging and text filtering; Step 2: Based on the entity relation extraction method of Conditional Random Field (CRF), obtain the set of aspect words and opinion words in the complaint text; use the semantically represented text word vectors obtained by encoding through the BERT pre-trained model as input to the bidirectional Long Short-Term Memory (LSTM) network, update the weights of the feature vectors through a multi-head attention mechanism, and use CRF for sequence labeling to obtain the entity set; wherein, the process of obtaining the text word vectors is as follows: use the BERT pre-trained model as an encoder to encode the sentence to obtain the vector representation W of each word in the text. S ={w1,w2,w3,...,w n }, where w k The vector representation of the k-th word; Step 3: Fuse text features and topic features to obtain an enhanced text context representation; use the topic of the complaint text as input to the BERT pre-trained model to obtain topic feature representation, and fuse topic features and text features to obtain an enhanced text context representation; Step 4: Calculate the probability of matching between aspect words and opinion words based on a three-affine mechanism using contextual representation as auxiliary information; use the contextual representation, aspect word representation, and opinion word representation as input to the three-affine mechanism to calculate the word pair matching score and filter out correct aspect-opinion pairs; wherein, the calculation of the probability of matching between aspect words and opinion words based on a three-affine mechanism using contextual representation as auxiliary information includes: Step 4.1 employs a three-affine mechanism to project the query vector, key vector, and value vector onto different low-dimensional spaces, and calculates the association relationships among them through an additional linear transformation; then, it obtains the aspect vector representation at = a through positional encoding. i a i+1 , ..., a i+q and viewpoint vector representation Step 4.2 uses aspect representation, viewpoint representation, and contextual feature representation as input to the three-affine mechanism. The specific process is as follows: r a，o =Sigmoid(Φ a，o，t ) Where W represents the learnable parameters, and and h represents the aspect representation and viewpoint representation from the entity recognition module, respectively. c Φ represents the contextual feature vector. a，o，t r represents the probability score of a word pair match. a，o This indicates the result of binary classification.

2. The method for extracting aspect-opinion pairs from new energy vehicle complaint texts based on context enhancement as described in claim 1, characterized in that, Step 1 includes: Step 1.1 Data filtering: Remove meaningless phrases that only have structural functions but do not provide useful information. The purpose is to reduce the interference of redundant information on the relationship between words and opinions in the model's perception. Remove complaint texts with fewer than 30 words and more than 200 words from the crawled data to ensure the balance of the dataset. Step 1.2 Part-of-speech tagging; The NITK part-of-speech tagging tool is used to tag the part-of-speech of the new energy vehicle complaint text, and then the BIO tagging method is used to annotate the data. The purpose is to enable the model to learn more distinct part-of-speech features.

3. The method for extracting aspect-opinion pairs from new energy vehicle complaint texts based on context enhancement as described in claim 1, characterized in that, The entity relation extraction method based on conditional random fields described in step 2, and the method for obtaining the set of aspect terms and opinion terms in the complaint text, include: The text forward hidden state is obtained using a Bidirectional Long Short-Term Memory (BiLSTM) network. With backward hidden state The sequences are then concatenated to obtain a complete sequence, thereby capturing long-term dependencies within the sequence. Step 2.1 Let the complaint text data P = C1, C2, ..., C n n≥2, each sentence contains m words, C p This represents the p-th sentence, w p，j This represents the j-th word in the p-th sentence; Step 2.2 uses a BERT pre-trained model as an encoder to encode the sentence and obtain text word vectors with semantic representation, as shown below: w S = w1, w2, w3, ..., w n w k The vector representation of the k-th word; Step 2.3 By capturing bidirectional information through a bidirectional long short-term memory network, contextual information containing candidate aspect items can be passed to the corresponding view item, and contextual information containing candidate view items can be passed to the corresponding aspect item to improve the interactivity between target entities. Step 2.4 employs a multi-head attention mechanism, considering multiple attention scores to increase the weight of aspect words and opinion words. This reduces redundant information interference in new energy vehicle complaint texts while obtaining richer text feature representations. The attention mechanism process is represented as follows: head j =Attention(Q j ,K j ,V j ) Where Q is the query vector, K is the key vector, V is the value vector matrix, and d k Let Attention((Q, K, V)) represent the word vector dimension, Attention((Q, K, V)) represent the individual self-attention score, head represent the calculation result of the j-th attention head, W is the weight matrix, and MHA(Q, K, V) represent the final attention score. Step 2.5 inputs the feature vector representation of the obtained complaint text into the CRF for sequence labeling, labeling aspect terms and opinion terms, and obtaining the aspect term set and opinion term set; let the current observation sequence be x = x1, x2, x3, ..., x n The label sequence is y = y1, y2, y3, ..., y n The calculation process is as follows: Here, W1 and W2 are both trainable weight vectors, and F(y, x) represents the combination of transition feature function and state feature function.

4. The method for extracting aspect-opinion pairs from new energy vehicle complaint texts based on context enhancement as described in claim 1, characterized in that, Step 3 includes: Step 3.1 uses a BERT pre-trained model as the encoder to obtain the sentence representation of the topic. By adding special markers [CLS] and [SEP] to the beginning and end of the topic sentence respectively as input, the hidden state representation of each position is obtained after passing through the encoding layer. The acquisition process is as follows: Here, the CLS vector is the hidden state representation at the first position in the input sequence, representing the sentence features of the complaint topic, denoted as h. CLS ; Step 3.2 To improve the correlation between the entity recognition and relationship detection modules, pooling is used to select the parts that are important for the extraction task. Then, the new feature representation is processed by max pooling, as follows: Step 3.2.1 Represent the text features h t Average pooling is performed to obtain a vector with the same dimensions as the topic feature representation. The new text feature vector is then concatenated with the topic feature vector to obtain a new feature representation. The purpose is to give balanced consideration to each feature vector, so as to better express the semantic information of the entity. The calculation process is as follows: V=Concat(X A ,h dis ) Among them, V A V represents the average information of the text features, and V represents the contextual representation after concatenating the topic features and text features; Step 3.2.2 involves performing max pooling on the concatenated new feature vector to obtain the final contextual feature representation. The purpose of max pooling is to reduce the model's sensitivity to noise and enhance its ability to represent contextual information. Furthermore, max pooling reduces the number of training parameters and improves training speed for the entire model. Specifically, h... t This represents the enhanced contextual features. h t =maxpooling(5)。

Images