A method and system for extracting aspect-level sentiment triples based on semantically enhanced dual encoders

Abstract

The present invention provides an aspect-level sentiment triple extraction method and system based on a semantic enhancement dual encoder, which relates to the technical field of aspect-level sentiment analysis, especially aspect-level sentiment triple extraction. The method for aspect-level sentiment triple extraction based on a semantic enhancement dual encoder includes: learning text surface semantic information based on a basic context encoder; learning specific semantic information of the text specific to the comment field based on a specific context encoder; receiving part-of-speech (POS) information and syntactic dependency information of the text, and modeling the syntactic and structural information of the text through a graph convolutional network; based on a novel encoding interaction strategy, fully integrating basic semantics and specific state semantics, so that the learned text semantics can extract potential sentiment triples in the text from different angles. This patent solves the problems of the lack of multi-angle encoders and the neglect of text syntactic and structural information in the current aspect-level sentiment triple extraction task.

Description

Technical Field

[0001] The present invention relates to the technical field of aspect-level sentiment analysis, and in particular to a method and system for extracting aspect-level sentiment triples based on a semantically enhanced dual encoder. Background Art

[0002] Aspect-level sentiment triple extraction (ASTE) is the most comprehensive subtask within aspect-level sentiment analysis (ABSA). It combines the three subtasks of extraction, matching, and classification to simultaneously capture aspect entities, opinion entities, and the corresponding sentiments and attitudes within a review. Previous work has performed aspect-level sentiment analysis (ABSA) by integrating several independent subtasks, primarily extraction, matching, and classification. Specifically, Aspect Terms Extraction (ATE) and Opinion Terms Extraction (OTE) extract only aspect or opinion entities from a sentence. Aspect-oriented Opinion Extraction (AOE) attempts to extract the corresponding opinion entity from a given aspect entity. Aspect-level Sentiment Classification (ALSC) determines the corresponding sentiment and attitude from aspect terms within a sentence. While these subtasks can address sentiment analysis, their performance degrades when applied in combination. Prior to the definition of the ASTE task, a sentiment triple consisted of an aspect term, an opinion term, and the corresponding sentiment. ASTE aims to extract triples from sentences in the form [Aspect, Opinion, Sentiment]. Existing research methods improve the ASTE task by constructing new table-filling or span-fusion strategies. However, these methods overlook several issues beyond the strategy itself. While simply changing the existing extraction strategy can improve model performance, ignoring the original text semantics significantly limits further model improvement. For one thing, they fail to consider the interactions between aspect and opinion words across multiple semantic domains using different encoders. Furthermore, they overlook the hidden grammatical features of each component within a triple. Summary of the Invention

[0003] (1) Technical problems solved

[0004] In response to the shortcomings of the existing technology, the present invention provides a semantically enhanced dual encoder method and system for aspect-level sentiment triple extraction, which encodes sentences from different perspectives and can comprehensively consider the interactions between the various components of the sentence to obtain a more refined text semantic representation, solving the problem that the current aspect-level sentiment triple extraction model cannot fully extract multi-angle semantic information.

[0005] (2) Technical solution

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0007] In the first aspect, a method for extracting aspect-level sentiment triples based on a semantically enhanced dual encoder is provided, comprising:

[0008] Learn text surface semantic information based on the basic context encoder;

[0009] Learning specific semantic information of the text that is specific to the review domain based on a specific context encoder;

[0010] Receive the part-of-speech (POS) information and syntactic dependency information of the text, and model the syntactic and structural information of the text through a graph convolutional network;

[0011] Design a novel encoding interaction strategy that fully integrates basic semantics and specific semantics, so that the learned text semantics can reflect the true expression of the sentence from different angles.

[0012] Preferably, a basic context encoder is used to obtain the surface semantics of the text, specifically including:

[0013] The BERT encoder is used to generate the hidden state vector of the text, which is used as the basic contextual semantics of the sentence. Specifically, a sentence representation "[CLS]sentence[SEP]" is constructed and fed into the encoder as input to obtain the basic contextual semantics h of the word, where [CLS] and [SEP] are unique tags added by BERT at the beginning and end of the sentence. In addition, since BERT splits a complete word based on subwords, the obtained contextual feature h does not conform to the sentence length in terms of dimension. To remedy this problem, this patent designs a new word segmenter Whitetokenizer for BERT, which records each word that is split into subwords. At the same time, the subwords are aligned by averaging the set to obtain the contextual semantics h b The process can be briefly described as follows:

[0014] h b =BERT(Whitetokenizer([[CLS],ω1,…,w n ,[SEP]]))

[0015]

[0016] It is worth noting that in h b The hidden states of the special marks [CLS] and [SEP] are removed, and the aligned where d b Represents the hidden state dimension of BERT.

[0017] Preferably, the specific context encoder is implemented using a module having a Bi-LSTM and a graph convolutional network, specifically including the following:

[0018] In order to accurately capture the semantic tendencies of words in different fields, we first use two different semantic lookup tables to initialize text embedding. Indicates the meaning of words in a broad context, and domain-specific embedding tables Represents the meaning of a word in a specific review context. Where |v| represents the size of the lookup table dictionary.

[0019] Different from the semantic information mentioned above, the word part-of-speech information establishes the distinction between words from a linguistic perspective. Words with different parts of speech often have specific collocations and distinctions. Use SpaCy to obtain the part-of-speech tag for each word. However, the total number of part-of-speech tags is large and the categories overlap. In order to distinguish and model the major categories of words, the part-of-speech tags are summarized as follows: where p noun , p verb , p adj , p adv , p others Represent nouns, verbs, adjectives, adverbs and other categories of words respectively. Based on this, a learnable POS embedding is designed Encode the part-of-speech information of a word.

[0020] Therefore, the final representation E of the word after 3-domain embedding is:

[0021]

[0022] As mentioned before, the components of a triple are highly contextual. Therefore, Bi-LSTM is used as a special encoder to learn the contextual features of the sentence. After several gating units and attention mechanisms, the special encoder takes the embedding sequence of the sentence as input to obtain the contextual features. where d l represents the dimension of the hidden state. This process can be simplified as follows:

[0023] X = Bi-LSTM(E)

[0024] Feature X reflects the contextual features of the text. In addition, the grammatical information of the text is also worthy of attention. In order to learn the grammatical structure information of the text, a multi-layer graph convolutional network (GCN) based on the syntactic dependency tree is designed. This can learn the grammatical information of each comment and directly enrich the representation of the current word through the state of directly related words. In the specific implementation, the result X of the Bi-LSTM is used as the initial state, and a multi-layer graph convolution operation is performed on the corresponding dependency tree to obtain the grammatically enhanced context state hp , the process can be summarized as follows:

[0025]

[0026] in, Represents the input features of each layer of the GCN network. W is the trainable parameter matrix of each layer, It is a 0-1 adjacency matrix that records the connections between words. yes The degree matrix records the number of neighbors of each node. Sum degree matrix Generated by the following formula:

[0027]

[0028]

[0029] So far, the basic context feature h learned based on BERT encoder has been obtained b and the specific context features h learned by the Bi-LSTM encoder and GCN network based on 3-domain embedding p Next, both are fed into the designed encoding interaction layer to fuse the contextual features obtained by the two different encoders.

[0030] Preferably, the encoding interaction strategy is implemented by an iterative encoding interaction layer based on the self-attention mechanism, specifically including the following:

[0031] Basic semantics b , which reflects the hidden state of each word based on the contextual semantics. Specific semantics h p , which reflects the semantics of the sentence in the context of the review domain, while also reflecting the differences in the part of speech of each word and the grammatical differences of the sentence. These two different semantics reflect the characteristics of the sentence from different perspectives. To integrate these two different aspects of information, this patent designs a clever encoding interaction layer to enhance the semantic features.

[0032] First, the basic semantics and specific semantics are fed into an interactive attention module to obtain the attention score α of each word under the two semantics. b and α p In the specific implementation, the interactive attention module is calculated by the following attention formula.

[0033]

[0034]

[0035] In theory, the attention score It records the semantic relevance between any words in a sentence, which can reflect the degree of semantic closeness between two words. It also carries a lot of information, which reflects the relevance of words in terms of part of speech and grammar. In order to interact the two types of semantics and enhance each other's expression, let the attention score α b (α p ) acts on the opposing semantic feature h p (h b ) to fuse the two semantics while reinforcing the original semantic features through self-looping. This implementation can be summarized as the following formula.

[0036] h b ′=Dropout(α p h b )+h b

[0037] h p ′=Dropout(α b h b )+h p

[0038] The purpose of setting a Dropout layer is to randomly mask some channels to resist gradient propagation errors and improve the robustness of the model.

[0039] The introduction of attention mechanism enables the interaction of two different semantic aspects. However, a single-layer interaction cannot fully integrate the semantics. A multi-layer interaction mechanism is further introduced to enhance the features. In each layer of interaction, h′ p Re-enter a BiLSTM encoding layer with multi-layer GCN. h′ b Directly serve as the input of the next layer of iteration. This process can be expressed as follows.

[0040]

[0041]

[0042]

[0043] in, represents the output of the lth layer interaction, g k Represents a k-layer GCN network. represents the input of the t-1th layer in the GCN network, W (t-1) It is the trainable parameter of the t-1th layer of the GCN network. Represents the output of the lth layer interaction

[0044] In the above interaction, It can repeatedly extract effective information from the BERT base encoder and fuse it with itself through the interactive attention mechanism. We can effectively focus on the specific context encoding h p The differences in part of speech and grammar are discussed, and the self-characteristics are strengthened through interaction. Through iterative interaction of the above formula, the result of the L-th layer interaction is selected. As the output of the encoding interaction layer, it is input into a BDTF-based triplet extraction layer.

[0045] In the second aspect, a semantically enhanced dual encoder-based system for aspect-level sentiment triple extraction is provided, comprising:

[0046] The basic encoder module is used to obtain the surface semantics of the text;

[0047] A specific encoder module is used to capture the deep semantics of the text and model deep grammatical information based on the part-of-speech information of words and the syntactic dependencies of sentences;

[0048] The encoding interaction module is used to iteratively fuse the acquired basic semantics and the semantic information specific to the review expression;

[0049] The BDTF classifier module is used to classify the obtained fusion semantics and extract possible sentiment triplets in the text.

[0050] Preferably, the BDTF classifier is designed based on a boundary-driven table filling strategy, specifically including the following:

[0051] First of all, it should be made clear that the boundary-driven table filling strategy is not the strategy proposed in this patent. It is only used as a preferred combination in this patent.

[0052] Boundary-Driven Table-Filling (BDTF) is a new high-performance strategy for ASTE tasks. Figure 4 As shown, the aspect-opinion pair is represented as a two-channel 2D boundary region, starting from the upper left corner of the region S[a s , o s ] and the lower right corner end position E[a e , o e ] localization. Based on this, ASTE is converted into a multi-channel region extraction and classification task.

[0053] For the obtained high-dimensional semantics First, we construct a relation-level representation r between any words. ij , the process can be simplified as follows.

[0054]

[0055] Among them, gelu is the activation function.

[0056] For a sentence of length n, the relationship level between any two words is represented by Constructing a 3D relationship matrix Furthermore, R is fed into an L-layer ResNet-style CNN encoding layer to extract high-dimensional relationships. The process is as follows.

[0057] R (l) =σ(Conv(R (l-1) ))+R (l-1)

[0058] Among them, σ is an activation function (such as Relu), and Conv represents the convolution operation. Select the output R of the last layer of CNN (L) As the extraction result.

[0059] Finally, after a boundary detection and classification layer, a potential region candidate pool P is obtained, which records the location of the predicted aspect-opinion pair [S(a s , o s ), E(a e , o e )] and its emotional attitude S. This process can be simply expressed as follows.

[0060] P=(S 1 [a s , o s ], E 1 [a e , o e ], S 1 ),…,(S v [a s , o s ], E v [a e , o e ], S v )

[0061]

[0062] S i [a s , o s ]=top k (sigmoid(Linear(R (L) )))

[0063] E i [a e , o e]=top k (sigmoid(Linear(R (L) ])))

[0064] Here, k is a hyperparameter.

[0065] In the decoding phase, a region classifier is used to decode potential triplets and exclude candidates with the sentiment label “Invalid”. Potential triplets are marked by two labels that record the start and end positions of the candidate regions, as shown in the attached figure. Figure 4 As shown, the labels of the potential triplet regions are (S[a2, o4], E[a3, o5], Positive), where a i 、o i They represent aspect and opinion words respectively, and the subscripts indicate the corresponding position numbers.

[0066] The ultimate goal of training is to minimize the cross entropy loss between the predicted triplet and the true value. Given a true triplet of a review The prediction result is in The loss function of training can be expressed as follows.

[0067]

[0068]

[0069]

[0070]

[0071] In a third aspect, a computer-readable storage medium is provided that stores one or more programs, wherein the one or more programs include instructions that, when executed by a computing device, cause the computing device to perform any of the methods described.

[0072] According to a fourth aspect, a computing device is provided, comprising:

[0073] One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing any of the methods.

[0074] (3) Beneficial effects

[0075] To validate the benefits of the proposed model, we tested it on two versions of four well-known datasets from the ASTE task: LAP14, REST4, REST5, and REST6. We also compared the results with previous baseline models to demonstrate the advanced performance of the proposed model.

[0076] Tables 1 and 2 list the experimental results on two benchmark datasets, using the well-known ASTE task metrics of precision (P), recall (R), and F1 as evaluation metrics. These results show that the model proposed in this patent achieves a clear advantage on each sub-dataset.

[0077] Specifically, the following beneficial effects can be obtained.

[0078] (1) In the ASTE-Data-V1 dataset, the model proposed in this patent achieved an F1 score improvement of 1.77%, 1.49%, 1.29%, and 0.78% compared with the best joint extraction methods in LAP14, REST14, REST15, and REST16.

[0079] (2) In the ASTE-Data-V2 dataset, compared with the best joint extraction methods in LAP14, REST14, REST15, and REST16, the proposed model achieves an F1 score improvement of 0.43%, 1.88%, 1.31%, and 1.93%, respectively. Compared with the best pipeline and span-based methods, the F1 improvements are 3.99%, 7.59%, 8.64%, 6.85%, and 0.82%, 3.61%, 2.38%, and 2.82%, respectively.

[0080] (3) Compared with the best values ​​of the corresponding metrics of the previous model, the average improvements of the precision (P·) and recall (R·) metrics on the V1 and V2 datasets are -0.20, 2.60 and -0.39, 1.17, respectively. These results indicate that the improvement in F1 score is more due to the improvement in recall. The improvement in recall metric indicates that the model is more comprehensive in extracting true triplets. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 Flow chart of the method of the present invention;

[0082] Figure 2 This is a schematic diagram of the structure of a semantically enhanced dual encoder for aspect-level sentiment triple extraction according to the present invention;

[0083] Figure 3 This is an example of the aspect-level sentiment triple extraction task of the present invention;

[0084] Figure 4This is an example of table filling for the BDTF classifier of the present invention;

[0085] Table 1 shows the experimental results of the proposed system on the ASTE-Data-V1 dataset;

[0086] Table 2 shows the experimental results of the proposed system on the ASTE-Data-V2 dataset.

[0087]

[0088] DETAILED DESCRIPTION

[0089] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0090] Example

[0091] like Figure 1 As shown, an embodiment of the present invention provides an aspect-level sentiment triple extraction method based on semantic enhancement dual encoder, comprising:

[0092] According to the surface language expression of the input text, the basic semantics of the text is obtained with the help of a basic encoder;

[0093] Generate a syntactic dependency tree that can be used in graph convolutional networks based on the syntactic structure information of the input text;

[0094] Use natural language processing tools, such as SpaCy, to obtain the part-of-speech sequence and position sequence of the input text to obtain the deep semantics of the text;

[0095] Use the designed 3-domain word embedding layer to obtain multi-domain word embedding of the input text;

[0096] Based on the obtained text embeddings and text dependencies, a specially designed encoder is used to learn the specific semantics of the text;

[0097] The two semantic information are fully integrated through an iterative encoding interaction module based on the self-attention mechanism;

[0098] The potential sentiment triples in the fused semantic information are obtained through a BDTF classification module and a decoding module.

[0099] As a further limitation of this embodiment, the basic semantics of the text are obtained with the help of a basic encoder based on the surface language expression of the input text, specifically including: using the BERT encoder to generate a hidden state vector of the text, and using this as the basic contextual semantics of the sentence. Specifically, a sentence representation "[CLS]sentence[SEP]" is constructed and fed into the encoder as input to obtain the basic contextual semantics h of the word, where [CLS] and [SEP] are unique tags added by BERT at the beginning and end of the sentence. In addition, since BERT splits a complete word based on subwords, the obtained contextual feature h does not conform to the sentence length in terms of dimension. In order to remedy this problem, this patent designs a new word segmenter Whitetokenizer for BERT, which records each word that is split into subwords. At the same time, the subwords are aligned by averaging the set to obtain the contextual semantics h. b The process can be briefly described as follows:

[0100] h b =BERT(Whitetokenizer([[CLS],ω1,…,w n ,[SEP]]))

[0101]

[0102] It is worth noting that in h b The hidden states of the special marks [CLS] and [SEP] are removed, and the aligned where d b Represents the hidden state dimension of BERT.

[0103] As a further limitation of this embodiment, learning the specific semantics of the text using a designed specific encoder based on the obtained text embedding and text dependency specifically includes:

[0104] In order to accurately capture the semantic tendencies of words in different fields, we first use two different semantic lookup tables to initialize text embedding. Indicates the meaning of words in a broad context, and domain-specific embedding tables Represents the meaning of a word in a specific review context. Where |v| represents the size of the lookup table dictionary.

[0105] Different from the semantic information mentioned above, the word part-of-speech information establishes the distinction between words from a linguistic perspective. Words with different parts of speech often have specific collocations and distinctions. Use SpaCy to obtain the part-of-speech tag for each word. However, the total number of part-of-speech tags is large and the categories overlap. In order to distinguish and model the major categories of words, the part-of-speech tags are summarized as follows: where pnoun , p verb , p adj , p adv , pothers Represent nouns, verbs, adjectives, adverbs and other categories of words respectively. Based on this, a learnable POS embedding is designed Encode the part-of-speech information of a word.

[0106] Therefore, the final representation E of the word after 3-domain embedding is:

[0107]

[0108] As mentioned before, the components of a triple are highly contextual. Therefore, Bi-LSTM is used as a special encoder to learn the contextual features of the sentence. After several gating units and attention mechanisms, the special encoder takes the embedding sequence of the sentence as input to obtain the contextual features. where d l represents the dimension of the hidden state. This process can be simplified as follows:

[0109] X = Bi-LSTM(E)

[0110] Feature X reflects the contextual features of the text. In addition, the grammatical information of the text is also worthy of attention. In order to learn the grammatical structure information of the text, a multi-layer graph convolutional network (GCN) based on the syntactic dependency tree is designed. This can learn the grammatical information of each comment and directly enrich the representation of the current word through the state of directly related words. In the specific implementation, the result X of the Bi-LSTM is used as the initial state, and a multi-layer graph convolution operation is performed on the corresponding dependency tree to obtain the grammatically enhanced context state h p , the process can be summarized as follows:

[0111]

[0112] in, Represents the input features of each layer of the GCN network. W is the trainable parameter matrix of each layer, It is a 0-1 adjacency matrix that records the connections between words. yes The degree matrix records the number of neighbors of each node. Sum degree matrix Generated by the following formula:

[0113]

[0114]

[0115] So far, the basic context feature h learned based on BERT encoder has been obtained b and the specific context features h learned by the Bi-LSTM encoder and GCN network based on 3-domain embedding p Next, both are fed into the designed encoding interaction layer to fuse the contextual features obtained by the two different encoders.

[0116] like Figure 2 As shown, another embodiment of the present invention provides an aspect-level sentiment triple extraction system based on semantic enhancement dual encoders, comprising:

[0117] The basic encoder module is used to obtain the surface semantics of the text;

[0118] A specific encoder module is used to capture the deep semantics of the text and model deep grammatical information based on the part-of-speech information of words and the syntactic dependencies of sentences;

[0119] The encoding interaction module is used to iteratively fuse the acquired basic semantics and the semantic information specific to the review expression;

[0120] The BDTF classifier module is used to classify the obtained fusion semantics and extract possible sentiment triplets in the text.

[0121] The function of the coding interaction module is specific: basic semantics h b , which reflects the hidden state of each word based on the contextual semantics. Specific semantics h p , which reflects the semantics of the sentence in the context of the review domain, while also reflecting the differences in the part of speech of each word and the grammatical differences of the sentence. These two different semantics reflect the characteristics of the sentence from different perspectives. To integrate these two different aspects of information, this patent designs a clever encoding interaction layer to enhance the semantic features.

[0122] First, the basic semantics and specific semantics are fed into an interactive attention module to obtain the attention score α of each word under the two semantics. b and α p In the specific implementation, the interactive attention module is calculated by the following attention formula.

[0123]

[0124]

[0125] In theory, the attention score It records the semantic relevance between any words in a sentence, which can reflect the degree of semantic closeness between two words. It also carries a lot of information, which reflects the relevance of words in terms of part of speech and grammar. In order to interact the two types of semantics and enhance each other's expression, let the attention score α b (α p ) acts on the opposing semantic feature h p (h b ) to fuse the two semantics while reinforcing the original semantic features through self-looping. This implementation can be summarized as the following formula.

[0126] h b ′=Dropout(α p h b )+h b

[0127] h p ′=Dropout(α b h b )+h p

[0128] The purpose of setting a Dropout layer is to randomly mask some channels to resist gradient propagation errors and improve the robustness of the model.

[0129] The introduction of attention mechanism enables the interaction of two different semantic aspects. However, a single-layer interaction cannot fully integrate the semantics. A multi-layer interaction mechanism is further introduced to enhance the features. In each layer of interaction, h′ p Re-enter a BiLSTM encoding layer with multi-layer GCN. h′ b Directly serve as the input of the next layer of iteration. This process can be expressed as follows.

[0130]

[0131]

[0132]

[0133] in, represents the output of the l-th layer interaction, g k Represents a k-layer GCN network. represents the input of the t-1th layer in the GCN network, W (t-1) It is the trainable parameter of the t-1th layer of the GCN network. Represents the output of the lth layer interaction

[0134] In the above interaction, It can repeatedly extract effective information from the BERT base encoder and fuse it with itself through the interactive attention mechanism. We can effectively focus on the specific context encoding hp about differences in parts of speech and grammar, while reinforcing self-characteristics through interactive attention.

[0135] As shown in Tables 1 and 2, the beneficial effects of the aspect-level sentiment triple extraction system proposed in this patent are verified. Testing was performed on two versions of the four well-known ASTE datasets: LAP14, REST4, REST5, and REST6. Comparisons with previous baseline models demonstrate the advanced performance of the proposed model.

[0136] Tables 1 and 2 list the experimental results on two benchmark datasets, using the well-known ASTE task metrics of precision (P), recall (R), and F1 as evaluation metrics. These results show that the model proposed in this patent achieves a clear advantage on each sub-dataset.

[0137] Specifically, the following beneficial effects can be obtained.

[0138] (1) In the ASTE-Data-V1 dataset, the model proposed in this patent achieved an F1 score improvement of 1.77%, 1.49%, 1.29%, and 0.78% compared with the best joint extraction methods in LAP14, REST14, REST15, and REST16.

[0139] (2) In the ASTE-Data-V2 dataset, compared with the best joint extraction methods in LAP14, REST14, REST15, and REST16, the proposed model achieves an F1 score improvement of 0.43%, 1.88%, 1.31%, and 1.93%, respectively. Compared with the best pipeline and span-based methods, the F1 improvements are 3.99%, 7.59%, 8.64%, 6.85%, and 0.82%, 3.61%, 2.38%, and 2.82%, respectively.

[0140] (3) Compared with the best values ​​of the corresponding metrics of the previous model, the average improvements of the precision (P.) and recall (R.) metrics on the V1 and V2 datasets are -0.20, 2.60 and -0.39, 1.17. These results indicate that the improvement in F1 score is more due to the improvement in recall. The improvement in recall metric indicates that the model is more comprehensive in extracting true triplets.

[0141] The embodiments of the present application can be provided as methods or computer program products. Therefore, the application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Python and directly interpreted scripting language JavaScript, etc.

[0142] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0143] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0144] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0145] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

Claims

1. A method for extracting aspect-level sentiment triples based on semantically enhanced dual encoders, characterized in that: include: Learn text surface semantic information based on the basic context encoder; The specific semantic information of a specific review domain of the text is learned based on a specific context encoder; Receive the part-of-speech (POS) information and syntactic dependency information of the text, and model the syntactic and structural information of the text through a graph convolutional network; Design a coding interaction strategy that integrates basic semantics and specific semantics, including: The basic semantics and specific semantics are fed into an interactive attention module to obtain the attention scores of each word under the two semantics. and , the interactive attention module is calculated by the following attention formula: Attention score Records the semantic relevance between any words in a sentence, which is used to reflect the semantic closeness between two words; attention score Used to reflect the relevance of words in terms of part of speech and grammar; In order to interact the two types of semantics and enhance each other’s expression, let the attention score Acting on semantic features h p Above, attention score Acting on semantic features h b In order to integrate the two semantics and strengthen the original semantic features through self-loop, the specific summary is as follows: Set one of The role of the layer is to randomly mask some channels; A multi-layer interaction mechanism is further introduced to enhance the feature. In each layer of interaction, Re-feed into a BiLSTM encoding layer with multi-layer GCN, Directly used as the input of the next layer of iteration, specifically expressed by the following formula: in, Indicates the The output of the layer BERT encoding interaction result, g （k） Indicates a Layer GCN network, Indicates the first The input of the layer, It is the GCN network The trainable parameters of the layer, Indicates the The output of the layer-specific context encoding interaction results, represents the activation function, is a Adjacency matrix, which records the connections between words; yes The degree matrix records the number of neighbors of each node.

2. The aspect-level sentiment triple extraction method based on semantically enhanced dual encoder according to claim 1 is characterized in that: The basic context encoder learns the surface semantic information of the text, including: Construct a sentence representation [CLS] sentence [SEP] and feed it as input to the base encoder, which is mainly composed of a BERT encoder to obtain the basic contextual semantics of the word ,in and It is a unique token added by BERT at the beginning and end of a sentence; in addition, a space tokenizer is designed for BERT—— , used to record each word that is split into subwords; at the same time, the subwords are aligned by averaging the set to obtain the contextual semantics The specific process is briefly described as follows: exist Remove special marks and The hidden state of ,in Represents the hidden state dimension of BERT.

3. The aspect-level sentiment triple extraction method based on semantically enhanced dual encoder according to claim 1 is characterized in that: The specific semantic information of the text that is specific to the review field is learned based on the specific context encoder, specifically including: Text embedding is initialized using two different semantic lookup tables, where the general domain embedding table Indicates the meaning of words in a broad context, and domain-specific embedding tables Indicates the meaning of a word in a specific review context, where Represents the size of the lookup table dictionary; use SpaCy to get the part-of-speech tag for each word and summarize the part-of-speech tags as ,in Represent nouns, verbs, adjectives, adverbs, and other categories of words respectively; based on this, a learnable part-of-speech (POS) embedding table is designed , used to encode the word's part-of-speech information; after the word is embedded in the 3-domain, it is finally represented as E. The specific calculation process is as follows: Use Bi-LSTM as a special encoder to learn the contextual features of the sentence. After several gating units and attention mechanisms, the special encoder takes the embedded sequence of the sentence as input to obtain the contextual features. ,in Represents the dimension of the hidden state, and the calculation process is as follows: 。 4. The aspect-level sentiment triple extraction method based on semantic enhancement dual encoder according to claim 1 is characterized in that: Receive the part-of-speech (POS) information and syntactic dependency information of the text, and use the graph convolutional network to model the syntactic and structural information of the text, including: A multi-layer graph convolutional network (GCN) based on syntactic dependency tree is designed to learn the grammatical information of each comment and directly enrich the representation of the current word through the state of directly related words; in the specific implementation, the results of Bi-LSTM are combined As the initial state, perform multi-layer graph convolution operations on the corresponding dependency tree to obtain the grammatically enhanced context state , the process is summarized as follows: in, Represents the input features of the l-1 layer of the GCN network; for the input layer , is the trainable parameter matrix of each layer.

5. A semantically enhanced dual encoder-based aspect-level sentiment triple extraction system, characterized by: The system is used to implement the method according to any one of claims 1 to 4, and the system includes: The basic encoder module is used to obtain the surface semantics of the text; A specific encoder module is used to capture the deep semantics of the text and model deep grammatical information based on the part-of-speech information of words and the syntactic dependencies of sentences; The encoding interaction module is used to iteratively fuse the acquired basic semantics and the semantic information specific to the review expression; The BDTF classifier module is used to classify the obtained fusion semantics and extract sentiment triplets from the text.

6. A computer-readable storage medium storing one or more programs, characterized in that: The one or more programs include instructions that, when executed by a computing device, cause the computing device to perform any one of the methods according to claims 1-4.

7. A device for extracting aspect-level sentiment triples based on semantically enhanced dual encoders, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for executing any one of the methods according to claims 1-4.

Images