Domain-adaptive span-level aspect emotion triple extraction model establishment method and application thereof

Abstract

The invention discloses a domain-adaptive span-level aspect emotion triple extraction model establishment method and application thereof, and belongs to the technical field of emotion analysis, and the method comprises the following steps: extracting semantic information by adopting a span-level feature extraction method combining syntactic and semantic information, and shielding the expression of a prediction enhancement feature on syntactic information through part-of-speech tagging to obtain a domain-adaptive span-level aspect emotion triple extraction model; syntactic information and semantic information are fully utilized by span features; the domain invariant features of the span are enhanced by taking the dependency relationship tree as the core, the feature distribution is optimized by utilizing a mutual information maximization loss function, the features aiming at the specific field are transferred to be field-independent, and the cross-field prediction is realized on the premise of not increasing the labeling cost; according to the method, simple span-level tags are designed and only comprise six tags, so that under the condition of ensuring accurate extraction of an emotion triple in the aspect of span level, the calculation burden in the training process is reduced, and a clearer learning target is provided. According to the method, on the premise that the labeling cost is not increased, cross-domain span-level aspect emotion triple extraction can be realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of sentiment analysis, and more specifically, relates to a method for establishing a domain-adaptive span-level sentiment triple extraction model and its application. Background Art

[0002] With the advancement of Internet technology and the rapid development of social media, a large amount of text data with emotional information is generated on social networks every day. By deeply processing and studying these texts, we can obtain rich emotional information data, thus providing a strong reference for decision-making and meeting needs. The rapid development of social media has prompted text sentiment analysis to become one of the research fields that has attracted much attention in natural language processing. At the same time, text sentiment analysis also has a wide range of application scenarios in financial markets, e-commerce, policy analysis, public opinion guidance, literary appreciation and other fields.

[0003] Sentiment Analysis (SA) is also called opinion mining. According to the different text processing granularity, text sentiment analysis can be divided into paragraph-level sentiment analysis, sentence-level sentiment analysis and aspect-based sentiment analysis (ABSA). Among them, aspect-based sentiment analysis can provide more fine-grained, comprehensive and personalized sentiment insights, helping users understand the sentiment tendency of specific aspects in the text.

[0004] In aspect-level sentiment analysis, the target can be described by different attributes or aspects (aspect items) of an entity (including products, services, organizations, individuals, issues, events, topics and attributes, etc.), while sentiment includes a detailed expression of opinion (viewpoint item) and a sentiment polarity. The three sentiment elements of aspect item, viewpoint item and sentiment polarity constitute the main content of ABSA task research: aspect item is the opinion target that appears clearly in a given text; viewpoint item is the expression method that appears clearly in a given text to convey sentiment; sentiment polarity describes the sentiment tendency of aspect categories or aspect items, usually including positive, negative and neutral.

[0005] Aspect Sentiment Triplet Extraction (ASTE) is a comprehensive complex task in aspect-level sentiment analysis. It aims to identify aspect items and their sentiment tendencies in a sentence and extract the related opinion items to form a complete <aspect item, opinion item, sentiment polarity> triple. In the field of natural language processing, ASTE is a highly complex and challenging problem. Current research models the ASTE problem in a variety of ways. Among them, the span-level model can directly process continuous segments of text and better extract the overall semantics of aspect items and opinion items. Therefore, it has always been a major research direction.

[0006] Existing span-level models can achieve good results in a single domain with labeled data supervision. However, many real-world scenarios involve texts from multiple domains, some of which may come from unknown domains. In this case, there are usually huge differences in the aspects involved in different domains. The model may lack prior knowledge of commonly used terms in unknown domains, resulting in poor performance of ASTE tasks in cross-domain situations. Summary of the invention

[0007] In view of the defects and improvement needs of the prior art, the present invention provides a domain-adaptive span-level aspect sentiment triple extraction model establishment method and its application, which aims to achieve cross-domain span-level aspect sentiment triple extraction without increasing the annotation cost.

[0008] To achieve the above object, according to one aspect of the present invention, a method for establishing a domain-adaptive span-level aspect sentiment triple extraction model is provided, comprising:

[0009] Construct a basic feature extraction module; the basic feature extraction module is used to split text data into token sequences through word segmentation and obtain the representation vector of each token in the token sequence;

[0010] Construct a span-level feature extraction module; the span-level feature extraction module is used to extract semantic features from the representation vector of each Token in the text data, obtain the feature vector of each Token, and construct the span-level feature matrix H of the text data Span ; Span-level feature matrix H Span In the i-th row and j-th column, the element Represents the feature vector of the span with the i-th Token as the head vector and the j-th Token as the tail vector; the feature vector of the span is extracted based on the feature vector of the head Token, the feature vector of the tail Token of the span, and the span width; i,j∈{1,2,…N}, N represents the Token sequence length of the text data;

[0011] The source domain with the span-level sentiment triples annotated and the target domain text data without the span-level sentiment triples annotated are used to perform domain-invariant feature enhancement to optimize the span-level feature extraction module; the domain-invariant feature enhancement includes: predicting the dependency tree of the text data as the label data of the dependency tree; extracting the span-level feature matrix H according to the span-level feature extraction module Span Predict the dependency tree of text data and build a loss function based on the prediction results of the dependency tree and label data to optimize the span-level feature extraction module;

[0012] Construct a span-level label classification module; the span-level label classification module is used to classify the span-level feature matrix H of the text data Span Predict the category labels of each span and obtain the span-level label matrix; the category labels include: aspect items, opinion items, sentiment polarity and no label;

[0013] The constructed span-level label classification module is used to predict the category label of each span in the source domain text data, and the span-level aspect sentiment triples in the source domain text data are decoded. A loss function is constructed based on the prediction results and labeling results of the span-level aspect sentiment triples to optimize the span-level classification module.

[0014] The basic feature extraction module, the optimized span-level feature extraction module and the optimized span-level label classification module are connected in sequence to obtain a domain-adapted span-level aspect sentiment triplet extraction model.

[0015] Furthermore, the domain invariant feature enhancement further includes: constructing a mutual information maximization loss function for optimizing the span-level feature extraction module; the mutual information maximization loss function is:

[0016]

[0017]

[0018]

[0019] Among them, L mi represents the mutual information maximization loss function; Y represents the set of category labels of the span; p y represents the probability that the span is predicted to have the category label y, avg represents the calculated average, P(y) = avg(p y ), ρ represents the hyperparameter threshold.

[0020] Furthermore, the domain-adaptive span-level aspect sentiment triple extraction model establishment method provided by the present invention further includes: performing token-level feature enhancement using source domain and target domain text data to optimize the span-level feature extraction module;

[0021] Token-level feature enhancements include:

[0022] Perform part-of-speech tagging on each token in the token sequence of the text data as label data for part-of-speech tagging;

[0023] After randomly masking some tokens in the token sequence of text data, the span-level feature extraction module is used to extract the feature vector of each token, and the part of speech of each token is predicted based on the feature vector of each token;

[0024] A loss function is constructed based on the part-of-speech prediction results and label data to optimize the span-level feature extraction module.

[0025] Furthermore, sentiment polarity is divided into three categories: positive, negative, and neutral.

[0026] Furthermore, the loss function constructed based on the prediction results and labeling results of span-level aspect sentiment triples is a weighted focal loss function.

[0027] Furthermore, in the weighted focal loss function, the weight of each span is determined as follows:

[0028] The feature vector of the span Input the pre-trained domain classifier for domain classification and obtain the probability that the span belongs to the source domain and the probability that the span belongs to the target domain calculate as the weight of the corresponding span.

[0029] Furthermore, the token representation vector is formed by concatenating the token embedding, segment embedding, and position embedding;

[0030] Among them, Token embedding represents the index corresponding to the Token in the vocabulary, and the vocabulary is a set of all Tokens in the source domain and target domain text data; segment embedding is used to represent the sentence to which the Token belongs; and position embedding is used to represent the position of the Token in the entire Token sequence.

[0031] According to another aspect of the present invention, a method for extracting span-level aspect sentiment triples is provided, comprising:

[0032] Inputting the text data to be analyzed in the target domain into the domain-adapted span-level aspect sentiment triple extraction model established by the domain-adapted span-level aspect sentiment triple extraction model establishment method provided by the present invention, and obtaining the span-level label matrix of the text data to be analyzed;

[0033] The span-level label matrix of the text data to be analyzed is decoded to obtain the span-level aspect sentiment triples in the text data to be analyzed.

[0034] Furthermore, the span-level label matrix of the text data to be analyzed is decoded, including:

[0035] Traverse the span-level label matrix to collect all spans whose category labels are aspect items and all spans whose category labels are opinion items, and determine the starting positions and ending positions of all aspect item spans and the starting positions and ending positions of all opinion item spans;

[0036] Pair the aspect item spans and opinion item spans one by one to obtain all possible aspect item-opinion item pairs;

[0037] Traverse all possible aspect item-opinion item pairs; for each aspect item-opinion item pair traversed, use the Token position corresponding to the aspect item span as the row index and the Token position corresponding to the opinion item span as the column index, delineate the rectangular area corresponding to the aspect item-opinion item pair in the span-level label matrix, and count the number of sentiment polarities in the rectangular area, and take the sentiment polarity with the largest number in the statistical results as the sentiment polarity of the corresponding aspect item-opinion item pair;

[0038] After removing the unlabeled aspect item-opinion item pairs, span-level aspect sentiment triplets are constructed for the remaining aspect item-opinion item pairs.

[0039] According to another aspect of the present invention, a computer program product is provided, comprising a computer program; when the computer program is executed by a processor, the above-mentioned domain-adaptive span-level aspect sentiment triple extraction model establishment method provided by the present invention is implemented, and / or the above-mentioned span-level aspect sentiment triple extraction method provided by the present invention is implemented.

[0040] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects:

[0041] (1) The dependency tree is a domain-invariant feature that can serve as a bridge between the source domain and the target domain to identify and understand the commonalities and differences between different domains. After establishing a span-level feature extraction module for extracting span-level features of text data, the present invention uses span-level features in the source domain and target domain data for dependency prediction, and optimizes the span-level feature extraction module based on the dependency prediction results, which can enhance the expression of domain-invariant features by span-level features and effectively achieve domain adaptation. Therefore, the span-level label classification module trained on the source domain can also accurately extract the span classification labels of the target domain text data, and finally realize the extraction of cross-domain span-level aspect sentiment triples without increasing the annotation cost.

[0042] (2) In the preferred embodiment of the present invention, the span-level feature extraction module is further optimized based on the mutual information maximization loss function, thereby realizing the optimization of the span-level feature matrix. Through this step, the span-level feature distribution is further improved, the boundary between different categories is expanded, and the decision boundary learned on the source domain data is less likely to overlap with the decision boundary of the target domain, so that the span-level feature can better distinguish different categories, thereby enhancing the domain transfer capability of the model and providing better adaptability for the model when facing a changing data environment.

[0043] (3) The sentences in the sentiment ternary extraction task contain a large amount of syntactic and semantic information. Both types of information are helpful to improve the task performance, but existing methods often only focus on semantic information and fail to make full use of syntactic information. In the preferred embodiment of the present invention, when extracting the span-level features of text data, the semantic features of the tokens are first extracted to obtain the token-level features, thereby extracting rich semantic information of the sentences. Next, the part-of-speech tags are partially shielded and predicted using the token-level features to strengthen the expression of syntactic information by the token-level features. Subsequently, for each possible span, its head and tail tokens and length information are selected for combination, and the span-level features are further extracted to more comprehensively express the semantic and syntactic information. The span-level features finally constructed contain rich semantic and syntactic information at the same time, which can be used for the label classification task of ASTE sentiment analysis and improve the classification accuracy.

[0044] (4) Generally speaking, the fewer the number of categories, the fewer decision boundaries between categories the model needs to learn, and the easier it is for the model to distinguish the differences between different categories, so the classification task complexity is lower. In the preferred embodiment of the present invention, a unified span-level two-dimensional matrix label is designed. The label design controls the number of labels to 6, specifically including aspect items, opinion items, sentiment polarity and no label, where sentiment polarity is specifically divided into three types: positive, negative and neutral. This label design can effectively reduce the complexity of the training process while ensuring the effective extraction of span-level sentiment triples, providing a clearer learning goal for the model.

[0045] (5) While reducing the number of labels and the complexity of model training, the present invention also introduces new challenges. Due to the reduction in the number of effective labels, the balance of label distribution has also changed greatly, facing the challenge of imbalance between positive and negative samples. In the preferred embodiment of the present invention, when optimizing the span-level label classifier, a weighted focal loss function is introduced. This loss function introduces a focal factor on the basis of the traditional cross entropy loss function. By adjusting the weights of different categories, it focuses on samples that are difficult to classify in a targeted manner, thereby improving the learning effect of the model for minority categories. While effectively alleviating the problem of imbalance between positive and negative samples, it also helps to improve the classification performance of the model for each category.

[0046] (6) In the domain adaptation scenario, each span carries the dual characteristics of domain-invariant and domain-specific. In the preferred embodiment of the present invention, a domain classifier is introduced to obtain the domain distribution of each span. In the subsequent training of the span-level label classification module, the probability ratio of the span in the target domain and the source domain is used as the weight of each span to achieve dynamic weighting of the span. In this way, the dual characteristics of domain-invariant and domain-specific can be fully utilized. Through this comprehensive optimization strategy, the emotional information in cross-domain data can be captured more accurately, thereby achieving better performance in practical applications.

[0047] (7) In the preferred embodiment of the present invention, in the domain-adaptive span-level sentiment triple extraction model established, when the basic feature extraction module constructs the representation vector of each Token, it comprehensively utilizes Token embedding, segment embedding and position embedding to achieve multi-level and multi-dimensional representation, which not only enhances the model's semantic understanding of a single Token, but also improves the model's ability to grasp the entire text structure, thereby ultimately improving the Token-level feature vector's deep understanding and expression capabilities of text semantics, relationships between sentences and structural information. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic diagram of a method for establishing a domain-adaptive span-level aspect sentiment triple extraction model provided by an embodiment of the present invention;

[0049] Figure 2 A schematic diagram of extracting span-level features by combining syntactic and semantic information provided by an embodiment of the present invention;

[0050] Figure 3 A schematic diagram of span-level domain adaptation based on domain-invariant feature enhancement provided by an embodiment of the present invention;

[0051] Figure 4 A schematic diagram of coarse-grained span-level label encoding provided by an embodiment of the present invention;

[0052] Figure 5 A schematic diagram of coarse-grained span-level label decoding provided by an embodiment of the present invention;

[0053] Figure 6 A schematic diagram of optimizing a domain-weighted span-level label classifier provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0055] In the present invention, the terms "first", "second", etc. (if any) in the present invention and the accompanying drawings are used to distinguish similar objects but not necessarily to describe a specific order or sequence.

[0056] The ASTE task requires the annotator to not only identify the aspect items and opinion items in the text, but also to be able to judge the sentiment polarity between the two. Data in these three dimensions are necessary. Due to the complexity of this process and the high requirements for the professionalism of the annotator, specific fields often lack labeled data, and the performance of the span-level model is highly dependent on the data annotation in the specific field, which will directly affect the training effect and final performance of the span-level model, resulting in the existing span-level aspect sentiment triple extraction method being unable to effectively achieve cross-domain prediction. In order to solve this problem, the present invention provides a domain-adaptive span-level aspect sentiment triple extraction model establishment method and its application. The overall idea is to propose a domain adaptation method based on domain-invariant feature enhancement, strengthen the domain-invariant features of the span with the dependency tree as the core, transfer the features for a specific field to be domain-independent, and achieve cross-domain prediction without increasing the annotation cost. On this basis, the feature distribution is optimized using the mutual information maximization loss function to further enhance the domain independence of the feature.

[0057] In addition, the existing span-level models still have the following problems to be solved:

[0058] (1) Existing span-level models rely too much on semantic information and fail to mine and utilize syntactic information. This bias may lead to incomplete analysis results.

[0059] (2) The label design of the span-level model is complex. The ASTE task consists of three subtasks: aspect item extraction, opinion item extraction, and sentiment polarity recognition. The label design scheme of the span-level model needs to include relevant information of each subtask in the text segment. Therefore, a segmented or complex classification label system is often used, which brings additional challenges to the training and prediction of the model.

[0060] In response to the above problems, the present invention further proposes a span-level feature extraction method that combines syntactic and semantic information. It extracts semantic information through a pre-trained language model, and uses part-of-speech tagging to shield the prediction and enhance the expression of syntactic information, so that both syntactic and semantic information in the span are fully expressed. In addition, a streamlined span-level label design scheme is proposed, and a corresponding label encoding and decoding algorithm is designed to effectively address the problem of high model complexity in domain adaptation scenarios, reduce the computational burden in the training process while ensuring task effectiveness, and provide clearer learning goals.

[0061] Before explaining the technical solution of the present invention in detail, the following key concepts involved in span-level aspect sentiment triple extraction are explained.

[0062] Token: The basic unit of text processing in the natural language processing (NLP) model, which can be a word, sentence, or other meaningful text element, depending on the word segmentation algorithm used.

[0063] Span: A continuous sequence of tokens in a text. In sentiment analysis, span-level interactions can help the model better understand and capture the relationship between aspect items and opinion items.

[0064] The following are examples.

[0065] Embodiment 1:

[0066] A domain-adaptive span-level aspect sentiment triple extraction model building method, such as Figure 1 As shown, including:

[0067] Construct a basic feature extraction module; the basic feature extraction module is used to split text data into token sequences through word segmentation and obtain the representation vector of each token in the token sequence;

[0068] Construct a span-level feature extraction module; the span-level feature extraction module is used to extract semantic features from the representation vector of each Token in the text data, obtain the feature vector of each Token, and construct the span-level feature matrix H of the text data Span ; Span-level feature matrix H Span In the i-th row and j-th column, the element Represents the feature vector of the span with the i-th Token as the head vector and the j-th Token as the tail vector; the feature vector of the span is extracted based on the feature vector of the head Token, the feature vector of the tail Token of the span, and the span width; i,j∈{1,2,…N}, N represents the Token sequence length of the text data;

[0069] Construct a span-level label classification module; the span-level label classification module is used to classify the span-level feature matrix H of the text data Span Predict the category label of each span and obtain the span-level label matrix; the category labels include: aspect items, opinion items, sentiment polarity and no label.

[0070] After the basic feature extraction module, span-level feature extraction module and span-level label classification module constructed in this embodiment are connected in sequence, a model that can be used to predict the category label of each span in the text data can be obtained. Based on the span category predicted by the model, the span-level sentiment triples in the text data can be decoded.

[0071] As a preferred embodiment, Figure 1 and Figure 2 As shown, in this embodiment, the token representation vector is formed by connecting token embedding, segment embedding and position embedding;

[0072] Among them, Token embedding represents the index corresponding to the Token in the vocabulary, and the vocabulary is a set of all Tokens in the source domain and target domain text data; segment embedding is used to represent the sentence to which the Token belongs; and position embedding is used to represent the position of the Token in the entire Token sequence.

[0073] The token representation vector constructed in this embodiment realizes multi-level and multi-dimensional representation, which not only enhances the model's semantic understanding of a single token, but also improves the model's ability to grasp the entire text structure, ultimately improving the token-level feature vector's deep understanding and expression capabilities of text semantics, relationships between sentences, and structural information.

[0074] Optionally, in this embodiment, a pre-trained language assurance model BERT (Bidirectional Encoder Representations from Transformers) is specifically used to extract the feature vector of the Token.

[0075] In this embodiment, the feature vector of any span It mainly includes three parts, namely the feature vector of the first Token The feature vector of the tail token And the width of the span is represented by wid(i,j). These three parts work together to provide a comprehensive feature representation for the span. As an optional implementation, this embodiment connects the three parts and uses a fully connected layer to obtain the corresponding feature vector. The calculation formula is as follows:

[0076]

[0077] Among them, W Span and b Span They are the weight matrix and bias vector for feature learning respectively.

[0078] In order to achieve cross-domain prediction without increasing the annotation cost, this embodiment proposes a domain adaptation method based on domain-invariant feature enhancement, which uses the dependency tree as the core to enhance the domain-invariant features of the span and transfer the features for a specific domain to domain-independent features. Figure 1 and Figure 3 As shown, in this embodiment, the dependency tree of the text data is first extracted by preprocessing as the tag data TAG dep , and then perform domain-invariant feature enhancement using source domain text data annotated with span-level aspect sentiment triples and target domain text data unannotated with span-level aspect sentiment triples to optimize the span-level feature extraction module; the domain-invariant feature enhancement includes: extracting the span-level feature matrix H according to the span-level feature extraction module Span Predict the dependency tree of text data. Specifically, input the span-level feature matrix into the dependency classifier, use the softmax function to perform classification prediction of the dependency, and obtain the distribution probability of the dependency as follows:

[0079]

[0080] in, Representation basis Predicted dependency distribution probability; W dep and b dep They are the weight matrix and bias vector of the dependency classifier respectively;

[0081] A loss function is constructed based on the prediction results of the dependency tree and the label data to optimize the span-level feature extraction module.

[0082] Optionally, this embodiment adds a custom tag "sub" to the dependency library to indicate the dependency between tokens within the same word, thereby maintaining the integrity and semantic coherence of the word. Specifically, when obtaining the true dependency tag, if a word is split into multiple tokens, a dependency named "sub" is established for the first token and the subsequent tokens, where the first token is the core word and the subsequent tokens are dependent words, and the first token retains the original dependency relationship related to the word.

[0083] In order to improve the accuracy of the model, the method uses the cross entropy loss function for optimization, as follows:

[0084]

[0085] Among them, Ldep represents the cross entropy loss function constructed based on the prediction results of the dependency tree and the label data; D represents the data set consisting of the source domain and target domain text data; The dependency label obtained in the data preprocessing stage indicates the true dependency label with the ith token as the core word and the jth token as the dependency word. Since the model only considers the dependency between tokens directly connected in the dependency tree as training objects, D(i,j) is added to the loss function to mark whether the ith token and the jth token are directly connected. If they are directly connected, the value of D(i,j) is 1; if they are not directly connected, the value of D(i,j) is 0.

[0086] The dependency tree is a domain-invariant feature that can serve as a bridge between the source domain and the target domain to identify and understand the commonalities and differences between different fields. After establishing a span-level feature extraction module for extracting span-level features of text data, this embodiment uses span-level features in the source domain and target domain data to predict dependencies, and optimizes the span-level feature extraction module based on the dependency prediction results, which can enhance the expression of domain-invariant features by span-level features and effectively achieve domain adaptation.

[0087] In order to further enhance the domain transfer capability of the model and provide better adaptability for the model in the face of changing data environments, such as Figure 1 and Figure 3 As shown, this embodiment further introduces a mutual information maximization loss function in domain invariant feature enhancement.

[0088] In information theory, mutual information (MI) is an extremely important concept used to measure the mutual dependence between two random variables. The mutual information maximization loss function can ensure that the model does not favor any specific category, while helping to retain more information about the label and prompting the model to learn more discriminative feature representations, thereby better understanding the relationship between features and labels, increasing the boundaries between categories, and improving the confidence of classification results.

[0089] The calculation of label classification information entropy requires the distribution probability of the label category. The method approximates the distribution probability of the label category with the average label prediction probability of the feature, as follows:

[0090] P(y)=avg(p y ) Among them, p y represents the probability that the span is predicted to have the category label y, and avg represents the calculation of the average value of a batch;

[0091] Based on this, the information entropy calculation formula for label classification is as follows:

[0092]

[0093] For the case where the average label prediction probability P(y) is 0, it is specified that 0 log(0) = 0. Similarly, for the conditional entropy of label classification under the condition of a given feature, the model can also be approximated by the label prediction probability of the feature, as follows:

[0094]

[0095] For p y When 0 is 0, 0 log(0) = 0. According to the mathematical expression of mutual information, mutual information can be calculated as L1-L2. The mutual information maximization loss function is the negative mutual information. The loss function is defined as:

[0096]

[0097] Where ρ is the hyperparameter threshold. Maximizing L1 in the above mutual information maximization loss function helps ensure that the model does not favor any particular category, thereby avoiding the class collapse problem. This is achieved by introducing a constraint in the loss function, that is, L1 is maximized only when the value of L1 is less than the preset threshold ρ. In this way, the model will be encouraged to explore more classification possibilities during training, rather than just focusing on a single, high-frequency category. In addition, by minimizing another loss function L2, the model is encouraged to be more confident in making classification decisions, thereby reducing the uncertainty of the classification results. This approach enables the model to further expand the boundaries between different categories, and the decision boundaries learned on the source domain data are more likely to overlap with the decision boundaries of the target domain, which is conducive to domain transfer.

[0098] Considering that the sentences in the aspect sentiment ternary extraction task contain a large amount of syntactic and semantic information, both of which are helpful to improve the task performance. In order to utilize both syntactic information and semantic information at the same time, as a preferred implementation, this embodiment also includes:

[0099] Through preprocessing, each token in the Token sequence of the text data is tagged with a part-of-speech tag as the tag data TAG of the part-of-speech POS ; Moreover, the present embodiment further includes: performing Token-level feature enhancement using source domain and target domain text data to optimize the span-level feature extraction module;

[0100] Token-level feature enhancements include:

[0101] Randomly mask some tokens in the token sequence of text data; optionally, this embodiment randomly selects about 25% of the tokens and replaces the original tokens and the corresponding part-of-speech tags with a special tag "[Mask]"; in addition, when performing the [Mask] masking operation, the impact of word segmentation needs to be considered. Therefore, in this embodiment, if a token is masked, other tokens belonging to the same word will also be masked; the representation vector of the masked token is represented as E Mask ;

[0102] After the masking process is completed, the representation vector of each token is input into the BERT model to obtain the feature vector of each token and the token-level features of the text data. The Token-level feature H of the text data Token Input the pre-trained part-of-speech tagging classifier to predict the part-of-speech distribution probability of each token as follows:

[0103]

[0104] Among them, W POS and b POS are the weight matrix and bias vector of the part-of-speech tagging classifier respectively.

[0105] After obtaining the probability distribution of the part-of-speech tag, the model compares the predicted result with the actual part-of-speech tag that is masked to optimize the prediction effect. This step is completed through the cross-entropy loss function, which is a common method to evaluate the difference between the predicted probability distribution and the true distribution. The optimization of the cross-entropy loss function helps to adjust the model parameters, thereby reducing the prediction error and improving the accuracy of the model in the part-of-speech tagging classification task. The specific formula of this optimization goal is as follows:

[0106]

[0107] Where, M(·) is the shielding indicator function; The real part-of-speech tag sequence obtained in the data preprocessing stage represents the real part-of-speech tag of the i-th Token. Since the model only considers the part-of-speech tags masked by [Mask] as training objects, M(i) is added to the loss function to indicate whether the i-th Token is masked by [Mask]. If the i-th Token is masked, the value of M(i) is 1; otherwise, if the i-th Token is not masked, the value of M(i) is 0.

[0108] The method for optimizing the span-level classification module in this embodiment includes:

[0109] The constructed span-level label classification module is used to predict the category labels of each span in the source domain text data, and the span-level aspect sentiment triples in the source domain text data are decoded. A loss function is constructed based on the prediction results and labeling results of the span-level aspect sentiment triples to optimize the span-level classification module.

[0110] Generally speaking, the fewer the number of categories, the fewer decision boundaries between categories the model needs to learn, and the easier it is for the model to distinguish the differences between different categories, so that the complexity of the classification task is lower. As a preferred implementation, this embodiment designs a unified span-level two-dimensional matrix label, which controls the number of labels to 6, including aspect items, opinion items, sentiment polarity and no labels, where sentiment polarity is specifically divided into three types: positive, negative and neutral, as shown in Table 1, where "A" and "O" represent aspect items and opinion items, respectively, "POS", "NEU" and "NEG" represent positive sentiment polarity, neutral sentiment polarity and negative sentiment polarity, respectively, and "N" represents no label.

[0111] Specifically, when the span-level label matrix predicted by the span-level label classification module, the label Tag i,j ∈{"A","O"}, it indicates that the span with the i-th Token as the head Token and the j-th Token as the tail Token in the text constitutes a complete aspect item or opinion item; when the tag Tag i,j ∈{"POS","NEU","NEG"}, it indicates that the aspect item with subscript i and the opinion item with subscript j in the span form an aspect-opinion item pair, and the corresponding sentiment polarity is positive, negative or neutral.

[0112] Table 1 Coarse-grained span-level ASTE label meaning table

[0113]

[0114] This embodiment needs to encode the source domain text data annotated with span-level sentiment triples into a corresponding span-level label matrix. Corresponding to the classification method shown in Table 1, the label encoding process can be described by the pseudo code shown in Table 2:

[0115] Table 2 Pseudocode of label encoding algorithm

[0116]

[0117]

[0118] When encoding, first, for a given triple, identify the position of the aspect item in the sentence. This step scans the sentence and identifies the phrases that match the aspect item. Once the position of the aspect item is determined, the algorithm records the start and end positions of the aspect item in the sentence and converts this position information into corresponding coordinate information. That is, the start and end subscripts of the aspect item are used as the x-axis and y-axis coordinates of the label, respectively, and the span label is designated as "A", indicating that this is an aspect item.

[0119] Then, identify the opinion item in the sentence, find its position in the sentence, and determine the starting subscript and ending subscript of the opinion item in the sentence. That is, the starting and ending subscripts of the opinion item are used as the x-axis and y-axis coordinates of the label, respectively, and the span label is designated as "O", indicating that this is an opinion item.

[0120] Finally, according to the sentiment tendency contained in the given triple, the areas corresponding to the aspect items and opinion items are sentimentally labeled. Specifically, the subscript of the aspect item is used as the x-axis coordinate, the subscript of the opinion item is used as the y-axis coordinate, and the corresponding sentiment label is set for the span in this coordinate area: "POS" represents positive sentiment, "NEU" represents neutral sentiment, and "NEG" represents negative sentiment.

[0121] Figure 4 The figure shows the actual encoding effect of the above encoding method in the example sentence "The Thai food was great but the service was poor". First, the sentiment triple <"Thai food", "great", "positive"> is processed. For the aspect item "Thai food", the sentence is scanned to determine that its starting subscript in the sentence is 1 and its ending subscript is 2. For the opinion item "great", its subscript in the sentence is identified as 4. Then, the algorithm sets the span label label[1][2] to "A" to indicate the aspect item, and sets the span label label[4][4] to "O" to indicate the opinion item. For "positive", the area corresponding to the aspect item and the opinion item is located, and the span labels label[1][4] and label[2][4] are set to "POS" to indicate a positive sentiment tendency. Next, for the triple <"service", "poor", "negative">, similar steps are performed to set the span label label[7][7] to "A", the span label label[9][9] to "O", and the span label label[7][9] to "NEG".

[0122] Correspondingly, the decoding process based on the span-level label matrix in this embodiment can be described by the pseudo code shown in Table 3:

[0123] Table 3 Label decoding algorithm pseudo code

[0124]

[0125] When decoding, first, traverse the label matrix. The label matrix stores the predicted results of span-level sentiment analysis of the test text, including the location information of aspect items and opinion items. During the traversal process, the goal of the algorithm is to collect all predicted aspect items and opinion items and record the start and end subscripts. This process is to accurately locate the position of each aspect item and opinion item in the original text in the subsequent steps and determine the query scope for predicting sentiment tendency.

[0126] Then, traverse all possible aspect item-opinion item pairs. Count the number of sentiment labels in the entire rectangular area corresponding to each aspect item-opinion item pair. The x-axis subscript of the rectangular area corresponds to the subscript of the aspect item, and the y-axis subscript of the rectangular area corresponds to the subscript of the opinion item. During the statistical process, count the three sentiment tendency labels "POS", "NEU", and "NEG" separately. After the statistics are completed, the sentiment tendency with the highest frequency will be used as the predicted sentiment tendency of the aspect item-opinion item pair.

[0127] Finally, according to the start and end subscripts of the aspect items and opinion items obtained at the beginning, the algorithm extracts the specific content of the aspect items and opinion items from the original text, and combines them with the predicted sentiment tendency to form a complete sentiment triple.

[0128] Figure 5 As shown, according to the above decoding method Figure 4 An example of decoding the span-level label matrix shown in Figure 1. First, the aspect items "Thai food" and "service" and the opinion items "great" and "poor" are extracted according to the predicted label matrix. All possible aspect-opinion item pairs are traversed, and the corresponding rectangular area in the label matrix is ​​scanned. The number of each sentiment tendency label is counted. If there is a sentiment label, the sentiment label with the largest number is used as the corresponding sentiment polarity. If there is no sentiment label, the aspect-opinion item pair is abandoned. For example, the rectangular area corresponding to "Thai food"-"great" has 2 "POS" labels, and the number of "NEU" and "NEG" labels is 0. Then the aspect-opinion item pair is established and the sentiment tendency is "positive"; there is no sentiment label in the rectangular area corresponding to "Thai food"-"poor", so the aspect-opinion item pair is abandoned. Two sentiment triplets <"Thai food", "great", "positive"> and <"service", "poor", "negative"> can be decoded.

[0129] During decoding, this embodiment pairs the aspect items and viewpoint items extracted from the span-level label matrix one by one, and searches for possible emotional tendencies, with a larger number of emotional tendencies as part of the emotional triple extraction results. This label decoding method extracts as many valid emotional triples as possible, and well alleviates the problem of emotional tendency misclassification that is prone to occur, thereby improving the diversity and accuracy of emotional triple extraction.

[0130] Furthermore, this embodiment takes into account that while reducing the number of labels reduces the complexity of model training, it also introduces new challenges. Due to the reduction in the number of effective labels, the balance of label distribution has also changed greatly, facing the challenge of imbalance between positive and negative samples. In order to meet this challenge, this embodiment constructs a weighted focal loss function when optimizing the span-level label classification model through training. This loss function introduces a focal factor on the basis of the traditional cross entropy loss function. By adjusting the weights of different categories, it focuses on samples that are difficult to classify in a targeted manner, thereby improving the learning effect of the model for minority categories. While effectively alleviating the problem of imbalance between positive and negative samples, it also helps to improve the classification performance of the model for each category. Figure 6 As shown, this embodiment further introduces a pre-trained domain classifier, and inputs the span-level feature matrix extracted by the span-level feature extraction module into the domain classifier to obtain the domain distribution probability of the corresponding span as follows:

[0131]

[0132] Among them, W d and b d are the weight matrix and bias vector of the domain classifier, respectively.

[0133] The domain distribution probability predicted by the domain classifier will be used to dynamically adjust the weight of each span in the weighted focal loss function. Specifically, the ratio of the target domain probability to the source domain probability of the span is calculated, that is, And use it as the weight of the corresponding span. In this way, the model can dynamically adjust the weight of the span so that the span that is more important in the target domain can play a greater role in span-level sentiment label classification.

[0134] Optionally, in this embodiment, the domain classifier is optimized using a cross entropy loss function during training, and the corresponding loss function calculation formula is as follows:

[0135]

[0136] In order to avoid the cross-domain generalization problem during the domain classifier training process, in this embodiment, the back propagation of the domain classifier is cut off so that L dOnly the weight matrix and bias vector in the domain classifier are optimized without affecting the training of other parts. This approach not only improves the adaptability of the model to the target domain, but also ensures accuracy in capturing key features and information, thereby achieving better performance on the main task, the ASTE task.

[0137] Based on the above weights, in this embodiment, the weighted focal loss function constructed for the span-level label classifier is expressed as follows:

[0138]

[0139] Among them, FL represents the focal loss function, which adds a modulation factor on the basis of the cross entropy loss function and reduces the weight of easy-to-classify samples in the loss function; is the span label predicted by the span-level label classifier, is the real span-level classification label obtained after the real emotion triplet is processed by the label encoding algorithm. Through this comprehensive optimization strategy, this embodiment can more accurately capture the emotion information in the cross-domain data, thereby achieving better performance in practical applications.

[0140] In order to further refine the span-level features, this embodiment also introduces a fully connected layer before inputting the span-level features into the span-level label classifier for classification, and transforms the features of each span so as to generate a more discriminative feature representation for each category, as follows:

[0141]

[0142] After that, the category features of the span after processing by the fully connected layer are input into the span-level label classifier to obtain the label distribution probability of each category of the span, as follows:

[0143]

[0144] In the test phase, the label distribution probabilities of each category in the span are compared, and the label with the highest probability is used as the prediction result of the span, as follows:

[0145]

[0146] In general, this embodiment proposes a span-level domain adaptation method based on domain-invariant feature enhancement and distribution optimization. The domain-invariant features of the span are strengthened with the dependency tree as the core, and the feature distribution is optimized using the mutual information maximization loss function, so that the features for specific fields are transferred to be domain-independent, and cross-domain prediction is achieved without increasing the annotation cost. On this basis, a span-level feature extraction method combining syntactic and semantic information is proposed. The semantic information is extracted through a pre-trained language model, and the expression of syntactic information by the prediction enhancement feature is shielded by part-of-speech tagging, so that both the syntactic information and the semantic information in the span are fully expressed. In addition, a unified and streamlined span-level label design scheme is proposed, and a corresponding label encoding and decoding algorithm is designed to effectively deal with the problem of high model complexity in domain adaptation scenarios, reduce the computational burden in the training process while ensuring the effectiveness of the task, and provide clearer learning goals.

[0147] Embodiment 2:

[0148] A span-level aspect sentiment triple extraction method, comprising:

[0149] Input the text data to be analyzed in the target domain into the domain-adapted span-level aspect sentiment triple extraction model established by the domain-adapted span-level aspect sentiment triple extraction model establishment method provided in the above embodiment 1, and obtain the span-level label matrix of the text data to be analyzed;

[0150] The span-level label matrix of the text data to be analyzed is decoded to obtain the span-level aspect sentiment triples in the text data to be analyzed.

[0151] In this embodiment, decoding the span-level label matrix of the text data to be analyzed includes:

[0152] Traverse the span-level label matrix to collect all spans whose category labels are aspect items and all spans whose category labels are opinion items, and determine the starting positions and ending positions of all aspect item spans and the starting positions and ending positions of all opinion item spans;

[0153] Pair the aspect item spans and opinion item spans one by one to obtain all possible aspect item-opinion item pairs;

[0154] Traverse all possible aspect item-opinion item pairs; for each aspect item-opinion item pair traversed, use the Token position corresponding to the aspect item span as the row index and the Token position corresponding to the opinion item span as the column index, delineate the rectangular area corresponding to the aspect item-opinion item pair in the span-level label matrix, and count the number of sentiment polarities in the rectangular area, and take the sentiment polarity with the largest number in the statistical results as the sentiment polarity of the corresponding aspect item-opinion item pair;

[0155] After removing the unlabeled aspect item-opinion item pairs, span-level aspect sentiment triplets are constructed for the remaining aspect item-opinion item pairs.

[0156] Comparative experiments were conducted on six source domains and target domains consisting of four data sets of the International Conference on Semantic Evaluation. The experimental results show that compared with the average optimal baseline method, the method proposed in this embodiment improves the F1 value evaluation standard by 3.99%, and also improves the precision and recall rates.

[0157] Embodiment 3:

[0158] A computer program product includes a computer program; when the computer program is executed by a processor, it implements the domain-adaptive span-level aspect sentiment triple extraction model establishment method provided in the above-mentioned embodiment 1, and / or the span-level aspect sentiment triple extraction method provided in the above-mentioned embodiment 2.

[0159] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A domain-adaptive span-level aspect sentiment triple extraction model establishment method, characterized in that: include: Construct basic feature extraction module; The basic feature extraction module is used to split the text data into a Token sequence through word segmentation processing, and obtain a representation vector of each Token in the Token sequence; Construct a span-level feature extraction module; the span-level feature extraction module is used to extract semantic features from the representation vector of each Token in the text data, obtain the feature vector of each Token, and construct the span-level feature matrix H of the text data Span ; The span-level feature matrix H Span In the i-th row and j-th column, the element Represents the feature vector of the span with the i-th Token as the head vector and the j-th Token as the tail vector; the feature vector of the span is extracted based on the feature vector of the head Token, the feature vector of the tail Token of the span, and the span width; i,j∈{1,2,…N}, N represents the Token sequence length of the text data; Performing domain-invariant feature enhancement using source domain text data labeled with span-level aspect sentiment triples and target domain text data without labeled span-level aspect sentiment triples to optimize the span-level feature extraction module; The domain invariant feature enhancement includes: predicting a dependency tree of text data as label data of the dependency tree; extracting a span-level feature matrix H according to the span-level feature extraction module; Span Predicting a dependency tree of text data, and constructing a loss function based on the prediction result of the dependency tree and the label data to optimize the span-level feature extraction module; Construct a span-level label classification module; the span-level label classification module is used to classify the span-level feature matrix H of the text data according to the span-level feature matrix H of the text data. Span Predict the category labels of each span and obtain the span-level label matrix; the category labels include: aspect items, opinion items, sentiment polarity and no label; The constructed span-level label classification module is used to predict the category label of each span in the source domain text data, and the span-level aspect sentiment triples in the source domain text data are decoded and obtained, and a loss function is constructed according to the prediction result and the labeling result of the span-level aspect sentiment triples to optimize the span-level classification module; The basic feature extraction module, the optimized span-level feature extraction module and the optimized span-level label classification module are connected in sequence to obtain the domain-adapted span-level aspect sentiment triple extraction model.

2. The method for establishing a domain-adaptive span-level aspect sentiment triple extraction model as described in claim 1, characterized in that: The domain invariant feature enhancement further includes: constructing a mutual information maximization loss function for optimizing the span-level feature extraction module; the mutual information maximization loss function is: Among them, L mi represents the mutual information maximization loss function; Y represents the set of category labels of the span; p y represents the probability that the span is predicted to have the category label y, avg represents the calculated average, P(y) = avg(p y ), ρ represents the hyperparameter threshold.

3. The method for establishing a domain-adaptive span-level aspect sentiment triple extraction model as described in claim 1, characterized in that: include: Performing token-level feature enhancement using source domain and target domain text data to optimize the span-level feature extraction module; The Token-level feature enhancements include: Perform part-of-speech tagging on each token in the token sequence of the text data as label data for part-of-speech tagging; After randomly masking some tokens in the token sequence of the text data, the span-level feature extraction module is used to extract the feature vector of each token, and the part of speech of each token is predicted according to the feature vector of each token; A loss function is constructed according to the prediction results of the part of speech and the label data to optimize the span-level feature extraction module.

4. The method for establishing a domain-adaptive span-level aspect sentiment triple extraction model as claimed in claim 1, characterized in that: The sentiment polarity is divided into three categories: positive, negative and neutral.

5. The method for establishing a domain-adaptive span-level aspect sentiment triple extraction model as described in claim 4, characterized in that: The loss function constructed based on the prediction results and labeling results of span-level aspect sentiment triples is a weighted focal loss function.

6. The method for establishing a domain-adaptive span-level aspect sentiment triple extraction model as claimed in claim 5, characterized in that: In the weighted focal loss function, the weight of each span is determined as follows: The feature vector of span Input the pre-trained domain classifier for domain classification and obtain the probability that the span belongs to the source domain and the probability that the span belongs to the target domain calculate as the weight of the corresponding span.

7. The method for establishing a domain-adaptive span-level aspect sentiment triple extraction model according to any one of claims 1 to 6, characterized in that: The token representation vector is formed by connecting the token embedding, segment embedding and position embedding; Among them, Token embedding represents the index corresponding to the Token in the vocabulary, and the vocabulary is a set of all Tokens in the source domain and target domain text data; the segment embedding is used to represent the sentence to which the Token belongs; and the position embedding is used to represent the position of the Token in the entire Token sequence.

8. A method for extracting span-level aspect sentiment triples, characterized in that: include: Inputting the text data to be analyzed in the target domain into the domain-adapted span-level aspect sentiment triple extraction model established by the domain-adapted span-level aspect sentiment triple extraction model establishment method according to any one of claims 1 to 7, and obtaining the span-level label matrix of the text data to be analyzed; The span-level label matrix of the text data to be analyzed is decoded to obtain the span-level aspect sentiment triples in the text data to be analyzed.

9. The method for extracting span-level aspect sentiment triples as claimed in claim 8, characterized in that: Decoding the span-level label matrix of the text data to be analyzed includes: Traverse the span-level label matrix to collect all spans whose category labels are aspect items and all spans whose category labels are opinion items, and determine the start and end positions of all aspect item spans and the start and end positions of all opinion item spans; Pair the aspect item spans and opinion item spans one by one to obtain all possible aspect item-opinion item pairs; Traverse all possible aspect item-opinion item pairs; for each aspect item-opinion item pair traversed, use the Token position corresponding to the aspect item span as the row index and the Token position corresponding to the opinion item span as the column index, delineate the rectangular area corresponding to the aspect item-opinion item pair in the span-level label matrix, and count the number of sentiment polarities in the rectangular area, and take the sentiment polarity with the largest number in the statistical results as the sentiment polarity of the corresponding aspect item-opinion item pair; After removing the unlabeled aspect item-opinion item pairs, span-level aspect sentiment triplets are constructed for the remaining aspect item-opinion item pairs.

10. A computer program product, characterized in that It includes a computer program; when the computer program is executed by a processor, it implements the domain-adaptive span-level aspect sentiment triple extraction model establishment method described in any one of claims 1 to 7, and / or the span-level aspect sentiment triple extraction method described in claim 8 or 9.

Images