Domain-adaptive span-level aspect sentiment triple extraction model establishment method and its application

Abstract

The present invention discloses a method for establishing a domain-adaptive span-level aspect sentiment triple extraction model and its application, belonging to the field of sentiment analysis technology, including: extracting semantic information using a span-level feature extraction method that combines syntactic and semantic information, and shielding the expression of syntactic information by predicting and strengthening features through part-of-speech tagging, so that span features can fully utilize syntactic and semantic information; strengthening the domain-invariant features of spans with a dependency tree as the core, and optimizing feature distribution using a mutual information maximization loss function, transferring features for specific fields to be field-independent, and achieving cross-domain prediction without increasing the labeling cost; designing a streamlined span-level label, which only includes 6 types of labels, while ensuring accurate extraction of span-level aspect sentiment triples, reducing the computational burden in the training process, and providing a clearer learning goal. The present invention can achieve cross-domain span-level aspect sentiment triple extraction without increasing the labeling cost.

Description

Technical Field

[0001] The present invention belongs to the field of sentiment analysis technology, and more specifically, relates to a method for establishing a domain-adaptive span-level sentiment triple extraction model and its application. Background Art

[0002] With advances in internet technology and the rapid growth of social media, massive amounts of textual data containing sentimental information are generated daily on social networks. Through in-depth processing and analysis of this text, we can obtain rich emotional data, providing a powerful reference for decision-making and fulfilling needs. The rapid development of social media has made text sentiment analysis a highly sought-after research area in natural language processing. It also has a wide range of applications in financial markets, e-commerce, policy analysis, public opinion guidance, literary appreciation, and other fields.

[0003] Sentiment Analysis (SA), also known as opinion mining, can be categorized into paragraph-level sentiment analysis, sentence-level sentiment analysis, and aspect-based sentiment analysis (ABSA), depending on the granularity of the text being processed. Aspect-based sentiment analysis provides more granular, comprehensive, and personalized sentiment insights, helping users understand sentiments about specific aspects of a text.

[0004] In aspect-level sentiment analysis, targets can be described using different attributes or aspects (aspect items) of an entity (including products, services, organizations, individuals, issues, events, topics, and attributes), while sentiment includes a detailed expression of opinion (viewpoint item) and a sentiment polarity. These three sentiment elements—aspect items, viewpoint items, and sentiment polarity—form the core of ABSA task research: aspect items are explicit opinion targets that appear in a given text; viewpoint items are explicit expressions that convey sentiment within a given text; and sentiment polarity describes the sentiment toward an aspect category or aspect item, typically including positive, negative, and neutral.

[0005] Aspect Sentiment Triplet Extraction (ASTE) is a comprehensive, complex task in aspect-level sentiment analysis. It aims to identify aspect terms and their sentiment within a sentence and extract the associated opinion terms, forming a complete <aspect term, opinion term, sentiment polarity> triple. In natural language processing, ASTE is a highly complex and challenging problem. Current research models ASTE using various approaches, among which span-level models, which can directly process continuous segments of text and better extract the overall semantics of aspect and opinion terms, have been a major research direction.

[0006] Existing span-level models can achieve good results when there is labeled data supervision in a single domain. However, many real-world scenarios involve text from multiple domains, some of which may come from unknown domains. In this case, there are usually huge differences in the aspects involved in different domains. The model may lack prior knowledge of commonly used terms in unknown domains, resulting in poor performance of ASTE tasks in cross-domain situations. Summary of the Invention

[0007] In response to the defects of the existing technology and the need for improvement, the present invention provides a domain-adaptive span-level aspect sentiment triple extraction model establishment method and its application, the purpose of which is to achieve cross-domain span-level aspect sentiment triple extraction without increasing the annotation cost.

[0008] To achieve the above objectives, according to one aspect of the present invention, a method for establishing a domain-adaptive span-level aspect sentiment triple extraction model is provided, comprising:

[0009] Build a basic feature extraction module; the basic feature extraction module is used to split text data into token sequences through word segmentation and obtain the representation vector of each token in the token sequence;

[0010] Construct a span-level feature extraction module; the span-level feature extraction module is used to extract semantic features from the representation vector of each Token in the text data, obtain the feature vector of each Token, and construct the span-level feature matrix H of the text data Span ; Span-level feature matrix H Span In the ith row and jth column, the element Represents the feature vector of the span with the i-th token as the head vector and the j-th token as the tail vector; the feature vector of the span is extracted based on the feature vector of the first token of the span, the feature vector of the tail token of the span, and the span width; i,j∈{1,2,…N}, N represents the token sequence length of the text data;

[0011] The domain-invariant feature enhancement is performed using source domain text data labeled with span-level sentiment triples and target domain text data without span-level sentiment triples to optimize the span-level feature extraction module. The domain-invariant feature enhancement includes: predicting the dependency tree of the text data as the label data of the dependency tree; and extracting the span-level feature matrix H from the span-level feature extraction module. Span Predict the dependency tree of text data and build a loss function based on the prediction results of the dependency tree and label data to optimize the span-level feature extraction module;

[0012] Construct a span-level label classification module; the span-level label classification module is used to classify the span-level feature matrix H of the text data Span Predict the category label of each span and obtain the span-level label matrix; the category labels include: aspect item, opinion item, sentiment polarity and no label;

[0013] The constructed span-level label classification module is used to predict the category label of each span in the source domain text data, and the span-level aspect sentiment triples in the source domain text data are decoded. A loss function is constructed based on the prediction and labeling results of the span-level aspect sentiment triples to optimize the span-level classification module.

[0014] The basic feature extraction module, the optimized span-level feature extraction module and the optimized span-level label classification module are connected in sequence to obtain a domain-adapted span-level aspect sentiment triplet extraction model.

[0015] Furthermore, the domain-invariant feature enhancement further includes: constructing a mutual information maximization loss function for optimizing the span-level feature extraction module; the mutual information maximization loss function is:

[0016]

[0017]

[0018]

[0019] Among them, L mi represents the mutual information maximization loss function; Y represents the set of category labels of the span; p y represents the probability that the span is predicted to have the category label y, avg represents the calculated average value, P(y) = avg(p y ); ρ represents the hyperparameter threshold.

[0020] Furthermore, the domain-adaptive span-level sentiment triple extraction model establishment method provided by the present invention further includes: performing token-level feature enhancement using source domain and target domain text data to optimize the span-level feature extraction module;

[0021] Token-level feature enhancements include:

[0022] Perform part-of-speech tagging on each token in the token sequence of the text data as label data for part-of-speech tagging;

[0023] After randomly masking some tokens in the token sequence of the text data, the span-level feature extraction module is used to extract the feature vectors of each token, and the part of speech of each token is predicted based on the feature vectors of each token;

[0024] A loss function is constructed based on the part-of-speech prediction results and label data to optimize the span-level feature extraction module.

[0025] Furthermore, sentiment polarity is divided into three categories: positive, negative, and neutral.

[0026] Furthermore, the loss function constructed based on the prediction results and labeling results of span-level sentiment triples is a weighted focal loss function.

[0027] Furthermore, in the weighted focal loss function, the weight of each span is determined as follows:

[0028] The feature vector of the span Input the pre-trained domain classifier to perform domain classification and obtain the probability that the span belongs to the source domain and the probability that the span belongs to the target domain calculate as the weight of the corresponding span.

[0029] Furthermore, the token representation vector is formed by concatenating the token embedding, segment embedding, and position embedding;

[0030] Among them, Token embedding represents the index of the token in the vocabulary, and the vocabulary is a set of all tokens in the source domain and target domain text data; segment embedding is used to represent the sentence to which the token belongs; position embedding is used to represent the position of the token in the entire token sequence.

[0031] According to another aspect of the present invention, a method for extracting span-level aspect sentiment triples is provided, comprising:

[0032] Inputting the text data to be analyzed in the target domain into the domain-adapted span-level aspect sentiment triple extraction model established by the domain-adapted span-level aspect sentiment triple extraction model establishment method provided by the present invention, and obtaining a span-level label matrix for the text data to be analyzed;

[0033] The span-level label matrix of the text data to be analyzed is decoded to obtain the span-level aspect sentiment triples in the text data to be analyzed.

[0034] Furthermore, the span-level label matrix of the text data to be analyzed is decoded, including:

[0035] Traverse the span-level label matrix to collect all spans whose category labels are aspect items and all spans whose category labels are opinion items, and determine the starting positions and ending positions of all aspect item spans and the starting positions and ending positions of all opinion item spans;

[0036] Pair the aspect item spans and opinion item spans one by one to obtain all possible aspect item-opinion item pairs;

[0037] Traverse all possible aspect item-opinion item pairs; for each aspect item-opinion item pair traversed, use the token position corresponding to the aspect item span as the row index and the token position corresponding to the opinion item span as the column index, delineate the rectangular area corresponding to the aspect item-opinion item pair in the span-level label matrix, and count the number of sentiment polarities in the rectangular area. The sentiment polarity with the largest number in the statistical results is taken as the sentiment polarity of the corresponding aspect item-opinion item pair;

[0038] After removing the unlabeled aspect item-opinion item pairs, span-level aspect sentiment triplets are constructed for the remaining aspect item-opinion item pairs.

[0039] According to another aspect of the present invention, a computer program product is provided, comprising a computer program; when the computer program is executed by a processor, the method for establishing a domain-adaptive span-level aspect sentiment triple extraction model provided by the present invention is implemented, and / or the method for extracting span-level aspect sentiment triples provided by the present invention is implemented.

[0040] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects:

[0041] (1) The dependency tree is a domain-invariant feature that can serve as a bridge between the source domain and the target domain to identify and understand the commonalities and differences between different domains. After establishing a span-level feature extraction module for extracting span-level features of text data, the present invention uses span-level features in the source domain and target domain data for dependency prediction, and optimizes the span-level feature extraction module based on the dependency prediction results, which can enhance the expression of domain-invariant features by span-level features and effectively achieve domain adaptation. Therefore, the span-level label classification module trained on the source domain can also accurately extract the span classification labels of the target domain text data, and finally realize the extraction of cross-domain span-level sentiment triples without increasing the annotation cost.

[0042] (2) In the preferred embodiment of the present invention, the span-level feature extraction module is further optimized based on the mutual information maximization loss function, thereby realizing the optimization of the span-level feature matrix. Through this step, the span-level feature distribution is further improved, the boundaries between different categories are expanded, and the decision boundaries learned on the source domain data are less likely to overlap with the decision boundaries of the target domain, so that the span-level features can better distinguish different categories, thereby enhancing the domain transfer capability of the model and providing better adaptability for the model when facing a changing data environment.

[0043] (3) The sentences in the sentiment ternary extraction task contain a large amount of syntactic and semantic information. Both types of information are helpful in improving task performance, but existing methods often only focus on semantic information and fail to fully utilize syntactic information. In the preferred embodiment of the present invention, when extracting span-level features of text data, semantic features are first extracted from tokens to obtain token-level features, thereby extracting rich semantic information of the sentence. Next, the part-of-speech tags are partially shielded and predicted using token-level features to enhance the expression of syntactic information by token-level features. Subsequently, for each possible span, its head and tail tokens and length information are selected and combined to further extract span-level features to more comprehensively express semantic and syntactic information. The span-level features finally constructed contain rich semantic and syntactic information at the same time, which can be used for label classification tasks of ASTE sentiment analysis and improve classification accuracy.

[0044] (4) Generally speaking, the fewer the number of categories, the fewer decision boundaries the model needs to learn between categories, and the easier it is for the model to distinguish the differences between different categories, thus reducing the complexity of the classification task. In the preferred embodiment of the present invention, a unified span-level two-dimensional matrix label is designed. This label design controls the number of labels to 6, specifically including aspect items, opinion items, sentiment polarity, and no label. The sentiment polarity is further divided into three types: positive, negative, and neutral. This label design can effectively reduce the complexity of the training process while ensuring the effective extraction of span-level sentiment triples, providing a clearer learning goal for the model.

[0045] (5) While reducing the number of labels and the complexity of model training, the present invention also introduces new challenges. Due to the reduction in the number of effective labels, the balance of label distribution has also changed greatly, facing the challenge of imbalance between positive and negative samples. In the preferred embodiment of the present invention, when optimizing the span-level label classifier, a weighted focus loss function is introduced. This loss function introduces a focus factor on the basis of the traditional cross-entropy loss function. By adjusting the weights of different categories, it focuses on samples that are difficult to classify in a targeted manner, thereby improving the learning effect of the model for minority categories. While effectively alleviating the problem of imbalance between positive and negative samples, it also helps to improve the classification performance of the model for each category.

[0046] (6) In the domain adaptation scenario, each span carries the dual features of domain-invariant and domain-specific. In the preferred embodiment of the present invention, a domain classifier is introduced to obtain the domain distribution of each span, and in the subsequent training of the span-level label classification module, the probability ratio of the span in the target domain and the source domain is used as the weight of each span to achieve dynamic weighting of the span. In this way, the dual features of domain-invariant and domain-specific can be fully utilized. Through this comprehensive optimization strategy, the emotional information in cross-domain data can be captured more accurately, thereby achieving better performance in practical applications.

[0047] (7) In the preferred embodiment of the present invention, in the domain-adaptive span-level sentiment triple extraction model established, when the basic feature extraction module constructs the representation vector of each token, it comprehensively utilizes token embedding, segment embedding and position embedding to achieve multi-level and multi-dimensional representation, which not only enhances the model's semantic understanding of a single token, but also improves the model's ability to grasp the entire text structure, thereby ultimately improving the token-level feature vector's deep understanding and expression of text semantics, inter-sentence relationships and structural information. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic diagram of a method for establishing a domain-adaptive span-level aspect sentiment triple extraction model provided by an embodiment of the present invention;

[0049] Figure 2 A schematic diagram of extracting span-level features by combining syntactic and semantic information according to an embodiment of the present invention;

[0050] Figure 3 A schematic diagram of span-level domain adaptation based on domain-invariant feature enhancement provided by an embodiment of the present invention;

[0051] Figure 4 A schematic diagram of coarse-grained span-level label encoding provided by an embodiment of the present invention;

[0052] Figure 5 A schematic diagram of coarse-grained span-level label decoding provided by an embodiment of the present invention;

[0053] Figure 6 Schematic diagram of optimization of a domain-weighted span-level label classifier provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0055] In the present invention, the terms "first", "second", etc. (if any) in the present invention and the drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0056] The ASTE task requires the annotator to not only identify the aspect items and opinion items in the text, but also to be able to judge the sentiment polarity between the two. Data in these three dimensions are necessary. Due to the complexity of this process and the high professional requirements for the annotator, specific fields often lack labeled data, and the performance of the span-level model is highly dependent on the data annotation of the specific field, which will directly affect the training effect and final performance of the span-level model, resulting in the inability of the existing span-level aspect sentiment triple extraction method to effectively achieve cross-domain prediction. In order to solve this problem, the present invention provides a domain-adapted span-level aspect sentiment triple extraction model establishment method and its application. The overall idea is to propose a domain adaptation method based on domain-invariant feature enhancement, which strengthens the domain-invariant features of the span with the dependency tree as the core, and transfers the features for the specific field to be domain-independent, so as to achieve cross-domain prediction without increasing the annotation cost. On this basis, the mutual information maximization loss function is used to optimize the feature distribution to further enhance the domain independence of the feature.

[0057] In addition, the existing span-level models still have the following problems to be solved:

[0058] (1) Existing span-level models rely too much on semantic information and do not adequately mine and utilize syntactic information. This bias may lead to incomplete analysis results.

[0059] (2) Span-level model label design is complex. The ASTE task consists of three subtasks: aspect item extraction, opinion item extraction, and sentiment polarity identification. The label design scheme of the span-level model needs to include relevant information of each subtask in the text segment. Therefore, a segmented or complex classification label system is often used, which brings additional challenges to model training and prediction.

[0060] To address these issues, this paper further proposes a span-level feature extraction method that combines syntactic and semantic information. This method extracts semantic information through a pre-trained language model and uses part-of-speech tagging to mask the predicted enhancement features for syntactic information, fully expressing both syntactic and semantic information within the span. Furthermore, a streamlined span-level label design scheme and corresponding label encoding and decoding algorithms are proposed to effectively address the high model complexity in domain adaptation scenarios. This approach reduces the computational burden of training while ensuring task effectiveness, providing clearer learning objectives.

[0061] Before explaining the technical solution of the present invention in detail, the following key concepts involved in span-level aspect sentiment triple extraction are explained.

[0062] Token: The basic unit of text processing in natural language processing (NLP) models, which can be a word, sentence, or other meaningful text element, depending on the word segmentation algorithm used.

[0063] Span: A continuous sequence of tokens in a text. In sentiment analysis, span-level interactions can help the model better understand and capture the relationship between aspect items and opinion items.

[0064] The following are examples.

[0065] Example 1:

[0066] A domain-adaptive approach to extracting span-level sentiment triples, such as Figure 1 Shown, including:

[0067] Build a basic feature extraction module; the basic feature extraction module is used to split text data into token sequences through word segmentation and obtain the representation vector of each token in the token sequence;

[0068] Construct a span-level feature extraction module; the span-level feature extraction module is used to extract semantic features from the representation vector of each Token in the text data, obtain the feature vector of each Token, and construct the span-level feature matrix H of the text data Span ; Span-level feature matrix H Span In the ith row and jth column, the element Represents the feature vector of the span with the i-th token as the head vector and the j-th token as the tail vector; the feature vector of the span is extracted based on the feature vector of the first token of the span, the feature vector of the tail token of the span, and the span width; i,j∈{1,2,…N}, N represents the token sequence length of the text data;

[0069] Construct a span-level label classification module; the span-level label classification module is used to classify the span-level feature matrix H of the text data Span Predict the category label of each span and obtain the span-level label matrix; the category labels include: aspect items, opinion items, sentiment polarity and no label.

[0070] After the basic feature extraction module, span-level feature extraction module and span-level label classification module constructed in this embodiment are connected in sequence, a model that can be used to predict the category labels of each span in the text data can be obtained. Based on the span category predicted by the model, the span-level sentiment triples in the text data can be decoded.

[0071] As a preferred embodiment, Figure 1 and Figure 2 As shown, in this embodiment, the token representation vector is formed by connecting token embedding, segment embedding and position embedding;

[0072] Among them, Token embedding represents the index of the token in the vocabulary, and the vocabulary is a set of all tokens in the source domain and target domain text data; segment embedding is used to represent the sentence to which the token belongs; position embedding is used to represent the position of the token in the entire token sequence.

[0073] The token representation vector constructed in this embodiment realizes multi-level and multi-dimensional representation, which not only enhances the model's semantic understanding of a single token, but also improves the model's ability to grasp the entire text structure, ultimately improving the token-level feature vector's deep understanding and expression of text semantics, inter-sentence relationships, and structural information.

[0074] Optionally, in this embodiment, a pre-trained language assurance model BERT (Bidirectional Encoder Representations from Transformers) is specifically used to extract the feature vector of the Token.

[0075] In this embodiment, the feature vector of any span It mainly includes three parts, namely the feature vector of the first Token The characteristic vector of the tail token The three parts work together to provide a comprehensive feature representation for the span. As an optional implementation, this embodiment connects the three parts and uses a fully connected layer to obtain the corresponding feature vector. The calculation formula is as follows:

[0076]

[0077] Among them, W Span and b Span are the weight matrix and bias vector for feature learning respectively.

[0078] In order to achieve cross-domain prediction without increasing the annotation cost, this embodiment proposes a domain adaptation method based on domain-invariant feature enhancement, which uses the dependency tree as the core to enhance the domain-invariant features of the span and transfer the features for a specific domain to domain-independent features. Figure 1 and Figure 3 As shown, this embodiment first extracts the dependency tree of text data through preprocessing as tag data TAG dep Then, the source domain with the span-level sentiment triples annotated and the target domain text data without the span-level sentiment triples annotated are used to perform domain-invariant feature enhancement to optimize the span-level feature extraction module; the domain-invariant feature enhancement includes: extracting the span-level feature matrix H according to the span-level feature extraction module Span Predict the dependency tree of text data. Specifically, input the span-level feature matrix into the dependency classifier and use the softmax function to perform dependency classification prediction. The distribution probability of the dependency is as follows:

[0079]

[0080] in, Basis Predicted dependency distribution probability; W dep and b dep are the weight matrix and bias vector of the dependency classifier respectively;

[0081] A loss function is constructed based on the prediction results of the dependency tree and the label data to optimize the span-level feature extraction module.

[0082] Optionally, this embodiment adds a custom tag "sub" to the dependency library to represent the dependency relationship between tokens within the same word, thereby maintaining the integrity and semantic coherence of the word. Specifically, when obtaining the true dependency relationship label, if a word is split into multiple tokens, a dependency relationship named "sub" is established between the first token and subsequent tokens, where the first token is the core word and the subsequent tokens are dependent words, and the first token retains the original dependency relationship related to the word.

[0083] In order to improve the accuracy of the model, the method uses the cross entropy loss function for optimization, as follows:

[0084]

[0085] Among them, Ldep represents the cross entropy loss function constructed based on the prediction results of the dependency tree and the label data; D represents the dataset consisting of the source domain and target domain text data; The dependency labels obtained during data preprocessing represent the true dependency labels, with the i-th token being the core word and the j-th token being the dependent word. Since the model only considers dependencies between tokens that are directly connected in the dependency tree for training, D(i,j) is added to the loss function to indicate whether the i-th and j-th tokens are directly connected. If they are directly connected, D(i,j) is 1; if not, D(i,j) is 0.

[0086] The dependency tree is a domain-invariant feature that can serve as a bridge between the source domain and the target domain to identify and understand the commonalities and differences between different fields. After establishing a span-level feature extraction module for extracting span-level features from text data, this embodiment uses span-level features to predict dependencies in the source and target domain data, and optimizes the span-level feature extraction module based on the dependency prediction results. This can enhance the expression of domain-invariant features by span-level features and effectively achieve domain adaptation.

[0087] In order to further enhance the domain transfer capability of the model and provide better adaptability for the model in the face of changing data environments, such as Figure 1 and Figure 3 As shown, this embodiment further introduces a mutual information maximization loss function in domain-invariant feature enhancement.

[0088] In information theory, mutual information (MI) is an extremely important concept used to measure the mutual dependence between two random variables. The mutual information maximization loss function can ensure that the model does not favor any specific category, while helping to retain more relevant label information and prompting the model to learn more discriminative feature representations, thereby better understanding the relationship between features and labels, increasing the boundaries between categories, and improving the confidence of classification results.

[0089] The calculation of label classification information entropy requires the distribution probability of the label category. The method approximates the distribution probability of the label category with the average label prediction probability of the feature as follows:

[0090] P(y)=avg(p y ) Among them, p y Indicates the probability that the span is predicted to have the category label y, and avg indicates calculating the average value of a batch;

[0091] Based on this, the information entropy calculation formula for label classification is as follows:

[0092]

[0093] For the case where the average label prediction probability P(y) is 0, it is stipulated that 0 log(0) = 0. Similarly, for the conditional entropy of label classification under the condition of a given feature, the model can also be approximated by the label prediction probability of the feature, as follows:

[0094]

[0095] For p y When 0 is 0, it is stipulated that 0 log(0) = 0. According to the mathematical expression of mutual information, mutual information can be calculated as L1-L2. The mutual information maximization loss function is the negative mutual information. The loss function is defined as:

[0096]

[0097] Here, ρ is a hyperparameter threshold. Maximizing L1 in the mutual information maximization loss function described above helps ensure that the model is not biased towards any particular category, thereby avoiding the class collapse problem. This is achieved by introducing a constraint in the loss function, namely, maximizing L1 is only considered when the value of L1 is less than the preset threshold ρ. In this way, the model is encouraged to explore more classification possibilities during training, rather than focusing solely on a single, frequently occurring category. In addition, by minimizing another loss function, L2, the model is encouraged to be more confident when making classification decisions, thereby reducing the uncertainty of the classification results. This approach enables the model to further expand the boundaries between different categories, making the decision boundaries learned on the source domain data more likely to overlap with the decision boundaries of the target domain, which is conducive to domain transfer.

[0098] Considering that sentences in the aspect sentiment ternary extraction task contain a large amount of syntactic and semantic information, both of which are helpful in improving task performance, in order to utilize both syntactic and semantic information, as a preferred implementation, this embodiment further includes:

[0099] Through preprocessing, each token in the Token sequence of the text data is tagged with part of speech as the tag data TAG of the part of speech POS ; Moreover, this embodiment further includes: performing token-level feature enhancement using source domain and target domain text data to optimize the span-level feature extraction module;

[0100] Token-level feature enhancements include:

[0101] Randomly mask some tokens in the token sequence of text data; optionally, this embodiment randomly selects about 25% of the tokens and replaces the original tokens and the corresponding part-of-speech tags with a special mark "[Mask]"; in addition, when performing the [Mask] masking operation, it is necessary to consider the impact of word segmentation. Therefore, in this embodiment, if a token is masked, then other tokens belonging to the same word will also be masked; the representation vector of the masked token is represented as E Mask ;

[0102] After the shielding process is completed, the representation vector of each token is input into the BERT model to obtain the feature vector of each token and the token-level features of the text data. The Token-level feature H of the text data Token Input the pre-trained part-of-speech tagging classifier to predict the part-of-speech distribution probability of each token as follows:

[0103]

[0104] Among them, W POS and b POS are the weight matrix and bias vector of the part-of-speech tagging classifier respectively.

[0105] After obtaining the probability distribution of the part-of-speech tags, the model compares the predicted results with the actual part-of-speech tags that have been masked to optimize the prediction effect. This step is accomplished using the cross-entropy loss function, which is a common method for evaluating the difference between the predicted probability distribution and the true distribution. Optimizing the cross-entropy loss function helps adjust the model parameters, thereby reducing prediction errors and improving the model's accuracy in the part-of-speech tagging classification task. The specific formula for this optimization objective is as follows:

[0106]

[0107] Where, M(·) is the shielding indicator function; The true part-of-speech tag sequence obtained during data preprocessing represents the true part-of-speech tag for the i-th token. Since the model only considers part-of-speech tags masked by [Mask] for training, M(i) is added to the loss function to indicate whether the i-th token is masked by [Mask]. If the i-th token is masked, the value of M(i) is 1; otherwise, if the i-th token is not masked, the value of M(i) is 0.

[0108] The method for optimizing the span-level classification module in this embodiment includes:

[0109] The constructed span-level label classification module is used to predict the category label of each span in the source domain text data, and the span-level aspect sentiment triples in the source domain text data are decoded. A loss function is constructed based on the prediction results and labeling results of the span-level aspect sentiment triples to optimize the span-level classification module.

[0110] Generally speaking, the fewer the number of categories, the fewer decision boundaries between categories the model needs to learn, and the easier it is for the model to distinguish the differences between different categories, thereby reducing the complexity of the classification task. As a preferred implementation method, this embodiment designs a unified span-level two-dimensional matrix label. The label design controls the number of labels to 6, specifically including aspect items, opinion items, sentiment polarity and no labels, where sentiment polarity is further divided into three types: positive, negative and neutral, as shown in Table 1, where "A" and "O" represent aspect items and opinion items, respectively, "POS", "NEU" and "NEG" represent positive sentiment polarity, neutral sentiment polarity and negative sentiment polarity, respectively, and "N" represents no label.

[0111] Specifically, when the span-level label matrix predicted by the span-level label classification module has the label Tag i,j ∈{"A","O"}, it indicates that the span with the i-th Token as the first Token and the j-th Token as the last Token in the text constitutes a complete aspect item or opinion item; when the tag Tag i,j ∈{"POS","NEU","NEG"}, it indicates that the aspect item with subscript i and the opinion item with subscript j in the span form an aspect-opinion item pair, and the corresponding sentiment polarity is positive, negative or neutral.

[0112] Table 1 Coarse-grained span-level ASTE label meaning

[0113]

[0114] This embodiment needs to encode the source domain text data annotated with span-level sentiment triples into a corresponding span-level label matrix. Corresponding to the classification method shown in Table 1, the label encoding process can be described by the pseudo code shown in Table 2:

[0115] Table 2 Pseudocode of label encoding algorithm

[0116]

[0117]

[0118] During encoding, the first step is to identify the position of the aspect item within a given triple in the sentence. This step scans the sentence and identifies phrases that match the aspect item. Once the aspect item's position is determined, the algorithm records its starting and ending positions within the sentence and converts this position information into corresponding coordinate information. Specifically, the starting and ending subscripts of the aspect item serve as the x- and y-axis coordinates of the label, respectively, and the span label is designated "A," indicating that it is an aspect item.

[0119] Next, identify the opinion item in the sentence, find its position in the sentence, and determine the starting and ending subscripts of the opinion item in the sentence. The starting and ending subscripts of the opinion item are used as the x-axis and y-axis coordinates of the label, respectively. The span label is assigned an "O" to indicate that it is an opinion item.

[0120] Finally, based on the sentiment contained in the given triples, we perform sentiment annotation on the regions corresponding to the aspect items and opinion items. Specifically, we use the subscripts of the aspect items as the x-axis coordinates, the subscripts of the opinion items as the y-axis coordinates, and assign corresponding sentiment labels to the spans within these coordinate regions: "POS" for positive sentiment, "NEU" for neutral sentiment, and "NEG" for negative sentiment.

[0121] Figure 4 As shown in the figure, the actual encoding effect of the above encoding method in the example sentence "The Thai food was great but the service was poor" is obtained. First, the sentiment triple <"Thai food", "great", "positive"> is processed. For the aspect item "Thai food", the sentence is scanned to determine that its starting subscript in the sentence is 1 and its ending subscript is 2. For the opinion item "great", its subscript in the sentence is identified as 4. Then, the algorithm sets the span label label[1][2] to "A", indicating the aspect item, and sets the span label label[4][4] to "O", indicating the opinion item. For "positive", the corresponding areas of the aspect item and the opinion item are located, and the span labels label[1][4] and label[2][4] are set to "POS", indicating a positive sentiment tendency. Then, for the triple <"service", "poor", "negative">, similar steps are performed, and the span label label[7][7] is set to "A", the span label label[9][9] is set to "O", and the span label label[7][9] is set to "NEG".

[0122] Correspondingly, the decoding process based on the span-level label matrix in this embodiment can be described by the pseudo code shown in Table 3:

[0123] Table 3 Pseudocode of label decoding algorithm

[0124]

[0125] During decoding, the label matrix is ​​first traversed. This matrix stores the predicted results of span-level sentiment analysis of the test text, including the location information of aspect and opinion items. During this traversal, the algorithm aims to collect all predicted aspect and opinion items and record their starting and ending subscripts. This process ensures that each aspect and opinion item is accurately located in the original text in subsequent steps and determines the query scope for predicting sentiment.

[0126] Next, we traverse all possible aspect-opinion pairs. We count the number of sentiment labels within the entire rectangular area corresponding to each aspect-opinion pair. The x-axis subscript of the rectangular area corresponds to the aspect subscript, and the y-axis subscript of the rectangular area corresponds to the opinion subscript. During the statistical process, we count the three sentiment labels "POS," "NEU," and "NEG." After the statistics are complete, the sentiment label with the highest frequency is used as the predicted sentiment for that aspect-opinion pair.

[0127] Finally, based on the start and end subscripts of the aspect and opinion items obtained initially, the algorithm extracts the specific content of the aspect and opinion items from the original text and combines them with the predicted sentiment tendency to form a complete sentiment triple.

[0128] Figure 5 As shown, according to the above decoding method Figure 4 An example of decoding the span-level label matrix shown in Figure 1 is shown. First, the aspect items "Thai food" and "service" and the sentiment items "great" and "poor" are extracted based on the predicted label matrix. All possible aspect-opinion item pairs are traversed, and the corresponding rectangular areas in the label matrix are scanned. The number of sentiment labels within each pair is counted. If a sentiment label exists, the sentiment label with the largest number is used as the corresponding sentiment polarity. If no sentiment label exists, the aspect-opinion item pair is discarded. For example, if the rectangular area corresponding to "Thai food" and "great" contains 2 "POS" labels and 0 "NEU" and "NEG" labels, then the aspect-opinion item pair is valid and has a "positive" sentiment polarity. If the rectangular area corresponding to "Thai food" and "poor" contains no sentiment label, then the aspect-opinion item pair is discarded. Two sentiment triplets can be decoded: <"Thai food", "great", "positive"> and <"service", "poor", "negative">.

[0129] During decoding, this embodiment pairs the aspect items and opinion items extracted from the span-level label matrix one by one, identifies possible sentiment tendencies, and uses the more numerous sentiment tendencies as part of the sentiment triple extraction results. This label decoding approach extracts as many valid sentiment triplets as possible, effectively mitigates the common problem of sentiment misclassification, and improves the diversity and accuracy of sentiment triple extraction.

[0130] Furthermore, this embodiment takes into account that while reducing the number of labels reduces the complexity of model training, it also introduces new challenges. Due to the reduction in the number of effective labels, the balance of label distribution has also changed greatly, facing the challenge of imbalance between positive and negative samples. In order to meet this challenge, this embodiment optimizes the span-level label classification model through training. The loss function constructed is a weighted focus loss function. This loss function introduces a focus factor on the basis of the traditional cross-entropy loss function. By adjusting the weights of different categories, it focuses on samples that are difficult to classify in a targeted manner, thereby improving the model's learning effect on minority categories. While effectively alleviating the problem of imbalance between positive and negative samples, it also helps to improve the model's classification performance for each category. Figure 6 As shown, this embodiment further introduces a pre-trained domain classifier, and inputs the span-level feature matrix extracted by the span-level feature extraction module into the domain classifier to obtain the domain distribution probability of the corresponding span as follows:

[0131]

[0132] Among them, W d and b d are the weight matrix and bias vector of the domain classifier, respectively.

[0133] The domain distribution probability predicted by the domain classifier will be used to dynamically adjust the weight of each span in the weighted focal loss function. Specifically, the ratio of the target domain probability to the source domain probability of the span is calculated, that is, And use it as the weight of the corresponding span. In this way, the model can dynamically adjust the weight of the span so that the span that is more important in the target domain can play a greater role in span-level sentiment label classification.

[0134] Optionally, in this embodiment, the domain classifier is optimized using a cross entropy loss function during training. The corresponding loss function calculation formula is as follows:

[0135]

[0136] In order to avoid the cross-domain generalization problem during the domain classifier training process, in this embodiment, the back propagation of the domain classifier is cut off so that L dOnly the weight matrix and bias vector in the domain classifier are optimized without affecting the training of other parts. This approach not only improves the adaptability of the model to the target domain, but also ensures the accuracy in capturing key features and information, thereby achieving better performance on the main task, namely the ASTE task.

[0137] Based on the above weights, in this embodiment, the weighted focal loss function constructed for the span-level label classifier is expressed as follows:

[0138]

[0139] Among them, FL represents the focal loss function, which adds a modulation factor on the basis of the cross entropy loss function, reducing the weight of easy-to-classify samples in the loss function; is the span label predicted by the span-level label classifier, The true span-level classification labels are obtained after the true emotion triples are processed by the label encoding algorithm. Through this comprehensive optimization strategy, this embodiment can more accurately capture the emotion information in cross-domain data, thereby achieving better performance in practical applications.

[0140] To further refine the span-level features, this embodiment introduces a fully connected layer before inputting the span-level features into the span-level label classifier for classification. This layer transforms the features of each span to generate a more discriminative feature representation for each category, as follows:

[0141]

[0142] After that, the category features of the span after processing by the fully connected layer are input into the span-level label classifier to obtain the label distribution probability of each category of the span, as follows:

[0143]

[0144] During the test phase, the label distribution probabilities of each category in the span are compared, and the label with the highest probability is used as the prediction result for the span, as follows:

[0145]

[0146] In general, this embodiment proposes a span-level domain adaptation method based on domain-invariant feature enhancement and distribution optimization, which strengthens the domain-invariant features of the span with the dependency tree as the core, and optimizes the feature distribution using the mutual information maximization loss function, so as to transfer the features for a specific domain to be domain-independent, and achieve cross-domain prediction without increasing the annotation cost. On this basis, a span-level feature extraction method combining syntactic and semantic information is proposed, which extracts semantic information through a pre-trained language model, and strengthens the expression of syntactic information through part-of-speech tagging to mask the prediction of the enhanced features, so that both syntactic and semantic information in the span are fully expressed. In addition, a unified and streamlined span-level label design scheme is proposed, and a corresponding label encoding and decoding algorithm is designed to effectively deal with the problem of high model complexity in domain adaptation scenarios, reduce the computational burden in the training process while ensuring the effectiveness of the task, and provide clearer learning goals.

[0147] Example 2:

[0148] A span-level aspect sentiment triple extraction method, comprising:

[0149] Inputting the text data to be analyzed in the target domain into the domain-adapted span-level aspect sentiment triple extraction model established by the domain-adapted span-level aspect sentiment triple extraction model establishment method provided in the above embodiment 1 to obtain a span-level label matrix for the text data to be analyzed;

[0150] The span-level label matrix of the text data to be analyzed is decoded to obtain the span-level aspect sentiment triples in the text data to be analyzed.

[0151] In this embodiment, decoding the span-level label matrix of the text data to be analyzed includes:

[0152] Traverse the span-level label matrix to collect all spans whose category labels are aspect items and all spans whose category labels are opinion items, and determine the starting positions and ending positions of all aspect item spans and the starting positions and ending positions of all opinion item spans;

[0153] Pair the aspect item spans and opinion item spans one by one to obtain all possible aspect item-opinion item pairs;

[0154] Traverse all possible aspect item-opinion item pairs; for each aspect item-opinion item pair traversed, use the token position corresponding to the aspect item span as the row index and the token position corresponding to the opinion item span as the column index, delineate the rectangular area corresponding to the aspect item-opinion item pair in the span-level label matrix, and count the number of sentiment polarities in the rectangular area. The sentiment polarity with the largest number in the statistical results is taken as the sentiment polarity of the corresponding aspect item-opinion item pair;

[0155] After removing the unlabeled aspect item-opinion item pairs, span-level aspect sentiment triplets are constructed for the remaining aspect item-opinion item pairs.

[0156] Comparative experiments were conducted on six source-target domains consisting of four datasets from the International Semantic Evaluation Conference. The experimental results show that compared with the average optimal baseline method, the method proposed in this embodiment improves the F1 value evaluation standard by 3.99%, and also improves the precision and recall rates.

[0157] Example 3:

[0158] A computer program product includes a computer program; when the computer program is executed by a processor, it implements the domain-adaptive span-level aspect emotion triple extraction model establishment method provided in the above-mentioned embodiment 1, and / or the span-level aspect emotion triple extraction method provided in the above-mentioned embodiment 2.

[0159] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A domain-adaptive span-level sentiment triple extraction model establishment method, characterized by: include: Build basic feature extraction module; The basic feature extraction module is used to split the text data into a token sequence through word segmentation processing and obtain the representation vector of each token in the token sequence; Construct a span-level feature extraction module; the span-level feature extraction module is used to extract semantic features from the representation vector of each Token in the text data, obtain the feature vector of each Token, and construct the span-level feature matrix H of the text data Span ; The span-level feature matrix H Span In the ith row and jth column, the element Represents the feature vector of the span with the i-th token as the head vector and the j-th token as the tail vector; the feature vector of the span is extracted based on the feature vector of the first token of the span, the feature vector of the tail token of the span, and the span width; i,j∈{1,2,…N}, N represents the token sequence length of the text data; performing domain-invariant feature enhancement using source domain text data annotated with span-level aspect sentiment triples and target domain text data not annotated with span-level aspect sentiment triples to optimize the span-level feature extraction module; The domain-invariant feature enhancement includes: predicting the dependency tree of text data as label data of the dependency tree; extracting the span-level feature matrix H according to the span-level feature extraction module; Span Predicting a dependency tree of text data, and constructing a loss function based on the prediction results of the dependency tree and the label data to optimize the span-level feature extraction module; Construct a span-level label classification module; the span-level label classification module is used to classify the span-level feature matrix H of the text data according to the span-level feature matrix H of the text data. Span Predict the category label of each span and obtain the span-level label matrix; the category labels include: aspect item, opinion item, sentiment polarity and no label; Using the constructed span-level label classification module to predict the category label of each span in the source domain text data, and decoding to obtain the span-level aspect sentiment triples in the source domain text data, constructing a loss function based on the prediction results and labeling results of the span-level aspect sentiment triples to optimize the span-level classification module; The basic feature extraction module, the optimized span-level feature extraction module and the optimized span-level label classification module are connected in sequence to obtain the domain-adapted span-level aspect emotion triplet extraction model.

2. The domain-adaptive span-level sentiment triple extraction model establishment method according to claim 1, characterized in that: The domain-invariant feature enhancement further includes: constructing a mutual information maximization loss function for optimizing the span-level feature extraction module; the mutual information maximization loss function is: Among them, L mi represents the mutual information maximization loss function; Y represents the set of category labels of the span; p y represents the probability that the span is predicted to have the category label y, avg represents the calculated average value, P(y) = avg(p y ); ρ represents the hyperparameter threshold.

3. The domain-adaptive span-level sentiment triple extraction model establishment method according to claim 1, characterized in that: include: Performing token-level feature enhancement using source and target domain text data to optimize the span-level feature extraction module; The token-level feature enhancements include: Perform part-of-speech tagging on each token in the token sequence of the text data as label data for part-of-speech tagging; After randomly masking some tokens in the token sequence of the text data, the span-level feature extraction module is used to extract the feature vector of each token, and the part of speech of each token is predicted based on the feature vector of each token; A loss function is constructed based on the part-of-speech prediction results and label data to optimize the span-level feature extraction module.

4. The domain-adaptive span-level sentiment triple extraction model establishment method according to claim 1, characterized in that: The sentiment polarity is divided into three categories: positive, negative and neutral.

5. The method for establishing a domain-adaptive span-level aspect sentiment triple extraction model according to claim 4, wherein: The loss function constructed based on the prediction results and labeling results of span-level sentiment triples is a weighted focal loss function.

6. The method for establishing a domain-adaptive span-level aspect sentiment triple extraction model according to claim 5, wherein: In the weighted focal loss function, the weight of each span is determined as follows: The feature vector of the span Input the pre-trained domain classifier to perform domain classification and obtain the probability that the span belongs to the source domain and the probability that the span belongs to the target domain calculate as the weight of the corresponding span.

7. The method for establishing a domain-adaptive span-level aspect sentiment triple extraction model according to any one of claims 1 to 6, characterized in that: The token representation vector is formed by concatenating the token embedding, segment embedding, and position embedding; Among them, Token embedding represents the index corresponding to the Token in the vocabulary, and the vocabulary is a set of all Tokens in the source domain and target domain text data; the segment embedding is used to represent the sentence to which the Token belongs; and the position embedding is used to represent the position of the Token in the entire Token sequence.

8. A span-level aspect sentiment triple extraction method, characterized by: include: Inputting the text data to be analyzed in the target domain into the domain-adapted span-level aspect sentiment triple extraction model established by the domain-adapted span-level aspect sentiment triple extraction model establishment method according to any one of claims 1 to 7, and obtaining a span-level label matrix of the text data to be analyzed; The span-level label matrix of the text data to be analyzed is decoded to obtain span-level aspect sentiment triples in the text data to be analyzed.

9. The method for extracting span-level aspect sentiment triples according to claim 8, wherein: Decoding the span-level label matrix of the text data to be analyzed includes: Traverse the span-level label matrix to collect all spans whose category labels are aspect items and all spans whose category labels are opinion items, and determine the starting positions and ending positions of all aspect item spans and the starting positions and ending positions of all opinion item spans; Pair the aspect item spans and opinion item spans one by one to obtain all possible aspect item-opinion item pairs; Traverse all possible aspect item-opinion item pairs; for each aspect item-opinion item pair traversed, use the token position corresponding to the aspect item span as the row index and the token position corresponding to the opinion item span as the column index, delineate the rectangular area corresponding to the aspect item-opinion item pair in the span-level label matrix, and count the number of sentiment polarities in the rectangular area. The sentiment polarity with the largest number in the statistical results is taken as the sentiment polarity of the corresponding aspect item-opinion item pair; After removing the unlabeled aspect item-opinion item pairs, span-level aspect sentiment triplets are constructed for the remaining aspect item-opinion item pairs.

10. A computer program product, characterized in that It includes a computer program; when the computer program is executed by a processor, it implements the domain-adaptive span-level aspect emotion triple extraction model establishment method described in any one of claims 1 to 7, and / or the span-level aspect emotion triple extraction method described in claim 8 or 9.

Images