Aspect-level emotion triple extraction method based on diffusion model

Abstract

The invention relates to the technical field of sentiment analysis, and provides an aspect-level sentiment triple extraction method based on a diffusion model. The method aims at solving the problem that in an existing generative model, only generation of a single word is concerned during autoregression decoding, and the ability of the model to utilize overall semantics when the model processes multi-word aspect / opinion terms is limited, the method is based on a non-autoregression diffusion model framework, an aspect-level emotion triple task is defined as a boundary diffusion denoising process, and an aspect-level emotion triple task is defined as a boundary diffusion denoising process. According to the method, boundary indexes are directly modeled, boundaries in a noise state are gradually refined and dynamically adjusted according to comprehensive context information, and a contrast denoising training strategy is introduced, so that repeated prediction with subtle changes introduced in a diffusion process is effectively relieved, and aspect-level emotion triple extraction performance is improved. The method is used for sentiment analysis, and especially when the method is used for processing user sentiment expression on social media and an online comment platform, sentiment information in a text can be mined more meticulously, and reference is provided for product optimization and enterprise decision making.

Description

Technical Field

[0001] The present invention relates to the technical field of sentiment analysis, and in particular to an aspect-level sentiment triple extraction method based on a diffusion model. Background Art

[0002] With the rapid development of social media and online review platforms, the ways in which users express their opinions and emotions on the Internet have become more diverse. In addition to the analysis of overall sentiment, in-depth mining of fine-grained sentiment information in text has become an important research topic. In this context, aspect-level sentiment triple extraction has gradually become a key technology in the field of natural language processing. Aspect-level sentiment triple extraction aims to simultaneously identify aspect words, related opinion words, and corresponding sentiment polarity (triplets: aspect words, opinion words, sentiment polarity) from text. For example, in a product review, a user may mention that "the screen display effect is very clear", where "screen" is an aspect word and "clear" is an opinion word, which expresses positive sentiment polarity as a whole. By extracting these triple information, companies can understand users' specific opinions on different product attributes in more detail, providing valuable reference for product optimization. This technology not only requires the model to have the ability to identify aspect words, but also to accurately locate the relevant opinion expressions and judge the sentiment polarity. Compared with traditional sentiment classification tasks, aspect-level sentiment triple extraction requires text analysis at a fine-grained level, so it is more complex and challenging.

[0003] Furthermore, existing studies mostly use autoregressive Seq2Seq generative models. However, this autoregressive approach only focuses on the generation of a single word in each decoding step, limiting the model's ability to utilize overall semantics when processing multi-word aspect / opinion terms. Therefore, this patent proposes an aspect-level sentiment triple extraction method based on a diffusion model, which significantly improves the performance of the aspect-level sentiment triple extraction model based on a non-autoregressive diffusion model framework and a contrastive denoising training strategy. Summary of the invention

[0004] The purpose of the present invention is to solve the problem that the existing generative model only focuses on the generation of a single word during autoregressive decoding, which limits the model's ability to utilize overall semantics when processing multi-word aspect / opinion terms. Therefore, based on the non-autoregressive diffusion model framework, the present invention defines the aspect-level sentiment triple extraction task as a boundary denoising diffusion process, and directly models the boundary index, gradually refines and dynamically adjusts the boundary under the noise state according to comprehensive contextual information. In addition, a contrast denoising training strategy is introduced, which effectively alleviates the repeated predictions with subtle changes introduced by the diffusion process, thereby solving the problems mentioned in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] The present invention provides an aspect-level sentiment triple extraction method based on a diffusion model, comprising the following steps:

[0007] Step S10: For the boundary sequence T b Gaussian noise is introduced to simulate the uncertainty in identifying term boundaries and obtain the noise sequence X t ;

[0008] Step S20: In the denoising network f θ (X t , S, t i ), using the noise boundary sequence X t and sentence S as input, and then predict the corresponding prediction boundary sequence

[0009] Step S30: the sentiment classifier processes the sequence representation through a feed-forward neural network FFN to output a probability distribution of sentiment categories;

[0010] Step S40: generating positive samples and negative samples by comparing denoising training strategies, improving the accuracy in refining term boundaries, and strengthening the sentiment classification process by reducing the generation of erroneous triples;

[0011] Step S50: define the loss function as matching loss and contrast denoising loss, and update the model parameters;

[0012] Step S60: In the inference phase, a noise sequence is randomly sampled from a Gaussian distribution, and then iteratively denoised using a back-diffusion process with learned boundary indices. The predicted probabilities derived from the denoising process correspond to the predicted probabilities associated with the boundary indices and their sentiment polarities.

[0013] In the above scheme, the specific steps of step S10 are as follows:

[0014] Step S101, the boundary index forward diffusion stage, gradually introduces Gaussian noise into the boundary sequence to simulate the inherent uncertainty in identifying term boundaries. For parallel training, The number N is normalized to N train , the standardized sequence is expressed as The noise sequence X at any given time step t is t The calculation using a single-step Markov transition is as follows:

[0015]

[0016] in is the cumulative noise retention ratio, representing all α from time step 0 to t s The continuous product of β s represents a predefined variance table, ∈ is a random vector, each element of which is a sample drawn from a normal distribution, represents the noise sampled from a standard Gaussian distribution, where α s Indicates the noise retention ratio at the s-th time step;

[0017] Step S102: From the noise state X t Initially, the back-diffusion process uses a non-Markov denoising strategy DDIM, which is used to accurately reconstruct term boundaries. The process involves selecting a subsequence τ of length γ from the full time step sequence [1, …, T] and using the predicted boundary sequence of the previous time step and the predicted noise Iteratively refine the boundary sequence Iterative refinement process, using a trainable denoising network conditioned on S As shown below:

[0018]

[0019]

[0020] in Indicates that at the τth i The noise retention ratio of time steps, represents the time step τ i The predicted boundary sequence of Indicates that at the τth i time-step noise boundary sequence, Indicates that at the τth i The noise of the predictions for time steps is determined as the noise boundary sequence and the prediction boundary sequence The normalized difference between and Combined and adjusted according to their respective standard deviations, this process is repeated iteratively, as shown in the expression:

[0021]

[0022] Indicates that at the τth i-1 The estimated value of the boundary sequence obtained in the back diffusion process of time steps;

[0023] Indicates that at the τth i-1 The noise retention ratio of time steps;

[0024] After γ iterations of DDIM, the noise boundary index is gradually refined and converges to the accurate boundary index.

[0025] In the above scheme, the specific steps of step S20 are as follows:

[0026] Step S201: The encoder converts an input sentence S = {w1, w1, ..., w M}Converted to h-dimensional sentence representation Implemented as a pre-trained language model PLM with bidirectional LSTM;

[0027] H s =BiLSTM(BERT(S))

[0028] BERT(*) indicates the use of a bidirectional encoder to represent the processing of the input sentence S from the converter BERT, and BiLSTM indicates a bidirectional long short-term memory network;

[0029] Step S202: The task of the decoder is to process the sentence representation H s , to obtain the noise boundary index sequence X t The semantic representation of , which represents the aspect and opinion items;

[0030] First, the noise sequence is discretized into word indices by rescaling, then the sequence is represented as It is calculated by mean pooling the tokens at the specified start and end indices of the aspect and opinion items, each represents the merged representation of the i-th sequence within the boundary sequence, which is calculated as follows:

[0031]

[0032] in, Indicates the starting index of the aspect item The hidden state of Indicates the end index of the aspect item The hidden state of This indicates the start index of the view item The hidden state of This indicates the start index of the view item The hidden state at; Pooling(*) represents the pooling operation;

[0033] The Transformer decoder with integrated self-attention and cross-attention layers is used to decode the refined sequence representation. The self-attention module uses the X The generated queries, keys, and values ​​facilitate interaction between sequences:

[0034] H sa =SelfAttention(H X )

[0035] Among them, SelfAttention(*) represents the attention mechanism, At the same time, the cross-attention mechanism further refines the sequence representation by incorporating the broader semantic context of the sentence, by utilizing the self-attention module H sa The output of is implemented as a query, with keys and values ​​coming from the sentence representation H s , expressed as:

[0036] H ca =CrossAttention(H sa , H s )

[0037] Among them, CrossAttention(*) represents the cross attention mechanism. To accommodate the iterative nature of the diffusion process, the sinusoidal code E corresponding to each time step t is t is integrated into the sequence representation, and the final noisy sequence representation The calculation is as follows:

[0038]

[0039] Four index pointers are used to predict the boundary indexes of aspect and opinion items respectively. For each index δ∈{a s ,a e ,o s , o e}, generate a fusion representation It combines the noisy sequence representation with the sentence representation, and each index is predicted to be the probability of a term boundary. as follows:

[0040]

[0041]

[0042] in is a learnable matrix, FFN(*) represents a feedforward network, It is a type code used to distinguish aspects or viewpoint boundaries.

[0043] In the above scheme, the specific steps of step S30 are as follows:

[0044] Step S301: The sentiment classifier processes the sequence representation through FFN The probability distribution of the output emotion category is expressed as:

[0045]

[0046] in, C represents the total number of sentiment polarity categories.

[0047] In the above solution, the denoising training strategy is compared in step S40:

[0048] By adding two different levels of noise λ1 and λ2 to N train Two types of samples, positive samples and negative samples, are generated from the true values, where λ1<λ2. After diffusion back-processing, the decoder takes two types of samples as input. The noise scale of the positive sample is less than λ1 and is used to reconstruct its corresponding true value. The noise scale of the negative sample is greater than λ1 and less than λ2, and is used to predict the sentiment category "Invalid" (invalid triplet) when the predicted aspect word does not match the opinion word, denoted as ε. If a sentence has N train The true value, compared to the denoising training, will have 2×N train samples, each true value generates a positive sample and a negative sample;

[0049] The positive sample X + and negative samples X - Input decoder, the specific calculation is to first pass self-attention and cross-attention:

[0050] H sa+ =SelfAttention(X + )

[0051] H sa- =SelfAttention(X - )

[0052] H ca+ =CrossAttention(H sa+ , H s )

[0053] H ca- =CrossAttention(H sa- , H s )

[0054] Among them, SelfAttention(*) represents the attention mechanism, CrossAttention(*) represents the cross attention mechanism, H s is the sentence representation. In order to adapt to the iterative nature of the diffusion process, the sinusoidal code E corresponding to each time step t is t is integrated into the sequence representation, and the final noisy sequence representation and The calculation is as follows:

[0055]

[0056]

[0057] Four index pointers are used to predict the boundary indexes of aspect and opinion items respectively. For each index δ∈{a s ,a e , o s , o e}, generate fusion representation It combines the noise sequence representation with the sentence representation to obtain the boundary probability of positive samples

[0058]

[0059]

[0060] in is a learnable matrix, FFN(*) represents a feedforward network, It is a type code used to distinguish aspects or viewpoint boundaries.

[0061] The classification probabilities of positive samples and negative samples calculated by the sentiment classifier are expressed as and

[0062]

[0063]

[0064] in, C represents the total number of sentiment polarity categories.

[0065] In the above scheme, the specific steps of step S50 are as follows:

[0066] Step S501, the training loss function includes matching loss and contrast denoising loss;

[0067] Matching loss: When processing N train predictions and the corresponding N train When the true value is expanded, the Hungarian algorithm is used to establish the best match between the predicted value set and the true value set. Represents the true situation corresponding to the i-th noise sequence. The matching loss includes boundary loss and sentiment classification loss. Subsequently, the inverse process is trained by maximizing the possibility of prediction:

[0068]

[0069] δ∈{a s , a e , o s , o e} indicates that this represents the boundary index of aspect items and opinion items, where a s and a e Respectively represent the start and end index of the aspect item, and o s and e Respectively represent the start and end index of the opinion item;

[0070] Indicates that during the matching process, for the i-th noise sequence, the true boundary index predicted by the model The probability of is the true boundary index corresponding to the i-th noise sequence determined during the matching process, is the probability distribution of the boundary index δ predicted by the model for the i-th noise sequence;

[0071] Indicates the true emotion category predicted by the model for the i-th noise sequence during the matching process The probability of is the probability distribution of the emotion category predicted by the model for the i-th noise sequence, is the true emotion category corresponding to the i-th noise sequence determined in the matching process; Contrastive denoising loss: Contrastive loss includes boundary loss and emotion classification loss. Specifically, only the boundary probability of positive samples Calculate the boundary loss and classification loss according to the classification probability of positive samples and negative samples respectively and Calculate, therefore, the contrast loss is calculated as follows:

[0072]

[0073] represents the probability of the true boundary index predicted by the i-th positive sample at the boundary index δ, Represents the true boundary index of the i-th positive sample;

[0074] represents the probability of the true emotion category predicted by the i-th positive sample on emotion category c, Represents the true sentiment category of the i-th positive sample;

[0075] Indicates the probability of predicting that the i-th negative sample belongs to the "Invalid" category;

[0076] Jointly optimize matching loss and contrast denoising loss The overall training loss can be expressed as:

[0077]

[0078] In the above solution, the specific steps of step S60 are as follows:

[0079] Step S601, in the inference phase, randomly sample N from a Gaussian distribution eval The noise sequence starts, and then it iteratively denoises based on the denoising time step τ using the learned boundary index back diffusion process. The predicted probability obtained from this denoising process corresponds to the boundary index probability and the emotion category probability;

[0080] Using the boundary index probability and sentiment category probability, the model decodes N eval Candidate sentiment triplets After decoding, post-processing steps are applied:

[0081] Data deduplication and filtering;

[0082] For triplets with the same term boundary index, the algorithm retains the triplet with the highest polarity probability;

[0083] In addition, the cumulative sum of predicted probabilities is below the threshold The triplets will be filtered.

[0084] The beneficial effects of the present invention are:

[0085] The purpose of the present invention is to solve the problem that the autoregressive decoding in the existing generative model only focuses on the generation of a single word, which limits the model's ability to utilize the overall semantics when processing multi-word aspect / opinion terms. Therefore, based on the non-autoregressive diffusion model framework, the present invention defines the aspect-level sentiment triple extraction task as a boundary denoising diffusion process, injects Gaussian noise into the aspect / opinion term boundary through the forward diffusion process, and systematically introduces uncertainty. Different from the traditional word-by-word generation paradigm, by directly modeling the boundary index, all boundary indexes can be efficiently predicted in a single step, avoiding the cumbersome process of word-by-word generation, and can fully utilize comprehensive context information in the prediction process. By gradually refining and dynamically adjusting the boundaries in the noisy state, the present invention significantly improves the accuracy of aspect and opinion term boundary prediction. In addition, in order to further improve the robustness and prediction performance of the model, the present invention introduces a contrast denoising training strategy. This strategy denoises positive and negative samples, further trains and optimizes the performance of the decoder, effectively alleviates the repeated predictions with subtle changes introduced by the diffusion process, thereby solving the multi-word term prediction problem mentioned in the above background technology and improving the aspect-level sentiment triple extraction performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1 The figure is a flow chart of the steps of the method of the present invention. DETAILED DESCRIPTION

[0087] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0088] See also Figure 1 , the present invention provides a technical solution:

[0089] The present invention provides the following technical solution: a method for extracting aspect-level sentiment triples based on a diffusion model. The specific steps are to extract aspect-level sentiment triples of restaurant reviews as an example. The specific steps of applying the aspect-level sentiment triple extraction method to the field of restaurant review sentiment triple extraction are as follows:

[0090] Step 1: In the forward diffusion phase, Gaussian noise is gradually added to the indices in these sequences. In contrast, the backward diffusion process aims to gradually restore the original boundary indices;

[0091] Step 1-1, the boundary index forward diffusion phase, gradually introduces Gaussian noise into the boundary sequence to simulate the inherent uncertainty in identifying term boundaries. The number N is normalized to N train , the standardized sequence is expressed as The noise sequence at any given time step t is calculated using a single-step Markov transition as follows:

[0092]

[0093] in is the cumulative noise retention ratio, representing all α from time step 0 to t s The continuous product of β s represents a predefined variance table, ∈ is a random vector, each element of which is a sample drawn from a normal distribution, represents the noise sampled from a standard Gaussian distribution, where α s Indicates the noise retention ratio at the s-th time step;

[0094] Step 1-2, starting from the noisy state, the back diffusion process uses a non-Markov denoising strategy DDIM. DDIM is used to accurately reconstruct term boundaries. The process involves selecting a subsequence τ with a length γ from the complete time step sequence [1, ..., T] and using the predicted boundary sequence of the previous time step and the predicted noise Iteratively refine the boundary sequence Iterative refinement process, using a trainable denoising network conditioned on S As shown below:

[0095]

[0096]

[0097] in Indicates that at the τth i The noise retention ratio of time steps, represents the time step τ i The predicted boundary sequence of Indicates that at the τth i time-step noise boundary sequence, Indicates that at the τth i The noise of the predictions for time steps is determined as the noise boundary sequence and the prediction boundary sequence The normalized difference between and Combined and adjusted according to their respective standard deviations, this process is repeated iteratively, as shown in the expression:

[0098]

[0099] Indicates that at the τth i-1 The estimated value of the boundary sequence obtained in the back diffusion process of time steps;

[0100] Indicates that at the τth i-1 The noise retention ratio of time steps;

[0101] After Y iterations of DDIM, the noise boundary index is gradually refined and converges to the accurate boundary index.

[0102] Step 2: In the denoising network f θ (X t , S, t i ), using the noise boundary sequence X t and sentence S as input, and then predict the corresponding prediction boundary sequence

[0103] Step 2-1: The encoder takes an input sentence S = {w1, w1, ..., w M}Converted to h-dimensional sentence representation Implemented as a pre-trained language model PLM with bidirectional LSTM;

[0104] H S =BiLSTM(BERT(S))

[0105] BERT(*) indicates the use of a bidirectional encoder to represent the processing of the input sentence S from the converter BERT, and BiLSTM indicates a bidirectional long short-term memory network;

[0106] Step 2-2, the decoder’s task is to process the sentence representation H s , to obtain the noise boundary index sequence X t The semantic representation of , which represents the aspect and opinion items;

[0107] First, the noise sequence is discretized into word indices by rescaling, then the sequence is represented as It is calculated by mean pooling the tokens at the specified start and end indices of the aspect and opinion items, each represents the merged representation of the i-th sequence within the boundary sequence, which is calculated as follows:

[0108]

[0109] in, Indicates the starting index of the aspect item The hidden state of Indicates the end index of the aspect item The hidden state of This indicates the start index of the view item The hidden state of This indicates the start index of the view item The hidden state at; Pooling(*) represents the pooling operation;

[0110] The Transformer decoder with integrated self-attention and cross-attention layers is used to decode the refined sequence representation. The self-attention module uses the X The generated queries, keys, and values ​​facilitate interaction between sequences:

[0111] H sa =SelfAttention(H X )

[0112] Among them, SelfAttention(*) represents the attention mechanism, At the same time, the cross-attention mechanism further refines the sequence representation by incorporating the broader semantic context of the sentence, by utilizing the self-attention module H sa The output of is implemented as a query, with keys and values ​​coming from the sentence representation H s , expressed as:

[0113] H ca =CrossAttention(H sa , H s )

[0114] Among them, CrossAttention(*) represents the cross attention mechanism. To accommodate the iterative nature of the diffusion process, the sinusoidal code E corresponding to each time step t is t is integrated into the sequence representation, and the final noisy sequence representation The calculation is as follows:

[0115]

[0116] Four index pointers are used to predict the boundary indexes of aspect and opinion items respectively. For each index δ∈{a s , a e , o s , o e}, generate a fusion representation It combines the noisy sequence representation with the sentence representation, and each index is predicted to be the probability of a term boundary. as follows:

[0117]

[0118]

[0119] in is a learnable matrix, FFN(*) represents a feedforward network, It is a type code used to distinguish aspects or viewpoint boundaries.

[0120] Step 3: The sentiment classifier processes the sequence representation through a feed-forward neural network FFN to output the probability distribution of sentiment categories;

[0121] Step 3-1: Sentiment classifier processes sequence representation through FFN The probability distribution of the output emotion category is expressed as:

[0122]

[0123] in, C represents the total number of sentiment polarity categories.

[0124] Step 4: Further improve the accuracy in refining term boundaries by contrasting the denoising training strategy and strengthen the sentiment classification process by reducing the generation of erroneous triplets;

[0125] In step 4-1, a certain degree of uncertainty is introduced during the diffusion process, resulting in repeated predictions around the initially predicted boundary index. It enables the model to flexibly explore various possible positions of the term start or end. However, it is worth noting that while this increased uncertainty helps to handle multi-word terms, it also carries the risk of incorrectly predicting boundary indices due to subtle changes. In order to further improve the accuracy of refined term boundaries and strengthen the sentiment classification process by reducing the generation of erroneous triplets, we introduced a contrastive denoising training strategy during the training phase.

[0126] By adding two different levels of noise λ1 and λ2 to N train Two types of samples are generated from the true values, positive samples and negative samples, where λ1<λ2. After diffusion back-processing, the decoder takes two more types of samples as input. The noise scale of the positive sample is less than λ1 and is used to reconstruct its corresponding true value. The noise scale of the negative sample is greater than λ1 and less than λ2, and is used to predict "Invalid", denoted as ε. If a sentence has N train The true value, compared to the denoising training, will have 2×N train samples, each true value generates a positive sample and a negative sample;

[0127] Similar to the previous calculation process, the boundary probability of the positive sample can be obtained The classification probabilities of positive samples and negative samples are expressed as and

[0128] Step 5: Update the model parameters by defining the loss function as matching loss and contrast denoising loss;

[0129] Step 5-1, the training loss function includes matching loss and contrast denoising loss;

[0130] Matching loss: When processing N train predictions and the corresponding N train When the true value is expanded, the Hungarian algorithm is used to establish the best match between the predicted boundary probability set and the sentiment classification probability set. Represents the true situation corresponding to the i-th noise sequence. The matching loss includes boundary loss and sentiment classification loss. Subsequently, the inverse process is trained by maximizing the possibility of prediction:

[0131]

[0132] δ∈{a s , a e , o s , o e} indicates that this represents the boundary index of aspect items and opinion items, where a s and a e Respectively represent the start and end index of the aspect item, and o s and e Respectively represent the start and end index of the opinion item;

[0133] Indicates that during the matching process, for the i-th noise sequence, the true boundary index predicted by the model The probability of is the true boundary index corresponding to the i-th noise sequence determined during the matching process, is the probability distribution of the boundary index δ predicted by the model for the i-th noise sequence;

[0134] Indicates the true emotion category predicted by the model for the i-th noise sequence during the matching process The probability of is the probability distribution of the emotion category predicted by the model for the i-th noise sequence, is the true emotion category corresponding to the i-th noise sequence determined in the matching process; Contrastive denoising loss: Contrastive loss includes boundary loss and emotion classification loss. Specifically, only the boundary probability of positive samples Calculate the boundary loss and classification loss according to the classification probability of positive samples and negative samples respectively and Calculate, therefore, the contrast loss is calculated as follows:

[0135]

[0136] represents the probability of the true boundary index predicted by the i-th positive sample at the boundary index δ, Represents the true boundary index of the i-th positive sample;

[0137] represents the probability of the true emotion category predicted by the i-th positive sample on emotion category c, Represents the true sentiment category of the i-th positive sample;

[0138] Indicates the probability of predicting that the i-th negative sample belongs to the "Invalid" category;

[0139] Jointly optimize matching loss and contrast denoising loss The overall training loss can be expressed as:

[0140]

[0141] Step 6: In the inference phase, we start by randomly sampling a noise sequence from a Gaussian distribution. This is followed by an iterative denoising process using the learned boundary indexes using a back-diffusion process. The predicted probabilities resulting from this denoising process correspond to the likelihood associated with the boundary index and its sentiment polarity.

[0142] Step 6-1, step S601, in the inference phase, randomly sample N from a Gaussian distribution eval The noise sequence starts, and then it iteratively denoises based on the denoising time step τ using the learned boundary index back diffusion process. The predicted probability obtained from this denoising process corresponds to the boundary index probability and the emotion category probability;

[0143] Using the boundary index probability and sentiment category probability, the model decodes N eval Candidate sentiment triplets After decoding, post-processing steps are applied:

[0144] Data deduplication and filtering;

[0145] For triplets with the same term boundary index, the algorithm retains the triplet with the highest polarity probability;

[0146] In addition, the cumulative sum of predicted probabilities is below the threshold The triplets will be filtered.

[0147] A diffusion model-based aspect-level sentiment triple extraction method includes a boundary forward diffusion module, a boundary denoising network module, and a contrast denoising training module, specifically:

[0148] In the boundary forward diffusion phase, Gaussian noise is gradually added to the indexes in these sequences to generate boundary indexes of the noise state;

[0149] In the denoising network f0(x t , S, t i ), a noise boundary sequence and a sentence representation are taken as input, and then the corresponding term boundaries and the corresponding sentiment polarity are predicted;

[0150] By contrasting the denoising training strategy, the accuracy in refining term boundaries is further improved, and the sentiment classification process is strengthened by reducing the generation of erroneous triplets, thereby improving the performance of aspect-level sentiment triple extraction.

[0151] The method of the present invention takes the aspect-level sentiment triple extraction method as the research object, and defines the aspect-level sentiment triple extraction task as a boundary denoising diffusion process based on the non-autoregressive diffusion model framework. By directly modeling the boundary index, the boundary under the noise state is gradually refined and dynamically adjusted according to the comprehensive context information. In addition, a contrast denoising training strategy is introduced, which effectively alleviates the repeated predictions with subtle changes introduced by the diffusion process and improves the aspect-level sentiment triple extraction performance.

[0152] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for extracting aspect-level sentiment triples based on a diffusion model, characterized in that: The following steps are involved: step S10: for boundary sequence T b Gaussian noise is introduced to simulate the uncertainty in identifying term boundaries and obtain the noise sequence X t ; Step S20: In the denoising network f θ (X t ,S,t i ), using the noise boundary sequence X t and sentence S as input, and then predict the corresponding prediction boundary sequence Step S30: the sentiment classifier processes the sequence representation through a feed-forward neural network FFN to output a probability distribution of sentiment categories; Step S40: generating positive samples and negative samples by comparing denoising training strategies, improving the accuracy in refining term boundaries, and strengthening the sentiment classification process by reducing the generation of erroneous triples; Step S50: define the loss function as matching loss and contrast denoising loss, and update the model parameters; Step S60: In the inference phase, a noise sequence is randomly sampled from a Gaussian distribution, and then iteratively denoised using a back-diffusion process with learned boundary indices. The predicted probabilities derived from the denoising process correspond to the predicted probabilities associated with the boundary indices and their sentiment polarities.

2. The method for extracting aspect-level sentiment triples based on a diffusion model according to claim 1, characterized in that: The specific steps of step S10 are as follows: Step S101, the boundary index forward diffusion stage, gradually introduces Gaussian noise into the boundary sequence to simulate the inherent uncertainty in identifying term boundaries. For parallel training, The number N is normalized to N train , the standardized sequence is expressed as The noise sequence X at any given time step t is t The calculation using a single-step Markov transition is as follows: in is the cumulative noise retention ratio, representing all α from time step 0 to t s The continuous product of β s represents a predefined variance table, ∈ is a random vector, each element of which is a sample drawn from a normal distribution, represents the noise sampled from a standard Gaussian distribution, where α s Indicates the noise retention ratio at the s-th time step; Step S102: From the noise state X t Initially, the back-diffusion process uses a non-Markov denoising strategy DDIM, which is used to accurately reconstruct term boundaries. The process involves selecting a subsequence τ of length γ from the full time step sequence [1, …, T] and using the predicted boundary sequence of the previous time step and the predicted noise Iteratively refine the boundary sequence Iterative refinement process, using a trainable denoising network conditioned on S As shown below: in Indicates that at the τth i The noise retention ratio of time steps, represents the time step τ i The predicted boundary sequence of Indicates that at the τth i time-step noise boundary sequence, Indicates that at the τth i The noise of the predictions for time steps is determined as the noise boundary sequence and the prediction boundary sequence The normalized difference between and Combined and adjusted according to their respective standard deviations, this process is repeated iteratively, as shown in the expression: Indicates that at the τth i-1 The estimated value of the boundary sequence obtained in the back diffusion process of time steps; Indicates that at the τth i-1 The noise retention ratio of time steps; After γ iterations of DDIM, the noise boundary index is gradually refined and converges to the accurate boundary index.

3. The method for extracting aspect-level sentiment triples based on a diffusion model according to claim 1, characterized in that: The specific steps of step S20 are as follows: Step S201: The encoder converts the input sentence S = {w 1， w 1， …, w M }Converted to h-dimensional sentence representation Implemented as a pre-trained language model PLM with bidirectional LSTM; H s =BiLSTM(BERT(S)) BERT(*) indicates the use of a bidirectional encoder to represent the processing of the input sentence S from the converter BERT, and BiLSTM indicates a bidirectional long short-term memory network; Step S202: The task of the decoder is to process the sentence representation H s , to obtain the noise boundary index sequence X t The semantic representation of , which represents the aspect and opinion items; First, the noise sequence is discretized into word indices by rescaling, then the sequence is represented as It is calculated by mean pooling the tokens at the specified start and end indices of the aspect and opinion items, each represents the merged representation of the i-th sequence within the boundary sequence, which is calculated as follows: in, Indicates the starting index of the aspect item The hidden state of Indicates the end index of the aspect item The hidden state of This indicates the start index of the view item The hidden state of This indicates the start index of the view item The hidden state at; Pooling(*) represents the pooling operation; The Transformer decoder with integrated self-attention and cross-attention layers is used to decode the refined sequence representation. The self-attention module uses the X The generated queries, keys, and values ​​facilitate interaction between sequences: H sa =SelfAttention(H X ) Among them, SelfAttention(*) represents the attention mechanism, At the same time, the cross-attention mechanism further refines the sequence representation by incorporating the broader semantic context of the sentence, by utilizing the self-attention module H sa The output of is implemented as a query, with keys and values ​​coming from the sentence representation H s , expressed as: H ca =CrossAttention(H sa ,H s ) Among them, CrossAttention(*) represents the cross attention mechanism. To accommodate the iterative nature of the diffusion process, the sinusoidal code E corresponding to each time step t is t is integrated into the sequence representation, and the final noisy sequence representation The calculation is as follows: Four index pointers are used to predict the boundary indexes of aspect and opinion items respectively. For each index δ∈{a s , a e , o s , o e }, generate a fusion representation It combines the noisy sequence representation with the sentence representation, and each index is predicted to be the probability of a term boundary. as follows: in is a learnable matrix, FFN(*) represents a feedforward network, It is a type code used to distinguish aspects or viewpoint boundaries.

4. The method for extracting aspect-level sentiment triples based on a diffusion model according to claim 1, characterized in that: The specific steps of step S30 are as follows: Step S301: The sentiment classifier processes the sequence representation through FFN The probability distribution of the output emotion category is expressed as: in, C represents the total number of sentiment polarity categories.

5. The method for extracting aspect-level sentiment triples based on a diffusion model according to claim 1, characterized in that: In step S40, the denoising training strategy is compared: By adding two different levels of noise λ1 and λ2 to N train Two types of samples are generated from the real values, positive samples X + and negative samples X - , where λ1<λ2. After diffusion back-processing, the decoder takes two types of samples as input. The noise scale of the positive sample is less than λ1 and is used to reconstruct its corresponding true value. The noise scale of the negative sample is greater than λ1 and less than λ2, which is used to predict the sentiment category "Invalid" (invalid triplet) when the aspect word does not match the opinion word, denoted as ε. If a sentence has N train The true value, compared to the denoising training, will have 2×N train samples, each true value generates a positive sample and a negative sample; The positive sample X + and negative samples X - Input decoder, the specific calculation is to first pass self-attention and cross-attention: H sa+ =SelfAttention(X + ) H sa- =SelfAttention(X - ) H ca+ =CrossAttention(H sa+ ,H s ) H ca- =CrossAttention(H sa- ,H s ) Among them, SelfAttention(*) represents the attention mechanism, CrossAttention(*) represents the cross attention mechanism, H s is the sentence representation. In order to adapt to the iterative nature of the diffusion process, the sinusoidal code E corresponding to each time step t is t is integrated into the sequence representation, and the final noisy sequence representation and The calculation is as follows: Four index pointers are used to predict the boundary indexes of aspect and opinion items respectively. For each index δ∈{a s , a e , o s , o e }, generate fusion representation It combines the noise sequence representation with the sentence representation to obtain the boundary probability of positive samples in is a learnable matrix, FFN(*) represents a feedforward network, It is a type code used to distinguish aspects or viewpoint boundaries. The classification probabilities of positive samples and negative samples calculated by the sentiment classifier are expressed as and in, C represents the total number of sentiment polarity categories.

6. The method for extracting aspect-level sentiment triples based on a diffusion model according to claim 1, characterized in that: The specific steps of step S50 are as follows: Step S501, the training loss function includes matching loss and contrast denoising loss; Matching loss: When processing N train predictions and the corresponding N train When the true value is expanded, the Hungarian algorithm is used to establish the best match between the predicted value set and the true value set. Represents the true situation corresponding to the i-th noise sequence. The matching loss includes boundary loss and sentiment classification loss. Subsequently, the inverse process is trained by maximizing the possibility of prediction: δ∈{a s , a e , o s , o e } indicates that this represents the boundary index of aspect items and opinion items, where a s and a e Respectively represent the start and end index of the aspect item, and o s and e Respectively represent the start and end index of the opinion item; Indicates that during the matching process, for the i-th noise sequence, the true boundary index predicted by the model The probability of is the true boundary index corresponding to the i-th noise sequence determined during the matching process, is the probability distribution of the boundary index δ predicted by the model for the i-th noise sequence; Indicates the true emotion category predicted by the model for the i-th noise sequence during the matching process The probability of is the probability distribution of the emotion category predicted by the model for the i-th noise sequence, is the true emotion category corresponding to the i-th noise sequence determined in the matching process; Contrastive denoising loss: Contrastive loss includes boundary loss and emotion classification loss. Specifically, only the boundary probability of positive samples Calculate the boundary loss and classification loss according to the classification probability of positive samples and negative samples respectively and Calculate, therefore, the contrast loss is calculated as follows: represents the probability of the true boundary index predicted by the i-th positive sample at the boundary index δ, Represents the true boundary index of the i-th positive sample; represents the probability of the true emotion category predicted by the i-th positive sample on emotion category c, Represents the true sentiment category of the i-th positive sample; Indicates the probability of predicting that the i-th negative sample belongs to the "Invalid" category; Jointly optimize matching loss and contrast denoising loss The overall training loss can be expressed as:

7. The method for extracting aspect-level sentiment triples based on a diffusion model according to claim 1, characterized in that: The specific steps of step S60 are as follows: Step S601, in the inference phase, randomly sample N from a Gaussian distribution eval The noise sequence starts, and then it iteratively denoises based on the denoising time step τ using the learned boundary index back diffusion process. The predicted probability obtained from this denoising process corresponds to the boundary index probability and the emotion category probability; Using the boundary index probability and sentiment category probability, the model decodes N eval Candidate sentiment triplets After decoding, post-processing steps are applied: Data deduplication and filtering; For triplets with the same term boundary index, the algorithm retains the triplet with the highest polarity probability; In addition, the cumulative sum of predicted probabilities is below the threshold The triplets will be filtered.

Images