Intelligent voice reading method and system based on sentiment analysis
Through the attribute-level emotion triple generation model based on the encoder-decoder structure, combined with the cross entropy loss and the comparison learning mechanism, the error propagation, calculation overhead and emotion judgment ability of attribute-level emotion triple extraction in the existing technology is solved. The limitation problem achieves more efficient and accurate triple extraction.
Patent Information
- Application Number
- CN202411035997.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-11-29
AI Technical Summary
The existing attribute-level emotional triple extraction method has error propagation, large calculation overhead, complex modeling and reasoning, insufficient pairing learning of attribute-opinion terms and limitations in emotional judgment capabilities, resulting in low accuracy and efficiency of the extraction results, which is difficult to Meet practical application needs.
Adopt the attribute-level emotion triple generation model based on the encoder-decoder structure, and use the T5 encoder and decoder for text encoding and triple generation, and combine the pairing layer and emotion through the cross entropy loss function and contrast learning mechanism. The loss function of the layer is trained in multi-task joint to improve the matching accuracy of attribute words and opinion words and the accuracy of emotional judgment.
It significantly improves the matching accuracy of attribute words and opinion words and the accuracy of emotional judgment, enhances the overall quality of triple extraction, improves the robustness and applicability of the model, and meets the practical application needs.
Smart Images

Figure CN119047477A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of natural language processing, and in particular to an intelligent voice reading method and system based on sentiment analysis. Background Art
[0002] There are several major shortcomings in existing research on the attribute-level sentiment triple extraction task. First, the pipeline approach adopted in the early days is prone to error propagation and increased computational overhead, which affects the accuracy and processing efficiency of the extraction results. Although later methods such as grid annotation and location-aware annotation can extract all triplets at once, these methods often require complex modeling and reasoning processes. Second, although the research based on the encoder-decoder paradigm utilizes advanced pre-trained language models, it is still insufficient in the paired learning of attribute-opinion term pairs, and often results in mismatches or missed matches.
[0003] In addition, existing models also have limitations in their ability to determine sentiment. Due to the uneven distribution of sentiment samples in the corpus and the implicitness of some sentiment expressions, current triple extraction models perform poorly in identifying subtle sentiment differences between different attribute items in the text. These problems limit the application effect of triple extraction tasks, making it difficult for existing technologies to meet actual needs. Summary of the invention
[0004] In view of the problems of error propagation, high computational overhead, complex modeling and reasoning, insufficient learning of attribute-opinion term pairings, and limited sentiment judgment capabilities in existing attribute-level sentiment triple extraction methods, the present invention is proposed.
[0005] Therefore, the problem to be solved by the present invention is how to improve the accuracy, efficiency and practicality of extraction, so as to better meet the actual application needs.
[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:
[0007] In a first aspect, an embodiment of the present invention provides an intelligent voice reading method and system based on sentiment analysis, which includes obtaining a review text dataset containing user evaluations; constructing an attribute-level sentiment triple generation baseline model based on an encoder-decoder structure; encoding the review text into a hidden representation using a T5 encoder, and generating attribute-level sentiment triples based on the hidden representation through a T5 decoder, wherein the attribute-level sentiment triples include two output paradigms, annotated and extracted; and calculating the triple generation loss L using a cross entropy loss function. e ; Perform multi-level pairing information enhancement, and use the contrastive learning of the pairing layer and the sentiment layer to calculate the pairing layer contrast loss L aop And the emotional layer contrast loss L sen; Combine the triplet generation loss, pairing layer contrast loss and sentiment layer contrast loss to construct a loss function and perform multi-task joint training; Use the trained attribute-level sentiment triples to generate a baseline model to extract attribute-level sentiment triplets from new review texts.
[0008] As a preferred solution of the intelligent voice reading method and system based on sentiment analysis described in the present invention, the specific formula of the T5 encoder is as follows:
[0009] H e =T5Encoder(x1,x2,...,x n )
[0010] Among them, H e ∈R n×d is the output hidden representation of the T5 encoder, d is the dimension of the hidden representation, T5Encoder(·) is the T5 encoder, (x1,x2,...,x n ) is the word sequence of the input comment text, n is the sequence length, and R is a set of real numbers.
[0011] The specific formula of the T5 decoder is as follows:
[0012]
[0013] in, is the output of the T5 decoder at time step t, d is the dimension of the hidden representation, T5Decoder(·) is the T5 decoder, W v and b v is the learnable parameter matrix and bias vector, Softmax(·) is the Softmax activation function, Y <t is the output sequence of the decoder before time step t, P t is the probability distribution of the vocabulary at the current time step t, H e ∈R n×d is the output hidden representation of the T5 encoder, and R is a set of real numbers.
[0014] As a preferred solution of the intelligent voice reading method and system based on sentiment analysis described in the present invention, wherein: multi-level pairing information enhancement includes pairing layer information enhancement and sentiment layer information enhancement, and the execution steps of pairing layer information enhancement are as follows: define a description information set, the description information set includes a matching type description and a non-matching type description; encode the pairing description information based on the BERT model to obtain the description information representation; extract the hidden layer representation of attribute words and opinion words from the output of the T5 encoder, randomly splice the hidden layer representation of attribute words and opinion words, and obtain the term pair representation through the linear layer; use the contrastive learning method to calculate the similarity between the description information representation and the term pair representation; optimize the parameters of the attribute-level sentiment triple generation baseline model based on the pairing layer contrast loss function to improve the matching of attribute words and opinion words.
[0015] As a preferred solution of the intelligent voice reading method and system based on sentiment analysis described in the present invention, the specific formula for obtaining the representation of the description information is as follows:
[0016]
[0017] in, is the output of the [CLS] position, W aop and b aop is a learnable parameter, d aop is the paired information description, m represents the number of words, BERT(·) is the pre-trained bidirectional Transformer encoder function, and LeakyReLU(·) is the leaky linear rectifier unit activation function.
[0018] The specific formula for term pair representation is as follows:
[0019]
[0020] Among them, h a and h o are the dense representations of attribute words and opinion words respectively, f is an average pooling function, and W s and b s is a learnable parameter, I is the position index of the attribute word and opinion word in the comment text X, and H e is the output hidden representation of the T5 encoder, h c is a term pair representation.
[0021] As a preferred solution of the intelligent voice reading method and system based on sentiment analysis described in the present invention, the execution steps of sentiment layer information enhancement are as follows: defining a sentiment category set, the sentiment category set includes three sentiment categories: positive, neutral and negative; encoding the sentiment category description based on the BERT model, and using a new linear layer to obtain sentiment information representation; extracting the hidden layer representation of the real attribute word and opinion word pair from the output of the T5 encoder; converting the hidden layer representation of the real term pair into a term pair representation through a linear layer; using a contrastive learning method to calculate the similarity between the sentiment information representation and the term pair representation; optimizing the parameters of the attribute-level sentiment triple generation baseline model based on the sentiment layer contrast loss function to improve its ability to understand and distinguish different sentiments; sentiment information representation d sen The specific formula is as follows:
[0022]
[0023] Among them, LeakyReLU(·) is the activation function of the linear rectifier unit with leakage, is the output of the [CLS] position, W sen and b sen are learnable parameters.
[0024] As a preferred solution of the intelligent voice reading method and system based on sentiment analysis described in the present invention, the pairing layer contrast loss function L sen The specific formula is as follows:
[0025]
[0026] in, is the set of representations of all true term pairs that do not belong to the current sentiment category, d sen To represent emotional information, is the real term pair information in the sequence, τ is the temperature coefficient, For real term information and the similarity of emotional information representation.
[0027] As a preferred solution of the intelligent voice reading method and system based on sentiment analysis described in the present invention, the specific formula of the comprehensive loss function L is as follows:
[0028] L=αL e +βL aop +γL sen
[0029] Among them, α, β, γ are weight hyperparameters, L e , L aop , L senGenerate loss functions for triples, contrastive loss functions for pairing layers, and contrastive loss functions for sentiment layers, respectively.
[0030] In the second aspect, an embodiment of the present invention provides an attribute-level sentiment triple extraction system based on pairing information enhancement, which includes a text data acquisition module for acquiring a comment text dataset containing user evaluations; a baseline model construction module for constructing an attribute-level sentiment triple generation baseline model based on an encoder-decoder structure; a triple generation module for encoding the comment text into a hidden representation using a T5 encoder, and based on the hidden representation, generating attribute-level sentiment triples through a T5 decoder, wherein the attribute-level sentiment triples include two output paradigms, annotated and extracted; a loss calculation module for calculating the triple generation loss using a cross-entropy loss function; a multi-level pairing information enhancement module for performing multi-level pairing information enhancement, and calculating the pairing layer contrast loss and the sentiment layer contrast loss using contrast learning of the pairing layer and the sentiment layer; a multi-task joint training module for combining the triple generation loss, the pairing layer contrast loss and the sentiment layer contrast loss to construct a comprehensive loss function and perform multi-task joint training; an attribute-level sentiment triple generation module for using the trained attribute-level sentiment triples to generate a baseline model to extract attribute-level sentiment triples from new comment texts.
[0031] In a third aspect, an embodiment of the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program instructions are executed by the processor, the steps of the intelligent voice reading method and system based on sentiment analysis as described in the first aspect of the present invention are implemented.
[0032] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program instructions are executed by a processor, the steps of the intelligent voice reading method and system based on sentiment analysis as described in the first aspect of the present invention are implemented.
[0033] The beneficial effects of the present invention are as follows: the present invention enhances the applicability of the model by designing two output paradigms, namely annotation and extraction. Through comparative learning of the pairing layer and the sentiment layer, the present invention significantly improves the matching accuracy of attribute words and opinion words, as well as the accuracy of sentiment judgment. By adopting the T5 model, the present invention avoids the error propagation problem in traditional non-generative methods, and at the same time can explore label semantics more deeply, improving the overall quality of triple extraction. By combining triple generation loss, pairing layer contrast loss and sentiment layer contrast loss, the present invention realizes multi-task joint training. This method not only improves the overall performance of the model, but also enhances the robustness of the model when dealing with complex emotional expressions. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0035] Figure 1 This is a framework flow chart of the intelligent voice reading method and system based on sentiment analysis.
[0036] Figure 2 This is the overall model architecture diagram of the intelligent voice reading method and system based on sentiment analysis.
[0037] Figure 3 This is a test framework diagram for the intelligent voice reading method and system based on sentiment analysis.
[0038] Figure 4 A computer device diagram of an intelligent voice reading method and system based on sentiment analysis. DETAILED DESCRIPTION
[0039] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.
[0040] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.
[0041] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.
[0042] Example 1
[0043] Reference Figure 1 to Figure 4 , which is the first embodiment of the present invention, and provides an intelligent voice reading method and system based on sentiment analysis, including:
[0044] S1: Get a review text dataset containing user reviews.
[0045] Specifically, the framework flow chart of the present invention is as follows Figure 1 As shown in Figure 2, the overall model architecture is as follows: Figure 2As shown in the figure, two types of description information are designed, one to indicate whether the attribute words and opinion words match, and the other to indicate the meaning of the three tendencies of positive, negative and neutral emotions. The two types of description information are encoded using an encoder independent of the triple extraction model. Subsequently, the dense vector representations of the attribute words and opinion words are extracted from the encoder output of the triple extraction model and randomly paired. Based on the pairing information embedding and the random term pair embedding, a contrastive learning strategy is adopted to enhance the model's ability to match attribute words and corresponding opinion words. At the same time, contrastive learning is applied between the emotional information embedding and the correct term pair embedding to enhance the model's understanding of the emotional representation of the term pair and improve the emotional judgment ability.
[0046] S2: Design an attribute-level sentiment triplet generation baseline model based on the encoder-decoder structure (i.e., T5 model).
[0047] It should be noted that traditional non-generative methods usually use task-specific classification networks to model and predict terminology and sentiment tendencies, and use category indexes as labels. On the one hand, this discriminative modeling method may bring about error propagation problems. On the other hand, the modeling is complex and usually ignores the rich semantics within the label. In contrast, the generative model can explore label semantics more deeply by gradually decoding sequences containing attribute words, opinion words, and sentiment tendencies. Based on this, the present invention uses T5 as the baseline model. T5 is an encoder-decoder based model that is pre-trained on a mixture of multiple unsupervised and supervised task datasets, and can solve triple extraction tasks in a generative manner.
[0048] S3: The review text is encoded into a hidden representation using the T5 encoder, and attribute-level sentiment triples are generated based on the hidden representation using the T5 decoder.
[0049] First, the review text X is encoded into a hidden representation H using the T5 encoder. e , the specific formula is as follows:
[0050] H e =T5Encoder(x1,x2,...,x n )
[0051] Among them, H e ∈R n×d is the output hidden representation of the T5 encoder, d is the dimension of the hidden representation, T5Encoder(·) is the T5 encoder, X=(x1,x2,...,x n ) is the word sequence of the input comment text, n is the sequence length, and R is a set of real numbers.
[0052] It should be noted that the present invention divides the output paradigm of the T5 model into annotation and extraction. The annotation output labels each attribute word with its corresponding opinion word and sentiment polarity to form a structure of [attribute word|opinion word|sentiment tendency]. This method intuitively presents the relationship between attributes, opinions, and sentiments, which is easy to understand and analyze. The extraction output adopts the concatenation and combination of triplets. Specifically, the model directly generates a complete triple sequence consisting of attribute words, opinion words, and sentiment polarity. This method is closer to the final application format, which is conducive to subsequent processing and utilization.
[0053] Example, review text: Nice keyboard, battery and screen work ok (translation: keyboard is good, battery and screen are also OK); generated target (annotation style): Nice
[0054] [keyboard|positive|Nice],[battery|neutral|ok]and[screen|neutral|ok]work ok; Generate target (extractive): (keyboard,Nice,postive);
[0055] (battery, ok, neutral); (screen, ok, neutral) (Translation: (keyboard, good, positive); (battery, ok, neutral); (screen, ok, neutral)).
[0056] Furthermore, in order to generate the output defined above, the T5 decoder uses the encoder output H e and the decoder output Y at the previous time step <t As input, to get the decoder output of the current step Subsequently, a fully connected softmax layer will Mapped to the vocabulary distribution, the word probability is generated. The specific formula is as follows:
[0057]
[0058] in, is the output of the T5 decoder at time step t, d is the dimension of the hidden representation, T5Decoder(·) is the T5 decoder, W v and b v is the learnable parameter matrix and bias vector, Softmax(·) is the Softmax activation function, Y <t is the output sequence of the decoder before time step t, Pt is the probability distribution of the vocabulary at the current time step t, H e ∈R n×d is the output hidden representation of the T5 encoder, and R is a set of real numbers.
[0059] S4: Use the cross entropy loss function to calculate the triple generation loss L e .
[0060] Furthermore, for the above triple generation, the cross entropy function is used to calculate the triple generation loss L e , the specific formula is as follows:
[0061]
[0062] Among them, y i is the correct label, indicating the attribute word, opinion word, sentiment label or other content in the target output.
[0063] S5: Perform multi-level pairing information enhancement and use contrastive learning between the pairing layer and the sentiment layer to calculate the pairing layer contrast loss L aop And the emotional layer contrast loss L sen .
[0064] Specifically, the multi-level pairing information enhancement includes pairing layer information enhancement and sentiment layer information enhancement, including the following steps:
[0065] S5.1: Perform paired layer information enhancement.
[0066] It should be noted that the present invention defines D aop ={d pair ,d unpair}As a set of descriptive information indicating whether the attribute words and opinion words match, the specific definitions are as follows: Pair (match): The tokens have an emotional connection. (Translation: These words have emotional connection); Unpair (mismatch): The tokens do not have any emotional connection. (Translation: These words have no emotional connection).
[0067] Specifically, a description information set is defined, which includes a matching type description and a non-matching type description. Based on the BERT model, a pairing information description d containing m words is aop Encode and obtain the description information representation. The specific formula is as follows:
[0068]
[0069] in, is the output of the [CLS] position, W aop and b aop is a learnable parameter, d aop is the paired information description, m represents the number of words, BERT(·) is the pre-trained bidirectional Transformer encoder function, and LeakyReLU(·) is the leaky linear rectifier unit activation function.
[0070] Furthermore, in order to generate correct paired and mismatched sample instances, the output H of the T5 encoder is e Extract the hidden representation of attribute words and opinion words, and randomly concatenate attribute words h a and opinion word h o The hidden layer representation is obtained through the linear layer to obtain the term pair representation. The specific formula is as follows:
[0071]
[0072] Among them, h a and h o are the dense representations of attribute words and opinion words, respectively, f is an average pooling function (mapping n output vectors into a representative vector), W s and b s is a learnable parameter, I is the position index of the attribute word and opinion word in the comment text X, and H e is the output hidden representation of the T5 encoder, h c is the term pair representation, This is a vector concatenation operation.
[0073] It should be noted that, due to random coding, h c It may be a true match or a false match.
[0074] Furthermore, a contrastive learning method is used to calculate the similarity between the descriptive information representation and the term pair representation; the parameters of the attribute-level sentiment triple generation baseline model are optimized based on the pairing layer contrast loss function to improve the matching of attribute words and opinion words. That is, in order to achieve pairing information enhancement, the present invention tests three methods to match known attribute words and opinion words, including random selection, linear function and contrastive learning. The test framework is shown in the attached figure. Figure 3 According to the test results, the present invention selects the contrastive learning method with the best performance. Contrastive learning ensures that the similarity between the hidden representation of the corresponding description information and the correct attribute-opinion word pair is maximized, while the similarity between the mismatched term pairs is minimized. This process is achieved through the paired layer contrast loss L aop accomplish:
[0075]
[0076] in, is the set of term pair representations that do not belong to the current pairing type, d aop is the pairing information description, and τ is the temperature coefficient.
[0077] Preferably, since the description information represents the center point of correct pairing and mismatching, it contains rich semantic information related to pairing. Through contrastive learning, the T5 model can extract pairs consisting of attributes and opinions and learn this information, thereby facilitating the extraction of valuable features for pairing. Therefore, this makes it possible to generate more accurate triples.
[0078] S5.2: Perform sentiment layer information enhancement.
[0079] Specifically, the present invention defines D sen ={d pos ,d neu ,d neg As a specific explanation of the three labels of positive emotion, negative emotion and neutral emotion, the definitions are as follows: Pos (positive): Feelings of happiness, joy, or satisfaction. (Translation: feelings of happiness, joy or satisfaction); Neu (neutral): Lack of significant emotional response. (Translation: lack of obvious emotional response); Neg (negative): Feelings of sadness, anger, or disappointments. (Translation: feelings of sadness, anger or disappointment).
[0080] Preferably, a set of sentiment categories is defined, and the sentiment category set includes three sentiment categories: positive, neutral and negative; the sentiment category description is encoded based on the BERT model, and the sentiment information representation is obtained using a new linear layer; the hidden layer representation of the real attribute word and opinion word pair is extracted from the output of the T5 encoder; the hidden layer representation of the real term pair is converted into a term pair representation through a linear layer; the similarity between the sentiment information representation and the term pair representation is calculated using a contrastive learning method; the parameters of the attribute-level sentiment triple generation baseline model are optimized based on the sentiment layer contrast loss function to improve its ability to understand and distinguish different sentiments.
[0081] It should be noted that the emotional information representation d sen The specific formula is as follows:
[0082]
[0083] Among them, LeakyReLU(·) is the activation function of the linear rectifier unit with leakage, is the output of the [CLS] position, Wsen and b sen are learnable parameters.
[0084] Furthermore, in order to learn how to correctly pair attribute words and opinion words, random term pair encoding is used; in the present invention, in order to learn how to determine the emotional tendency expressed by the term pair, only the real term pair information in the sequence is extracted. Because the randomly paired term pairs contain combinations without actual emotional information, the model will learn incorrect emotional associations.
[0085] Specifically, through contrastive learning, the model can narrow the distance between the sentiment label and the corresponding term pair, thereby enhancing its ability to understand and distinguish different sentiments. This process is achieved through the pairing layer contrast loss function L sen The specific formula is as follows:
[0086]
[0087] in, is the set of representations of all true term pairs that do not belong to the current sentiment category, d sen To represent emotional information, is the real term pair information in the sequence, τ is the temperature coefficient, For real term information and the similarity of emotional information representation.
[0088] Furthermore, through contrastive learning of sentiment views, the model can learn representations of three different sentiments: positive, negative, and neutral, to better understand the sentiment characteristics of term pairs and make more accurate predictions in sentiment determination tasks.
[0089] S6: Combine triplet generation loss, pairing layer contrast loss and sentiment layer contrast loss to construct a comprehensive loss function and perform multi-task joint training.
[0090] Preferably, the specific formula of the comprehensive loss function L is as follows:
[0091] L=αL e +βL aop +γL sen
[0092] Among them, α, β, and γ are weight hyperparameters, which are set to 0.8, 0.15, and 0.05 in the present invention. e , L aop , L sen Generate loss functions for triples, contrastive loss functions for pairing layers, and contrastive loss functions for sentiment layers, respectively.
[0093] S7: Use the trained attribute-level sentiment triples to generate a baseline model to extract attribute-level sentiment triplets from new review texts.
[0094] It should be noted that the present invention is based on the generative sentiment triple generation model training process, which, on the one hand, solves the problem of mismatching between attribute words and opinion words in the attribute-level sentiment triple generation process, and on the other hand, solves the problem of sentiment judgment errors caused by incorrect term pairing.
[0095] Furthermore, the present embodiment also provides an attribute-level sentiment triple extraction system based on pairing information enhancement, including a text data acquisition module for acquiring a comment text dataset containing user evaluations; a baseline model construction module for constructing an attribute-level sentiment triple generation baseline model based on an encoder-decoder structure; a triple generation module for encoding the comment text into a hidden representation using a T5 encoder, and based on the hidden representation, generating attribute-level sentiment triples through a T5 decoder, wherein the attribute-level sentiment triples include two output paradigms, annotated and extracted; a loss calculation module for calculating the triple generation loss using a cross-entropy loss function; a multi-level pairing information enhancement module for performing multi-level pairing information enhancement, and calculating the pairing layer contrast loss and the sentiment layer contrast loss using contrast learning of the pairing layer and the sentiment layer; a multi-task joint training module for combining the triple generation loss, the pairing layer contrast loss and the sentiment layer contrast loss to construct a comprehensive loss function and perform multi-task joint training; an attribute-level sentiment triple generation module for using the trained attribute-level sentiment triple generation baseline model to extract attribute-level sentiment triples from new comment texts.
[0096] This embodiment also provides a computer device, which is suitable for the intelligent voice reading method and system based on sentiment analysis, including a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the intelligent voice reading method and system based on sentiment analysis proposed in the above embodiments.
[0097] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.
[0098] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method and system for intelligent voice reading based on sentiment analysis as proposed in the above embodiment are implemented.
[0099] In summary, the present invention enhances the applicability of the model by designing two output paradigms, annotation and extraction. Through comparative learning of the pairing layer and the sentiment layer, the present invention significantly improves the matching accuracy of attribute words and opinion words, as well as the accuracy of sentiment judgment. By adopting a generative model based on T5, the present invention avoids the error propagation problem in traditional non-generative methods, and at the same time can explore label semantics more deeply, improving the overall quality of triple extraction. By combining triple generation loss, pairing layer contrast loss and sentiment layer contrast loss, the present invention realizes multi-task joint training. This method not only improves the overall performance of the model, but also enhances the robustness of the model when dealing with complex emotional expressions.
[0100] Example 2
[0101] Reference Figure 1 to Figure 4 , which is the second embodiment of the present invention, and this embodiment provides an intelligent voice reading method and system based on sentiment analysis. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.
[0102] In order to verify the effectiveness of the technical solution of the present invention, a simulation experiment based on product reviews on an e-commerce platform was designed and executed. The experiment used 100,000 mobile phone product review data collected from a large e-commerce platform, of which 80,000 were used for training, 10,000 for verification, and 10,000 for testing. The experiment compared the performance of the standard T5 model (as a baseline model) and the T5 model enhanced with multi-level pairing information (the model proposed by the present invention).
[0103] The experimental results show that in the triple extraction task, the proposed model outperforms the baseline model in all evaluation indicators. Specifically, the accuracy of the proposed model reaches 0.85, which is 7 percentage points higher than the baseline model; the F1 score is 0.81, which is significantly improved compared to 0.73 of the baseline model. This confirms that the multi-level pairing information enhancement mechanism effectively improves the overall performance of the model.
[0104] In the attribute word-opinion word matching task, the accuracy of the model proposed by the present invention reaches 0.89, which is 8 percentage points higher than the 0.81 of the baseline model. This result fully verifies the effectiveness of information enhancement at the pairing layer, enabling the model to more accurately identify and match attribute words and opinion words.
[0105] The results of the sentiment classification task are also significant. The classification accuracy of the proposed model for positive, neutral and negative sentiments is 0.88, 0.82 and 0.85 respectively, which are higher than the baseline model. This shows that the enhancement of sentiment layer information has indeed improved the model's ability to understand and distinguish different sentiments.
[0106] Case analysis further verifies the improvement effect of the model. For the input review "This phone has a great camera, but the battery life is average and the screen display is OK", the model proposed in this invention can more accurately identify attribute words (such as "battery life" instead of just "battery") and is more accurate in sentiment judgment (judging "OK" as neutral instead of positive).
[0107] It should be noted that the training time (12 hours) of the model proposed in this invention is slightly longer than that of the baseline model (8 hours), but considering the significant improvement in performance, the increase in time cost is acceptable. In summary, the experimental results strongly support the effectiveness of the technical solution of the present invention and demonstrate its potential in practical applications. This technical improvement is of great significance for improving the quality of user review analysis on e-commerce platforms, and further optimizing product recommendations and customer service.
[0108] Preferably, the comparison indicators of the present invention and the traditional method are shown in Table 1.
[0109] Table 1 Comparison indexes of the present invention and the traditional method
[0110] index The present invention Traditional methods Model Architecture Classification network or sequence labeling model Generative model based on T5 Triple extraction method Carry out in steps, there are errors Generate complete triples end-to-end Pairing information enhancement none Introducing pairing and sentiment layers Multi-task learning Usually design specialized models for different tasks Using multi-task joint training Accuracy 0.78 0.85 F1 score 0.73 0.81 Attribute word-opinion word matching accuracy 0.81 0.89
[0111] As shown in Table 1, the main differences between the present invention and traditional methods are reflected in the model architecture, triple extraction method, information enhancement mechanism and multi-task learning strategy. The present invention adopts a generative model based on T5 to achieve end-to-end triple generation, and introduces innovative pairing layer and sentiment layer information enhancement mechanisms, while adopting a multi-task joint training strategy. These innovations make the present invention significantly superior to traditional methods in key performance indicators such as accuracy, F1 score and attribute word-opinion word matching accuracy, which are improved by 7 percentage points, 8 percentage points and 8 percentage points respectively. These improvements fully demonstrate the superiority and practical value of the present invention in attribute-level sentiment analysis tasks.
[0112] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.
Claims
1. An intelligent voice reading method and system based on sentiment analysis, characterized in that: include, Get a review text dataset containing user reviews; Design a baseline model for attribute-level sentiment triple generation based on an encoder-decoder structure; The review text is encoded into a hidden representation using the T5 encoder, and based on the hidden representation, attribute-level sentiment triples are generated through the T5 decoder. The attribute-level sentiment triples include two output paradigms: annotation and extraction. The cross entropy loss function is used to calculate the triple generation loss L e ; Perform multi-level pairing information enhancement and use the contrastive learning of the pairing layer and the sentiment layer to calculate the pairing layer contrast loss L aop And the emotional layer contrast loss L sen ; Combine triplet generation loss, pairing layer contrast loss and sentiment layer contrast loss to construct a comprehensive loss function and perform multi-task joint training; Use the trained attribute-level sentiment triples to generate a baseline model and extract attribute-level sentiment triplets from new review texts.
2. The intelligent voice reading method and system based on sentiment analysis according to claim 1, characterized in that: The specific formula of the T5 encoder is as follows: H e =T5Encoder(x1,x2,...,x n ) Among them, H e ∈R n×d is the output hidden representation of the T5 encoder, d is the dimension of the hidden representation, T5Encoder(·) is a T5 encoder, (x1,x2,...,x n ) is the word sequence of the input comment text, n is the sequence length, and R is a real number set; The specific formula of the T5 decoder is as follows: in, is the output of the T5 decoder at time step t, d is the dimension of the hidden representation, T5Decoder(·) is the T5 decoder, W v and b v is the learnable parameter matrix and bias vector, Softmax(·) is the Softmax activation function, Y <t is the output sequence of the decoder before time step t, P t is the probability distribution of the vocabulary at the current time step t, H e ∈R n×d is the output hidden representation of the T5 encoder.
3. The intelligent voice reading method and system based on sentiment analysis as claimed in claim 1, characterized in that: The multi-level pairing information enhancement includes pairing layer information enhancement and emotion layer information enhancement. The execution steps of the pairing layer information enhancement are as follows: Defining a description information set, wherein the description information set includes a matching type description and a non-matching type description; Encode the paired description information based on the BERT model to obtain the description information representation; Extract the hidden representations of attribute words and opinion words from the output of the T5 encoder, randomly concatenate the hidden representations of attribute words and opinion words, and obtain term pair representations through a linear layer; Using a contrastive learning method, calculating the similarity between the description information representation and the term pair representation; The parameters of the baseline model for attribute-level sentiment triples generation are optimized based on the pairwise contrastive loss function to improve the matching of attribute words and opinion words.
4. The intelligent voice reading method and system based on sentiment analysis as claimed in claim 3, characterized in that: The specific formula for obtaining the description information representation is as follows: in, is the output of the [CLS] position, W aop and b aop is a learnable parameter, d aop is the paired information description, m represents the number of words, BERT(·) is the pre-trained bidirectional Transformer encoder function, LeakyReLU(·) is the activation function of the linear rectifier unit with leakage; The specific formula for characterizing the term pair is as follows: Among them, h a and h o are the dense representations of attribute words and opinion words respectively, f is an average pooling function, and W s and b s is a learnable parameter, I is the position index of the attribute word and opinion word in the comment text X, and H e is the output hidden representation of the T5 encoder, h c is a term pair representation.
5. The intelligent voice reading method and system based on sentiment analysis as claimed in claim 3, characterized in that: The execution steps of the emotional layer information enhancement are as follows: Defining a sentiment category set, wherein the sentiment category set includes three sentiment categories: positive, neutral and negative; Encode the emotional category description based on the BERT model and use a new linear layer to obtain emotional information representation; Extract the hidden representation of the real attribute word and opinion word pair from the output of the T5 encoder; The hidden representation of the true term pair is converted into a term pair representation through a linear layer; Using a contrastive learning method, calculating the similarity between the sentiment information representation and the term pair representation; Optimize the parameters of the baseline model for attribute-level sentiment triple generation based on the sentiment layer contrast loss function to improve its ability to understand and distinguish different sentiments. The emotional information representation d sen The specific formula is as follows: Among them, LeakyReLU(·) is the activation function of the linear rectifier unit with leakage, is the output of the [CLS] position, W sen and b sen are learnable parameters.
6. The intelligent voice reading method and system based on sentiment analysis as claimed in claim 5, characterized in that: The paired layer contrast loss function L sen The specific formula is as follows: in, is the set of representations of all true term pairs that do not belong to the current sentiment category, d sen To represent emotional information, is the real term pair information in the sequence, τ is the temperature coefficient, For real term information and the similarity of emotional information representation.
7. The intelligent voice reading method and system based on sentiment analysis according to claim 6, characterized in that: The specific formula of the comprehensive loss function L is as follows: L=αL e +βL aop +γL sen Among them, α, β, γ are weight hyperparameters, L e , L aop , L sen Generate loss function for triples, contrast loss function for pairing layer and contrast loss function for sentiment layer respectively.
8. An attribute-level sentiment triple extraction system based on pairing information enhancement, based on the sentiment analysis-based intelligent voice reading method and system according to any one of claims 1 to 7, characterized in that: Also includes, A text data acquisition module is used to obtain a review text dataset containing user reviews; Baseline model building module, used to build an attribute-level sentiment triple generation baseline model based on the encoder-decoder structure; A triple generation module is used to encode the comment text into a hidden representation using a T5 encoder, and based on the hidden representation, generate an attribute-level sentiment triple through a T5 decoder, wherein the attribute-level sentiment triple includes two output paradigms: annotative and extractive; Multi-level paired information enhancement module, used to perform multi-level paired information enhancement, using the contrastive learning of the pairing layer and the sentiment layer to calculate the pairing layer contrast loss and the sentiment layer contrast loss; The multi-task joint training module is used to combine the triple generation loss, pairing layer contrast loss and sentiment layer contrast loss to construct a comprehensive loss function and perform multi-task joint training; The attribute-level sentiment triplet generation module is used to use the trained attribute-level sentiment triplet generation baseline model to extract attribute-level sentiment triplets from new review texts.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the intelligent voice reading method and system based on sentiment analysis described in any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the intelligent voice reading method and system based on sentiment analysis described in any one of claims 1 to 7 are implemented.
Citation Information
Cited By
Attribute-level emotion triple extraction method based on pairing information enhancement
CN118916446A
Emotion ternary extraction model optimization method, recognition method, equipment and storage medium
CN120688570A
