An aspect-based sentiment analysis method and system

Abstract

This invention discloses an aspect-based sentiment analysis method and system. First, data preparation and processing of the dataset are performed. The processed dataset is then input into a multi-task learning model for feature extraction. The multi-task learning model is trained, utilizing aspect terminology extraction to assist in aspect sentiment classification, and outputting the extracted aspect terms and sentiment polarities. The multi-task learning model involves two typical sub-tasks: aspect terminology extraction and aspect polarity classification. The aspect terminology extraction task is used as an auxiliary task to assist in aspect sentiment polarity classification, thereby improving the performance of aspect polarity classification. Furthermore, a multi-head attention mechanism is used to link dependency sequences with aspect extraction, enabling the model to focus more on words closely related to aspects, significantly enhancing the performance of aspect polarity classification. This invention can be applied to sentiment analysis of reviews of food, goods, scenery, etc., providing more accurate and refined sentiment prediction services.

Description

Technical Field

[0001] This invention belongs to the field of sentiment analysis technology, specifically relating to an aspect-based sentiment analysis method and system. Background Technology

[0002] Sentiment analysis is an important field in natural language processing. Compared to traditional sentiment analysis, aspect-level sentiment analysis is a fine-grained sentiment analysis task that requires the model to automatically extract aspects and predict the sentiment polarity of all aspects. For example, in the sentence "I like the service at the restaurant, but the ambiance isn't very good," the aspect terms are "service" and "ambiance." The output sentiment polarities for these two aspects are positive and negative, respectively. Aspect-level sentiment analysis mainly includes two sub-tasks: aspect term extraction and aspect sentiment classification.

[0003] As a subtask of aspect-level sentiment analysis, aspect term extraction aims to identify the different aspects mentioned in a given sentence. Aspect terms refer to specific features or attributes of the product or service discussed in the review. The extraction of aspect terms can be viewed as a textual entity. Aspect term extraction methods have evolved from traditional methods to deep learning methods.

[0004] Aspect sentiment classification is essentially a text classification task: given a sentence and a particular aspect within it, predict its corresponding sentiment polarity. Compared to aspect term extraction, aspect sentiment classification has been extensively studied and is a crucial foundational research area in natural language processing. Methods for solving aspect sentiment classification have evolved from feature engineering to deep learning-based approaches, most commonly recurrent neural networks (RNNs) and convolutional neural networks (CNNs). In recent years, with the advent of pre-trained models, numerous studies have focused on unsupervised pre-trained language models (Bidirectional Encoder Representation from Transformers BERT) for natural language processing tasks, achieving excellent results on many tasks. BERT models are also well-suited for aspect sentiment classification. Furthermore, a sentence contains not only semantic information but also syntactic structural information, such as the sentence's dependency tree structure. This syntactic structure is helpful for aspect sentiment classification because it can capture sentiment words related to the aspect.

[0005] Most of the aforementioned methods focus on aspect sentiment classification tasks while neglecting the importance of aspect term extraction. This is particularly true when transferring models to new corpora, where most models fail to perform well due to a lack of aspect extraction capabilities. In such cases, a single aspect sentiment classification model first needs to leverage other models to extract aspects from sentences before classifying them; however, this approach does not utilize the features acquired during aspect extraction.

[0006] In recent years, many sentiment analysis methods have treated the dependency tree as an adjacency matrix and then used a graph neural network to encode the entire adjacency matrix to extract features from the dependency relationships. However, most studies directly input the dependency relationships of sentences into the graph neural network, which does not take into account the influence of factors and fails to highlight important dependency relationships. Summary of the Invention

[0007] The technical problem to be solved by this invention is to address the shortcomings of the prior art by providing an aspect-based sentiment analysis method and system. This method and system can utilize aspect term extraction tasks to assist in aspect sentiment classification tasks, realize information interaction between subtasks, and use multi-head attention to highlight important dependencies to provide more accurate sentiment polarity prediction. This addresses the technical problems of insufficient information interaction between subtasks and the inability to highlight important dependencies in the field of fine-grained sentiment analysis, as well as the lack of performance in aspect sentiment polarity classification.

[0008] The present invention adopts the following technical solution:

[0009] An aspect-based sentiment analysis method includes the following steps:

[0010] S1. Preprocess the data with sentiment polarity labels;

[0011] S2. Input the preprocessed data from step S1 into a multi-task learning model for sentiment analysis for feature extraction.

[0012] S3. Use the features extracted in step S2 to train a multi-task learning model for sentiment analysis. The trained multi-task learning model is then used to assist in the aspect-based sentiment classification task, and the extracted aspect terms and sentiment polarities are output to complete the sentiment analysis.

[0013] Specifically, in step S1, for the aspect term extraction task, the text is preprocessed using IOB tags, identifying each word in the sentence with a different tag, and then using B... asp The first word in terms of signage, I asp The internal words of the identifier are marked with 'O', while the rest of the words in the sentence are marked with 'O'.

[0014] Specifically, in step S1, for the aspect sentiment classification task, using...<T,P> The binary representation of aspects and the corresponding emotional polarity, T = (w i ,w i+1 ,…,w j P represents an aspect, where i and j represent the starting and ending positions of the aspect in the sentence, respectively. P = p, p ∈ {Positive, Negative, Neural}, including three emotional polarities: positive, negative, and neutral.

[0015] Specifically, in step S2, the multi-task learning model for sentiment analysis includes:

[0016] The global feature extraction module extracts global features of sentences and aspects based on the BERT model.

[0017] The relationship graph attention module encodes dependency relationship labels to extract dependency relationship features;

[0018] The aspect term extraction module uses the BERT model to extract aspects from sentences and provides features for multi-head attention computation with dependency sequence.

[0019] The multi-head attention module performs two subtasks: aspect term extraction and aspect sentiment classification. It integrates dependency labels into the features extracted by the BERT-ATE module and performs multiple scaling dot product attention calculations to obtain feature h. mha .

[0020] Furthermore, in the global feature extraction module, the extraction of global features for sentences and aspects based on the BERT model specifically involves:

[0021] The sentence and aspect are input into the model as a whole, separated by the character [SEP]. The input format is [CLS]+sentence+[SEP]+aspect+[SEP]. After feature extraction by the BERT model, a feature sequence h is obtained, where each feature in the sequence corresponds to a word in the input. h = (h... cls ,h1,h2,…,h n ,h sep ,h i ,h i+1 ,…,h j ,h sep ), h cls The pooling vector for the entire sentence, i.e., h apc , is used to characterize the features of the entire input.

[0022] Furthermore, in the relationship graph attention module, the extraction of dependency relationship features specifically involves:

[0023] For a sentence S, the parser first obtains the dependency relations, and the tree reconstruction algorithm obtains the dependency relation labels between words, constructing a dependency tree represented by a graph with n nodes, where each node represents a word in the sentence. Then, the RGAT module extracts the final dependency relation features h. rgat This yields the dependency sequence of the sentences, and the final dependency feature h. rgat as follows:

[0024]

[0025] Where M is the number of attention heads in RGAT, h j W is the feature vector corresponding to word j. m Let a be the weight matrix. ij N represents the elements in the range (0,1) calculated using the softmax function. i Let i be the domain word node of word node i.

[0026] Furthermore, in the aspect term extraction module, the sentence sequence S = (w1, w2, ..., w n The tag sequence Y = (t1, t2, ..., t) is obtained by processing the sentence sequence S with IOB tags. n ),t i ∈{O,B asp ,I asp Given [CLS], [SEP], and inputting [CLS] + W + [SEP] into the BERT model, a feature sequence F is obtained after encoding by the BERT model. Each feature in the sequence corresponds one-to-one with the input word: F = (F cls ,F1,F2,…,F n ,F sep ),F cls That is, h ate .

[0027] Specifically, the trained multi-task learning model includes two outputs: aspect terminology and aspect sentiment polarity. In the aspect terminology task, a linear layer is used to classify and obtain the probability p(l) of each word belonging to each label l. In the aspect sentiment polarity task, the outputs are concatenated to obtain the final feature h. f We use a linear layer to classify and obtain the probability p(c) of belonging to each sentiment category c.

[0028] Furthermore, in step S3, the probability p(l) of each word belonging to each label l and the probability p(c) of each sentiment category c are calculated as follows:

[0029]

[0030]

[0031] Among them, W l and b l These are model parameters, where L is the IOB tag set and W is... f and h f These are the model parameters, and C is the set of sentiment categories;

[0032] Loss function of a trained multi-task learning model for:

[0033]

[0034] Where α is a hyperparameter, The loss function for the sentiment classification task is... The loss function for extracting terms from the task.

[0035] Secondly, embodiments of the present invention provide an aspect-based sentiment analysis system, including:

[0036] The preprocessing module preprocesses the data labeled with sentiment polarity.

[0037] The feature module takes the data preprocessed by the preprocessing module and inputs it into a multi-task learning model for sentiment analysis to extract features.

[0038] The analysis module uses the features extracted by the feature module to train a multi-task learning model for sentiment analysis. The trained multi-task learning model is then used to assist in the aspect-based sentiment classification task using aspect terms extraction task, and the extracted aspect terms and sentiment polarity are output to complete the sentiment analysis.

[0039] Compared with the prior art, the present invention has at least the following beneficial effects:

[0040] This invention presents an aspect-based sentiment analysis method that simultaneously learns two sub-tasks—aspect term extraction and aspect sentiment polarity classification—through multi-task learning, enabling information interaction between tasks. The aspect term extraction task is introduced as an auxiliary in the aspect sentiment polarity classification task, taking into account both global and local features. Furthermore, a multi-head attention mechanism is used to link dependency sequences with aspect extraction, allowing the model to focus more on words closely related to aspects, further enhancing local features and significantly improving the performance of aspect sentiment classification. In the aspect term extraction task, IOB tags are used to preprocess the text, identifying each word in the sentence with a different tag. asp The first word in terms of signage, I aspThe internal words of the aspect term are labeled, while the remaining words in the sentence are labeled with 'O'. Treating the aspect term extraction task as a sequence labeling task and jointly training it with the aspect sentiment classification task enables information interaction between subtasks, which can significantly improve the performance of the aspect sentiment classification task. Furthermore, in the aspect sentiment classification task, using...<T,P> The binary representation of aspects and the corresponding emotional polarity, T = (w i ,w i+1 ,…,w j Let P represent an aspect, where i and j represent the start and end positions of the aspect in the sentence, respectively. P = p, p ∈ {Positive, Negative, Neural}, and there are three sentiment polarities: positive, negative, and neutral. The aspect sentiment classification task is used as the primary task, and auxiliary tasks are employed to improve the accuracy of the sentiment polarity predicted by the aspect sentiment classification task.

[0041] Furthermore, the aspect-based sentiment analysis multi-task model comprises four parts: BERT-APC (Global Feature Extraction Module), RGAT (Graph Attention Module), BERT-ATE (Aspect Term Extraction Module), and Multi-head Attention Module. These four modules can comprehensively extract sentence features from multiple perspectives, taking into account both global and local features, achieving feature fusion and enhancement. Specifically, the BERT-APC global feature extraction module extracts global features of sentences and aspects based on the BERT model. The sentence and aspect are input as a whole into the model, separated by the [SEP] character, using the form [CLS]+sentence+[SEP]+aspect+[SEP] as input. After BERT extracts features, a feature sequence h is obtained, where each feature in the sequence corresponds to each word in the input, h = (h... cls ,h1,h2,…,h n ,h sep ,h i ,h i+1 ,…,h j ,h sep ), h cld It is the pooling vector of the entire sentence, that is, h. apc This is used to represent the global features of the entire input, so as to facilitate subsequent feature enhancement and fusion, thereby improving the performance of sentiment classification tasks.

[0042] Furthermore, the RGAT relation graph attention module can encode dependency relation labels to extract dependency relation features. For a sentence D, the parser first obtains the dependency relations, and the tree reconstruction algorithm obtains the dependency relation labels between words, constructing a dependency tree represented by a graph with n nodes, where each node represents a word in the sentence. Then, the RGAT module extracts the final dependency relation features h. rgat The dependency sequence of the sentence is obtained. The relation graph attention module can focus on words that are more relevant to the aspect, highlight important dependency relationships, and extract local features of the sentence to facilitate subsequent feature enhancement and fusion.

[0043] Furthermore, the BERT-ATE aspect term extraction module primarily uses BERT to extract aspects from sentences and provides a feature for multi-head attention computation with dependency sequences; for sentence sequences S = (w1, w2, ..., w n After processing with IOB tags, the resulting tag sequence is Y = (t1, t2, ..., t...). n ),t i ∈{O,B asp ,I asp Given the sequence "[CLS]++[SEP]", inputting "[CLS]++[SEP]" into BERT results in a feature sequence F after BERT encoding. Each feature in the sequence corresponds one-to-one with the input word: F = (F... cls ,F1,F2,…,F n ,F sep ), F cls That is, h ate The BERT-ATE aspect term extraction module can extract aspects from sentences to achieve aspect term extraction tasks and provides an aspect-based feature.

[0044] Furthermore, the multi-task learning model includes two outputs: aspect terminology and aspect sentiment polarity. In the aspect terminology task, a linear layer is used to classify and obtain the probability p(l) of each word belonging to each label l. In the aspect sentiment polarity task, the outputs are concatenated to obtain the final feature h. f h f =[h apc h ate h mh h rgat The model is classified using a linear layer to obtain the probability p(c) of belonging to each sentiment category c. The model's performance can be validated, quantitative results can be generated, and the model can be optimized by predicting the output of aspect terms and aspect sentiment polarities.

[0045] Furthermore, by calculating the probabilities, the predicted outputs of aspect term extraction and aspect sentiment classification can be obtained. These outputs can be compared with the correct outputs, and the model can be continuously optimized by calculating the loss function to help the aspect sentiment classification task achieve better performance and improve the accuracy of polarity classification.

[0046] It is understandable that the beneficial effects of the second aspect mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0047] In summary, this invention can utilize aspect term extraction tasks to assist in aspect sentiment classification tasks, realize the interaction between aspects and dependencies, and provide more accurate sentiment polarity prediction.

[0048] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0050] Figure 2 This is a schematic diagram of the model structure in an embodiment of the present invention. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] In the description of this invention, it should be understood that the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0053] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0054] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Additionally, the character " / " in this document generally indicates that the preceding and following objects have an "or" relationship.

[0055] It should be understood that although terms such as first, second, third, etc., may be used in the embodiments of the present invention to describe the preset range, these preset ranges should not be limited to these terms. These terms are only used to distinguish the preset ranges from one another. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0056] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0057] The accompanying drawings illustrate various structural schematic diagrams according to embodiments disclosed in this invention. These drawings are not to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.

[0058] This invention provides an aspect-based sentiment analysis method. Through multi-task learning, it simultaneously learns two sub-tasks: aspect terminology extraction and aspect sentiment classification, achieving information interaction between tasks. An aspect terminology extraction task is introduced as an auxiliary in the aspect sentiment classification task. During aspect extraction, the model pays more attention to words related to aspects, which is helpful for aspect sentiment classification. This invention employs four modules to comprehensively extract sentence features from multiple perspectives, significantly improving the accuracy of aspect sentiment classification. It can be applied to reviews of food, goods, scenery, etc., providing more accurate sentiment prediction services.

[0059] Please see Figure 1 The present invention provides an aspect-based sentiment analysis method, comprising the following steps:

[0060] S1. Preprocess the data with sentiment polarity labels;

[0061] The labels, dependencies, sentiment polarity, and location information are extracted, and the training and test sets are reconstructed. The training set will be used when training the multi-task learning model for sentiment analysis, and the test set will be used when validating the model.

[0062] In the aspect term extraction task, text is preprocessed using IOB tags, identifying each word in a sentence with a different tag, and then using B... asp The first word in terms of signage, I asp The internal words of the identifier are marked with 'O', while the rest of the words in the sentence are marked with 'O'.

[0063] In the emotion classification task, using<T,P> The binary representation of aspects and the corresponding emotional polarity, T = (w i ,w i+1 ,…,w j P represents an aspect, where i and j represent the starting and ending positions of the aspect in the sentence, respectively. P = p, p ∈ {Positive, Negative, Nrural}, and there are three emotional polarities: positive, negative, and neutral.

[0064] S2. Input the preprocessed data into a multi-task learning model for sentiment analysis for feature extraction;

[0065] Please see Figure 2 The multi-task learning model for sentiment analysis consists of four modules: BERT-APC (Global Feature Extraction Module), RGAT (Graph Attention Module), BERT-ATE (Aspect Term Extraction Module), and Multi-head Attention (Multi-head Attention Module).

[0066] The BERT-APC global feature extraction module is responsible for extracting global features from sentences and aspects using the BERT model. First, the sentence and aspect are input as a whole into the model, separated by the [SEP] character. The input format is "[CLS]+sentence+[SEP]+aspect+[SEP]". After BERT extracts the features, a feature sequence h is obtained, where each feature in the sequence corresponds to a word in the input. h = (h... cls ,h1,h2,…,h n ,h sep ,h i ,h i+1 ,…,h j ,h sep ), h cls It is the pooling vector of the entire sentence, that is, h. apc , is used to characterize the features of the entire input.

[0067] The RGAT graph attention module encodes dependency relation labels to extract dependency relation features. For a sentence S, the parser first obtains its dependencies, and the tree reconstruction algorithm obtains the dependency relation labels between words. Then, the RGAT module extracts the final dependency relation features h. rgat This yields the dependency sequence of the sentences, and the final dependency features are calculated using the following formula:

[0068] g jj =σ(relu(r) ij W m1 +b m1 W m2 +b m2 )

[0069]

[0070]

[0071] Among them, W m1 and W m2 These are the weight matrices, b m1 and b m2 These are the bias terms, r ij h is a label representing the relationship between words i and j. j Let be the feature vector corresponding to word j, calculated by the BERT-APC module, and M be the number of attention heads in RGAT. Concatenate the M calculation results, h rgat These are the features extracted by the relation graph attention module.

[0072] The BERT-ATE aspect term extraction module primarily uses BERT to extract aspects from sentences and provides a feature for multi-head attention computation with dependency sequences. Assume there is a sentence sequence S = (w1, w2, ..., w...). n The tag sequence Y = (t1, t2, ..., t) after processing it with IOB tags. n ),t i ∈{O,B asp ,I asp Given "[CLS]+W+[SEP]", input "[CLS]+W+[SEP]" into BERT. After BERT encoding, a feature sequence F is obtained, where each feature in the sequence corresponds one-to-one with the input word: F = (F cls ,F1,F2,…,F n ,F sep ),F cls That is, h ate .

[0073] The multi-head attention module is used to connect the two subtasks of aspect term extraction and aspect sentiment classification. It integrates dependency labels into the features extracted by the BERT-ATE module and performs multiple scaling dot product attention calculations to obtain feature h. mha This better highlights the important dependency relationship labels.

[0074] To connect the two subtasks of aspect term extraction and aspect sentiment classification and highlight the important dependencies, a multi-head attention mechanism was used, which allows the model to pay better attention to aspect-related words, thereby improving the performance of aspect sentiment classification.

[0075] S3. Train a multi-task learning model for sentiment analysis, complete the information exchange between aspect term extraction and aspect sentiment classification tasks, use the aspect term extraction task to assist in the aspect sentiment classification task, and output the extracted aspect terms and sentiment polarity.

[0076] The multi-task learning model for sentiment analysis has two outputs: aspect terms and aspect sentiment polarity. In the aspect term (ATE) task, the BERT-ATE aspect term extraction module obtains the feature sequence corresponding to each word, and then uses a linear layer to classify the probability of each word belonging to each label class l.

[0077]

[0078] Among them, W l and b l These are model parameters, where L is the IOB tag set, L = {O, B}. asp ,I asp ,[CLS],[SEP]}.

[0079] The four modules of the multi-task learning model for sentiment analysis each extract a feature. In the aspect-based sentiment polarity (APC) task, these four features are concatenated to obtain the final feature h. f ,h f =[h apc h ate h mha h rgat ;] and classify it using a linear layer to obtain the probability that it belongs to each sentiment category c:

[0080]

[0081] Among them, W f and h f These are the model parameters, and C is the set of sentiment categories, where C = Positive, Negative, Neutral.

[0082] This multi-task learning model optimizes the loss functions of two tasks simultaneously. Both sub-tasks use the cross-entropy loss function. The loss function for the aspect term extraction task is:

[0083]

[0084] Where N is the number of categories of sequence label tags, and k is the number of characters in the input sequence.

[0085] The loss function for the sentiment classification task is:

[0086]

[0087] Where C is the set of emotion categories.

[0088] The final loss function is:

[0089]

[0090] Here, α is a hyperparameter that represents the proportional relationship between the losses of the two tasks.

[0091] S4. Validate the model using two metrics: accuracy and macro average F1 score. The higher the accuracy and macro average F1 score, the better the model's performance.

[0092] The accuracy rate is calculated as follows:

[0093]

[0094] Among them, T pre C represents the total number of predictions. pre This indicates the number of correct predictions.

[0095] Calculating the macro-average F1 score first requires calculating the F1 score for each class and then averaging them. For multi-class classification, precision, recall, and F1 score are all calculated for a single class. The precision, recall, and F1 score for each class are calculated first, as shown below:

[0096]

[0097]

[0098]

[0099] Where A∈{Positive, Negative, Neutral}, C A T represents the number of classes correctly predicted as A. A1 T represents the total number of cases predicted as class A. A2This actually represents the total number of items in category A.

[0100] The macro average F1 calculation method is as follows:

[0101]

[0102] Where n is the number of categories, F1 i The F1 score for each category.

[0103] In another embodiment of the present invention, an aspect-based sentiment analysis system is provided, which can be used to implement the above-mentioned aspect-based sentiment analysis method. Specifically, the aspect-based sentiment analysis system includes a preprocessing module, a feature module, and an analysis module.

[0104] The preprocessing module preprocesses the data with sentiment polarity annotations.

[0105] The feature module takes the data preprocessed by the preprocessing module and inputs it into a multi-task learning model for sentiment analysis to extract features.

[0106] The analysis module uses the features extracted by the feature module to train a multi-task learning model for sentiment analysis. The trained multi-task learning model is then used to assist in the aspect-based sentiment classification task using aspect terms extraction task, and the extracted aspect terms and sentiment polarity are output to complete the sentiment analysis.

[0107] In another embodiment of the present invention, a terminal device is provided, comprising a processor and a memory. The memory stores a computer program, the computer program including program instructions, and the processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to implement a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used for the operation of aspect-based sentiment analysis methods, including:

[0108] The data with sentiment polarity labels are preprocessed; the preprocessed data is input into a multi-task learning model for sentiment analysis for feature extraction; the extracted features are used to train the multi-task learning model for sentiment analysis, and the trained multi-task learning model is obtained. The aspect term extraction task is used to assist in the aspect sentiment classification task, and the extracted aspect terms and sentiment polarities are output to complete the sentiment analysis.

[0109] In another embodiment of the present invention, a storage medium is also provided, specifically a computer-readable storage medium (memory). This computer-readable storage medium is a memory device in a terminal device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and extended storage media supported by the terminal device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device.

[0110] One or more instructions stored in a computer-readable storage medium can be loaded and executed by a processor to implement the corresponding steps of the aspect-based sentiment analysis method in the above embodiments; one or more instructions in the computer-readable storage medium are loaded and executed by the processor to perform the following steps:

[0111] The data with sentiment polarity labels are preprocessed; the preprocessed data is input into a multi-task learning model for sentiment analysis for feature extraction; the extracted features are used to train the multi-task learning model for sentiment analysis, and the trained multi-task learning model is obtained. The aspect term extraction task is used to assist in the aspect sentiment classification task, and the extracted aspect terms and sentiment polarities are output to complete the sentiment analysis.

[0112] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0113] This method has been experimentally validated on three commonly used public datasets: Restaurant14, Laptop14, and Twitter. Restaurant14 and Laptop14 contain sentiment reviews in the restaurant and laptop domains, while the Twitter dataset mainly consists of tweets. The sentiment polarity in these datasets includes three categories: Positive, Negative, and Neutral.

[0114] The present invention is compared with the most advanced relevant models at home and abroad, and the accuracy and macro-average F1 are used as evaluation indicators. The comparison results are shown in Table 1:

[0115] Table 1. Experimental Results Comparing Methods

[0116]

[0117] In Table 1, w / o BERT means without BERT, meaning the model does not use BERT, while w BERT means with BERT, meaning the model uses the pre-trained BERT model.

[0118] As can be seen from Table 1, the present invention outperforms other models in both Accuracy and Macro-average on the Restaurant14, Laptop14 and Twitter datasets, proving the effectiveness of the present invention. It can better establish the connection between aspect term extraction and aspect sentiment classification, thereby enhancing the performance of aspect sentiment classification.

[0119] In summary, this invention presents an aspect-based sentiment analysis method and system, involving two typical subtasks: aspect term extraction and aspect polarity classification. The aspect term extraction task is used as an auxiliary task to assist in aspect sentiment polarity classification, thereby improving the classification performance. Furthermore, a multi-head attention mechanism is utilized to link dependency sequences with aspect extraction, enabling the model to focus more on words closely related to aspects, significantly enhancing aspect polarity classification performance. This invention can be applied to sentiment analysis of evaluations of food, goods, scenery, etc., providing more accurate and refined sentiment prediction services.

[0120] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0121] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0122] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0123] In the embodiments provided by this invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0124] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0125] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0126] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0127] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0128] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0129] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0130] The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. An aspect-based sentiment analysis method, characterized in that, Includes the following steps: S1. Preprocess the data with sentiment polarity labels; S2. Input the preprocessed data from step S1 into a multi-task learning model for sentiment analysis for feature extraction. S3. Using the features extracted in step S2, train a multi-task learning model for sentiment analysis. The trained model is then used to assist in aspect-based sentiment classification, outputting the extracted aspect terms and sentiment polarities to complete the sentiment analysis. The trained multi-task learning model includes two outputs: aspect terms and aspect-based sentiment polarities. In the aspect term task, a linear layer is used to classify each word to determine its category. probability In the aspect-based emotion polarity task, the final features are obtained by piecing together. Classification is performed using linear layers to obtain the values ​​belonging to each emotion category. probability ; Each word belongs to various tags. probability and each emotion category probability The calculations are as follows: in, and These are model parameters. It is a collection of IOB tags. These are model parameters. It is a collection of emotion categories; Loss function of a trained multi-task learning model for: in, For hyperparameters, The loss function for the sentiment classification task is... The loss function for extracting terms from the task.

2. The aspect-based sentiment analysis method according to claim 1, characterized in that, In step S1, for the aspect term extraction task, the text is preprocessed using IOB tags, identifying each word in the sentence with a different tag, and then... The first word in terms of signage, The internal words of the identifier, the rest of the words in the sentence are used Logo.

3. The aspect-based sentiment analysis method according to claim 1, characterized in that, In step S1, for the aspect sentiment classification task, using The representation of binary pairs and their corresponding emotional polarities To indicate one aspect, These indicate the beginning and end positions of an aspect in a sentence, respectively. It includes three emotional polarities: positive, negative, and neutral.

4. The aspect-based sentiment analysis method according to claim 1, characterized in that, In step S2, a multi-task learning model for sentiment analysis is developed. include: The global feature extraction module extracts global features of sentences and aspects based on the BERT model. The relationship graph attention module encodes dependency relationship labels to extract dependency relationship features; The aspect term extraction module uses the BERT model to extract aspects from sentences and provides features for multi-head attention computation with dependency sequence. The multi-head attention module performs two subtasks: aspect term extraction and aspect sentiment classification. It integrates dependency relation labels into the features extracted by the BERT-ATE module and performs multiple scaling dot product attention calculations to obtain the final features. .

5. The aspect-based sentiment analysis method according to claim 4, characterized in that, In the global feature extraction module, the extraction of global features for sentences and aspects based on the BERT model is specifically as follows: Input sentences and aspects as a whole into the model, separating sentences and aspects with the [SEP] character, and use... The input is a given form, which is then processed by the BERT model to extract features, resulting in a feature sequence. Each feature in the sequence corresponds to each word in the input. , The pooling vector for the entire sentence, i.e. , is used to characterize the features of the entire input.

6. The aspect-based sentiment analysis method according to claim 4, characterized in that, In the relation graph attention module, the extraction of dependency relation features specifically involves: For a sentence First, the parser obtains the dependency relations, and the tree reconstruction algorithm obtains the dependency relation labels between words, constructing a dependency tree represented by a graph with n nodes, where each node represents a word in the sentence. Then, the RGAT module extracts the final dependency relation features. This yields the dependency sequence of the sentences, and the final dependency features. as follows: in, The number of attention heads in RGAT. For words The corresponding feature vector, This is the weight matrix. The elements in the range (0,1) obtained by using the softmax function. Let i be the domain word node of word node i.

7. The aspect-based sentiment analysis method according to claim 4, characterized in that, In the terminology extraction module, sentence sequence For sentence sequence Tag sequence after processing with IOB tags ,Will The input is fed into the BERT model, and after encoding by the BERT model, a feature sequence is obtained. Each feature in the sequence corresponds one-to-one with an input word: , Right now .

8. An aspect-based sentiment analysis system, characterized in that, include: The preprocessing module preprocesses the data labeled with sentiment polarity. The feature module takes the data preprocessed by the preprocessing module and inputs it into a multi-task learning model for sentiment analysis to extract features. The analysis module uses features extracted by the feature module to train a multi-task learning model for sentiment analysis. This trained model then uses an aspect term extraction task to assist in aspect-based sentiment classification, outputting the extracted aspect terms and sentiment polarities to complete the sentiment analysis. The trained multi-task learning model includes two outputs: aspect terms and aspect-based sentiment polarities. In the aspect term task, a linear layer is used to classify each word to determine its category. probability In the aspect-based emotion polarity task, the final features are obtained by piecing together. Classification is performed using linear layers to obtain the values ​​belonging to each emotion category. probability ; Each word belongs to various tags. probability and each emotion category probability The calculations are as follows: in, and These are model parameters. It is a collection of IOB tags. These are model parameters. It is a collection of emotion categories; Loss function of a trained multi-task learning model for: in, For hyperparameters, The loss function for the sentiment classification task is... The loss function for extracting terms from the task.

Images