Attribute-level sentiment classification method and equipment

Abstract

The embodiment of the invention discloses an attribute-level sentiment classification method and device, and relates to the technical field of natural language processing. The method comprises the following steps: obtaining a text and context representations of attribute words in the text, a context distance between each word and the attribute word in the text and a context distance representation thereof, a syntactic dependency distance and a syntactic dependency distance representation thereof, and a semantic perception distance and a semantic perception distance representation thereof; according to a context distance, a syntax dependency distance and a semantic perception distance between each word and the attribute word in the text, obtaining a multi-distance perception representation; according to the text, the context representation of the attribute word in the text, the context distance representation between each word and the attribute word in the text, the syntax dependency distance representation and the semantic perception distance representation, obtaining semantic noise perception representation; and obtaining the sentiment polarity of the attribute word according to the multi-distance perception representation and the semantic noise perception representation.

Description

Technical Field

[0001] The present disclosure relates to the technical field of natural language processing, and in particular to an attribute-level sentiment classification method, an attribute-level sentiment classification device, a computer device, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the advent of the mobile Internet era, users are increasingly inclined to publicly express their opinions and comments on Internet social platforms and online shopping websites. It is important to study how to extract valuable information from these opinions and comments. In order to solve this problem, sentiment analysis methods are considered to be an effective solution and have been successfully applied to public opinion supervision, public opinion surveys, word-of-mouth analysis, and market sentiment analysis.

[0003] Sentiment analysis methods can be generally divided into document-level sentiment analysis methods, sentence-level sentiment analysis methods, and attribute-level sentiment analysis methods according to their granularity. However, document-level sentiment analysis methods and sentence-level sentiment analysis methods only analyze the overall sentiment tendency of the text and cannot obtain the sentiment tendency of specific entities (attribute words) in the text. Summary of the invention

[0004] The disclosed embodiment provides an attribute-level sentiment classification method, which includes: obtaining a text and a context representation of attribute words in the text, a context distance between each word in the text and the attribute word and its context distance representation, a syntactic dependency distance and its syntactic dependency distance representation, and a semantic perception distance and its semantic perception distance representation; obtaining a multi-distance perception adjacency matrix based on the context distance, syntactic dependency distance, and semantic perception distance between each word in the text and the attribute word; obtaining a multi-distance perception representation based on the multi-distance perception adjacency matrix; obtaining a semantic noise perception representation based on the text and the context representation of the attribute words in the text, the context distance representation, syntactic dependency distance representation, and semantic perception distance representation between each word in the text and the attribute word; obtaining a multi-information aggregated sentiment representation based on the multi-distance perception representation and the semantic noise perception representation; obtaining the sentiment polarity of the attribute word based on the multi-information aggregated sentiment representation.

[0005] The disclosed embodiment provides an attribute-level sentiment classification device, which includes: a multi-semantic and distance embedding module, which is used to obtain a context representation of a text and an attribute word in the text, a context distance between each word in the text and the attribute word and its context distance representation, a syntactic dependency distance and its syntactic dependency distance representation, and a semantic perception distance and its semantic perception distance representation; a multi-distance perception adjacency matrix module, which is used to obtain a multi-distance perception adjacency matrix according to the context distance, syntactic dependency distance, and semantic perception distance between each word in the text and the attribute word; a semantic and syntactic perception module, which includes a semantic noise perception attention submodule and a multi-distance perception adjacency matrix module; The invention discloses a distance-aware graph convolutional network, wherein the semantic noise-aware attention submodule obtains a semantic noise-aware representation according to the contextual representation of the text and the attribute words in the text, the contextual distance representation between each word in the text and the attribute words, the syntactic dependency distance representation and the semantic-aware distance representation; the multi-distance-aware graph convolutional network is used to obtain a multi-distance-aware representation according to the multi-distance-aware adjacency matrix; the attribute-aware aggregation module is used to obtain a multi-information aggregated sentiment representation according to the multi-distance-aware representation and the semantic noise-aware representation; the sentiment prediction module is used to obtain the sentiment polarity of the attribute words according to the multi-information aggregated sentiment representation.

[0006] An embodiment of the present disclosure provides a computer device, including a processor, a memory, and an input-output interface; the processor is connected to the memory and the input-output interface, respectively, wherein the input-output interface is used to receive data and output data, the memory is used to store a computer program, and the processor is used to call the computer program so that the computer device including the processor executes the attribute-level sentiment classification method in any embodiment of the present disclosure.

[0007] An embodiment of the present disclosure provides a computer-readable storage medium storing a computer program, wherein the computer program is suitable for being loaded and executed by a processor so that a computer device having the processor executes the attribute-level sentiment classification method in any embodiment of the present disclosure.

[0008] The embodiments of the present disclosure provide a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in various optional modes in any embodiment of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 It is a network interaction architecture diagram of an attribute-level sentiment classification method provided by an embodiment of the present disclosure.

[0010] Figure 2 It is a flowchart of an attribute-level sentiment classification method provided by an embodiment of the present disclosure.

[0011] Figure 3 It is a flowchart of another attribute-level sentiment classification method provided by an embodiment of the present disclosure.

[0012] Figure 4 Schematic diagram of an attribute-level sentiment classification device provided in an embodiment of the present disclosure.

[0013] Figure 5 is a schematic diagram of another attribute-level sentiment classification device provided in an embodiment of the present disclosure.

[0014] Figure 6 It is a schematic diagram of another attribute-level sentiment classification device provided in an embodiment of the present disclosure.

[0015] Figure 7 It is a structural diagram of a computer device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0017] In the embodiments of the present disclosure, see Figure 1 , Figure 1 1 is a network interaction architecture diagram of an attribute-level sentiment classification method provided by an embodiment of the present disclosure, and the embodiment of the present disclosure can be implemented by a terminal and / or a server. The terminal can obtain data from a computer device 101 and display the data, wherein the computer device 101 can exchange data with the terminal, and the computer device 101 can be a server where an application is located, or can belong to a terminal (i.e., the background of the terminal), etc., and no limitation is made here.

[0018] Among them, the terminal can be terminal 102a, terminal 102b or terminal 102c, etc., and the embodiment of the present disclosure can be implemented by any one of the terminals 102a, terminal 102b or terminal 102c, etc. Specifically, taking terminal 102b as an example, terminal 102b can be used for display. The data displayed by terminal 102b (such as the sentiment polarity of the attribute word) can be obtained from the computer device 101, and the computer device 101 can be the server corresponding to the target application, or the background of the terminal 102b, etc., which is not limited here.

[0019] The terminal may be a mobile phone (such as terminal 102c) or a laptop (such as terminal 102b), or a playback device in a vehicle (such as terminal 102a), etc., without limitation. The terminal 102a may be considered as a playback device in the vehicle 103, and the target application may be displayed in the terminal 102a. Figure 1 The terminals in the example are only some of the devices. In the present disclosure, the terminals are not limited to the Figure 1 The target application in the present disclosure may be any application program capable of displaying information / data.

[0020] It is understandable that the terminal mentioned in the embodiments of the present disclosure may be a computer device, and the computer device in the embodiments of the present disclosure includes but is not limited to a terminal or a server. In other words, the computer device may be a server or a terminal, or a system consisting of a server and a terminal. Among them, the terminal mentioned above may be an electronic device, including but not limited to a mobile phone, a tablet computer, a desktop computer, a laptop computer, a PDA, a vehicle-mounted device, an augmented reality / virtual reality (AR / VR) device, a helmet display, a smart TV, a wearable device, a smart speaker, a digital camera, a camera, and other mobile Internet devices (mobile internet device, MID) with network access capabilities, or terminals in scenarios such as trains, ships, and flights.

[0021] Among them, the servers mentioned above can be independent physical servers, or they can be server clusters or distributed systems composed of multiple physical servers. They can also be cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, vehicle-road collaboration, content delivery networks (Content Delivery Network, CDN), as well as big data and artificial intelligence platforms.

[0022] Optionally, the data involved in the embodiments of the present disclosure may be stored in a computer device, or the data may be stored based on cloud storage technology, which is not limited here.

[0023] In order to overcome the problem that document-level sentiment analysis and sentence-level sentiment analysis in related technologies cannot obtain the sentiment tendency of attribute words in the text, the present disclosure attempts to use attribute-level sentiment classification methods to solve this problem. Attribute-level sentiment classification is a fine-grained sentiment analysis task, and its goal is to identify the sentiment polarity of attribute words in texts such as sentences or paragraphs. Attribute-level sentiment analysis can provide fine-grained sentiment information and can better meet people's needs. The goal of attribute-level sentiment classification can be to extract the sentiment polarity of the attribute word given a sentence and an attribute word (also called an opinion object). Sentiment polarity can be divided into positive, negative, and neutral.

[0024] The goal of sentence-level and document-level sentiment analysis is to determine the sentiment polarity of the entire sentence or document. They cannot handle scenarios where there are multiple attribute words with different sentiment polarities in a sentence or document. Attribute-level sentiment classification can better handle this type of fine-grained sentiment classification.

[0025] For example, a user's comment on a restaurant is "This restaurant has a good location, but poor service." The sentiment polarity of the attribute word "location" in the comment is positive, while the sentiment polarity of the attribute word "service" is negative.

[0026] There are three types of attribute-level sentiment classification methods: rule-based methods, machine learning-based methods, and deep learning-based methods. Among them, rule-based methods use manually defined rules or sentiment dictionaries to identify the sentiment polarity of attribute words. Such methods do not perform well in scenarios where sentiment resources are limited. Machine learning-based methods use statistical machine learning methods (e.g., support vector machines, maximum entropy, naive Bayes, etc.) to extract sentiment polarity. However, the performance of such methods is heavily dependent on the quality of the designed features, and designing features is a very time-consuming and labor-intensive task.

[0027] The disclosed embodiment adopts a deep learning-based method, which can automatically learn features suitable for specific tasks and achieve good performance.

[0028] Figure 2 It is a flowchart of an attribute-level sentiment classification method provided by an embodiment of the present disclosure. Figure 2 The method provided in the embodiment can be executed by any computer device, such as the above Figure 1 Any terminal and / or server in the embodiment is executed for illustration. Figure 2 As shown, the method provided by the embodiment of the present disclosure may include the following steps.

[0029] In S210, a text and context representations of attribute words in the text are obtained, and the context distance between each word in the text and the attribute word and its context distance representation, the syntactic dependency distance and its syntactic dependency distance representation, and the semantic perception distance and its semantic perception distance representation are obtained.

[0030] The text in the embodiment of the present disclosure refers to the text whose sentiment polarity of the attribute words contained therein is to be classified. Sentiment polarity, also known as the duality or complexity of emotion, refers to the fact that emotion is generated by the interaction between a person's subjective experience, consciousness and objective environment. In the embodiment of the present disclosure, sentiment polarity includes three categories: positive, negative, and neutral. The text can be a paragraph, a document, or a sentence. The text can contain multiple words, and the multiple words contained therein include one or more attribute words. If the text contains multiple attribute words, for example, assuming that it contains q attribute words, q is a positive integer greater than 1, then the method provided by the embodiment of the present disclosure can be used each time to process the text and one attribute word in the text, and identify the sentiment polarity of one attribute word in the text at a time. If it is necessary to identify the sentiment polarity of each of the q attribute words in the text, the method provided by the embodiment of the present disclosure can be called q times, and the sentiment polarity of a single attribute word can be predicted each time.

[0031] In the disclosed embodiment, after obtaining the text to be classified, the attribute word to be classified is obtained from the text, and the attribute word to be classified may be any one of the attribute words contained in the text. Attribute words refer to words that describe a certain attribute of a viewpoint object in a text such as a sentence or a paragraph. Attribute words may be words that evaluate various things such as certain commodities, events or people in a review text. For example, in the restaurant review "The food in this restaurant is really delicious", the attribute word is "food". For another example, in a laptop computer review, "The battery capacity of this laptop computer is very large", the attribute word is "battery capacity".

[0032] In the embodiments of the present disclosure, the "representation" in the embodiments of the present disclosure may also be referred to as a representation, which may be a vector or a matrix. The contextual representation of a text refers to the word vector / vector representation of each word in the text obtained by vectorizing each word in the text. The contextual representation of an attribute word refers to the word vector / vector representation of each word in the attribute word. The word vector of each word is obtained based on the context in which the word is located, so it is also called a contextual representation.

[0033] In the disclosed embodiment, the context distance is used to indicate the distance relationship between each word in the text and the position of the attribute word in the text, which reflects the closeness between each word in the text and the attribute word. It can be considered that in the text, the closer the context distance is, the more relevant the word is to the attribute word, and vice versa, the less likely the relevance is. The context distance representation refers to the vector representation of the context distance between each word in the text and the attribute word.

[0034] In the disclosed embodiments, the syntactic dependency distance is used to indicate the syntactic or grammatical distance relationship between each word in the text and the attribute word obtained through syntactic dependency analysis, which can be used to reflect the syntactic or grammatical closeness between each word in the text and the attribute word. It can be considered that in the text, the smaller the syntactic dependency distance, the more syntactic or grammatical correlation the word has with the attribute word, and vice versa, the less likely the correlation is. The syntactic dependency distance representation refers to the vector representation of the syntactic dependency distance between each word in the text and the attribute word.

[0035] In the disclosed embodiments, the semantically perceived distance is used to indicate the semantic distance relationship between each word in the text and the attribute word, obtained by analyzing the semantics of the attribute word and the semantics of each word in the text, which can be used to reflect the semantic closeness between each word in the text and the attribute word. It can be considered that in the text, the smaller the semantically perceived distance, the more semantically related the word is to the attribute word, and vice versa. The possibility of association is smaller. The semantically perceived distance representation refers to the vector representation of the semantically perceived distance between each word in the text and the attribute word.

[0036] In an exemplary embodiment, the context distance and context distance representation, syntactic dependency distance and syntactic dependency distance representation, and semantic perception distance and semantic perception distance representation between each word in the text and the attribute word are obtained, including: splicing the first character and the second character at both ends of the attribute word and the text, respectively, to obtain the splicing sequence of the attribute word and the splicing sequence of the text, the length of the splicing sequence of the attribute word is m+2, m is the length of the attribute word; the length of the splicing sequence of the text is n=m s +2, m s is the length of the text, m and m sare all positive integers greater than or equal to 1, and n is a positive integer greater than or equal to 3; the context distance between each word in the concatenated sequence of the text and the attribute word is calculated; according to the context distance between each word in the concatenated sequence of the text and the attribute word and the context distance embedding matrix, the context distance representation of each word in the concatenated sequence of the text is obtained; the syntactic dependency distance between each word in the concatenated sequence of the text and the attribute word is calculated; according to the syntactic dependency distance between each word in the concatenated sequence of the text and the attribute word and the syntactic dependency distance embedding matrix, the syntactic dependency distance representation of each word in the concatenated sequence of the text is obtained; the semantic perception distance between each word in the concatenated sequence of the text and the attribute word is calculated; according to the semantic perception distance between each word in the concatenated sequence of the text and the attribute word and the semantic perception distance embedding matrix, the semantic perception distance representation of each word in the concatenated sequence of the text is obtained.

[0037] In an exemplary embodiment, a contextual representation of a text and attribute words in the text is obtained, including: obtaining a contextual representation of each word in a concatenated sequence of the text through a pre-trained language model; obtaining a contextual representation of each word in a concatenated sequence of the attribute words through a pre-trained language model; and performing average pooling on the contextual representation of each word in the concatenated sequence of the attribute words to obtain an overall representation of the attribute words.

[0038] In the embodiments of the present disclosure, the pre-trained language model refers to a language model obtained through pre-training, and the pre-trained language model can be used to extract the contextual representation of attribute words and the contextual representation of each word in the text. In the following embodiments, the pre-trained language model is BERT (Bidirectional Encoder Representation from Transformers, bidirectional encoder representation from Transformer (converter)) for example, but the present disclosure is not limited to this. Any other model or method that can vectorize attribute words and each word in the text can be used. Among them, BERT is a pre-trained language model. BERT aims to pre-train deep bidirectional representations by jointly adjusting the left and right contexts in all layers.

[0039] In the embodiment of the present disclosure, the first character and the second character are spliced ​​at both ends of the attribute word and the text respectively, so as to make the attribute word and the text conform to the input format or specification of the model for extracting its context representation. For example, taking the BERT model as an example, the first character can be "CLS" and the second character can be "SEP", but the present disclosure is not limited to this.

[0040] In an exemplary embodiment, the context distance con between the i-th word in the text and the attribute word is obtained according to the following formula:i :

[0041]

[0042] Among them, l asp_left Indicates the position of the leftmost word constituting the attribute word in the text; l asp_right Indicates the position of the rightmost word constituting the attribute word in the text; l i represents the position of the i-th word in the text, and i is a positive integer greater than or equal to 1 and less than or equal to n. However, the present disclosure is not limited thereto, and the context distance can also be obtained by other methods.

[0043] In an exemplary embodiment, the syntactic dependency distance dep between the i-th word in the text and the attribute word is obtained according to the following formula: i :

[0044]

[0045] in, It represents the shortest syntactic distance between the i-th word in the text and the attribute word in the syntactic dependency tree. If the attribute word includes multiple words, it is the shortest syntactic distance between the rightmost word constituting the attribute word; k is a hyperparameter of the syntactic dependency distance, and k is an integer greater than 0 and less than n. k can be used to limit the obtained syntactic dependency distance to be too large, so as to avoid the bad influence of the syntactic dependency distance on the classification of the sentiment polarity of the attribute word. By calculating and analyzing the syntactic dependency distance, it can help the computer better understand the grammatical structure of the text, and then perform more accurate analysis. However, the present disclosure is not limited to this, and the syntactic dependency distance can also be obtained by other means.

[0046] In the embodiment of the present disclosure, in the model (for example Figure 5 or Figure 6 Before starting training, set the values ​​of various hyperparameters. During the training process, you can keep the values ​​of these hyperparameters unchanged. If you find that the training effect is not good and does not meet your expectations during the training process, you can adjust the values ​​of these hyperparameters and then retrain the model.

[0047] In an exemplary embodiment, the semantic perception distance dis between the i-th word in the text and the attribute word is obtained according to the following formula: i :

[0048]

[0049] Among them, C i =1 indicates that there is a direct syntactic connection between the attribute word and the i-th word in the text; C i=0 indicates that there is an indirect syntactic connection or no syntactic connection between the attribute word and the i-th word in the text; l i represents the position of the i-th word in the text, l asp_right Indicates the position of the rightmost word constituting the attribute word in the text. However, the present disclosure is not limited thereto, and the semantic perception distance can also be obtained by other methods.

[0050] In S220, a multi-distance-aware adjacency matrix is ​​obtained according to the context distance, syntactic dependency distance and semantic perception distance between each word in the text and the attribute word; and a multi-distance-aware representation is obtained according to the multi-distance-aware adjacency matrix.

[0051] In an exemplary embodiment, the multi-distance-aware adjacency matrix is ​​obtained according to the following formula:

[0052]

[0053]

[0054] Among them, con i and con j Respectively represent the context distance between the i-th word and the j-th word in the concatenated sequence of the text and the attribute word, |con i -con j | indicates con i and con j The absolute value of the difference between them, or the relative distance between the context distance between the i-th word and the j-th word in the concatenated sequence of the text and the attribute word; dep i and dep j dep respectively represent the syntactic dependency distance between the i-th word and the j-th word in the concatenated sequence of the text and the attribute word, i -dep j | indicates dep i and dep j The absolute value of the difference between them, or the relative distance between the syntactic dependency distance between the i-th word and the j-th word in the concatenated sequence of the text and the attribute word; i and dis j respectively represent the semantic perception distance between the i-th word and the j-th word in the concatenated sequence of the text and the attribute word, |dis i -dis j | indicates dis i and dis j The absolute value of the difference between them, or the relative distance between the semantic perception distance between the i-th word and the j-th word in the concatenated sequence of the text and the attribute word; represents the initial association weights of the i-th word and the j-th word in the concatenated sequence of the text; A i,j represents the association weights of the i-th word and the j-th word in the concatenated sequence of the text in the multi-distance-aware adjacency matrix; i and j are both positive integers greater than or equal to 1 and less than or equal to n; k d is a hyperparameter of the multi-distance-aware adjacency matrix.

[0055] The multi-distance-aware adjacency matrix in the disclosed embodiment simultaneously integrates the context distance, syntactic dependency distance, and semantic perception distance between each word in the text and the attribute word, and can more accurately obtain the association relationship between each word in the text and the attribute word. Based on the extracted association relationship, a multi-distance-aware adjacency matrix can be generated, which quantifies the association relationship between words and attribute words in the text in a matrix manner.

[0056] In the embodiment of the present disclosure, the calculation method of the multi-distance perception adjacency matrix is ​​not limited to the above formula (4), as long as it simultaneously integrates the three distance information of context distance, syntactic dependency distance and semantic perception distance between each word in the text and the attribute word. For example, the following formula can also be used for calculation:

[0057] or

[0058]

[0059] In an exemplary embodiment, the multi-distance perception representation is obtained by the following formula:

[0060]

[0061]

[0062] Among them, tanh is the hyperbolic tangent function; W mda 、b mda It is the parameter of the multi-distance perception graph convolutional network, which can be initialized first and then updated or adjusted during the model training process. It is a parameter that can be learned through model training; and Respectively represent the multi-distance aggregation representation of the i-th word in the concatenated sequence of the text output by the t-th layer and the t-1-th layer of the multi-distance perception graph convolutional network, t is a positive integer greater than 1, and t is less than the total number of layers of the multi-distance graph convolutional network, the total number of layers is preset and can be set according to actual needs, and the present disclosure does not limit this; represents the multi-distance aggregation representation of the i-th word in the concatenated sequence of the text output by the last layer of the multi-distance-aware graph convolutional network; h mdaRepresents a multi-distance-aware representation of the multi-distance-aware graph convolutional network output.

[0063] In the disclosed embodiment, a GCN (Graph Convolution Network) is used to process a multi-distance-aware adjacency matrix to obtain a multi-distance-aware representation. Therefore, the GCN may also be referred to as a multi-distance GCN.

[0064] In the disclosed embodiment, each word in the text can be used as a node of the graph. The multi-distance-aware adjacency matrix represents the association relationship between each word in the text, that is, it reflects the association relationship between the nodes of the graph, which can be understood as the weight of the edge between the nodes in the graph. GCN performs convolution operations on the graph and aggregates the feature information of neighboring nodes layer by layer, thereby capturing the spatial information of the nodes in the graph and the association relationship between the nodes, thereby obtaining a multi-distance-aware representation, that is, a vector representation that simultaneously integrates multiple distance information such as the above-mentioned context distance, syntactic dependency distance, and semantic perception distance. This spatial information refers to the information of the structure composed of the node and the surrounding connected nodes.

[0065] In S230, a semantic noise perception representation is obtained based on the context representation of the text and the attribute words in the text, the context distance representation, the syntactic dependency distance representation and the semantic perception distance representation between each word in the text and the attribute words.

[0066] In an exemplary embodiment, the semantic noise perception representation is obtained according to the following formula:

[0067]

[0068]

[0069]

[0070]

[0071] in, Respectively represent the context representation, context distance representation, syntactic dependency distance representation, and semantic perception distance representation of the i-th word in the concatenated sequence of the text; h asp Represents the overall representation of the attribute word; W represents the contextual semantic representation of the i-th word in the concatenated sequence of the text; sn Parameters for semantic noise-aware attention; and Respectively represent the unnormalized semantic noise-aware attention weights of the i-th word and the j-th word in the concatenated sequence of the text; represents the normalized semantic noise-aware attention weight of the ith word in the concatenated sequence of the text, h sna The semantic noise-aware representation representing the output of the semantic noise-aware attention; i and j are both positive integers greater than or equal to 1 and less than or equal to n.

[0072] In the embodiments of the present disclosure, BiLSTM (Bidirectional Long Short-Term Memory) is used to obtain semantic noise perception representation for example, but the present disclosure is not limited thereto, and for example, LSTM or GRU (Gated Recurrent Unit) may also be used. BiLSTM is an improvement based on LSTM, including forward LSTM and reverse LSTM, which can consider bidirectional information at the same time and can effectively process long-term dependencies in data.

[0073] In the present disclosure embodiment, first and h asp Splicing, and then using BiLSTM and tanh to process in sequence, it is possible to obtain the semantic noise-aware attention weight of each word in the text to the attribute word under the guidance of attribute words, context distance, syntactic dependency distance, semantic perception distance, etc. The semantic noise-aware attention weight is used to indicate the importance of each word in the text to the attribute word. The larger the semantic noise-aware attention weight, the higher the importance of the word in the judgment of the sentiment polarity of the attribute word, and vice versa. If a word in the text is useless or has little effect on the judgment of the sentiment polarity of the attribute word, its corresponding semantic noise-aware attention weight is 0 or close to 0, so that the noise word in the text that is noise for the judgment of the sentiment polarity of the attribute word can be identified. Therefore, semantic noise-aware representation refers to the vector representation that represents the attribute word by identifying and eliminating the noise in the text.

[0074] In S240, a multi-information aggregated emotion representation is obtained according to the multi-distance perception representation and the semantic noise perception representation.

[0075] In the disclosed embodiment, aggregation may include one or more operations such as addition, weighting, and concatenation. Sentiment representation refers to a vector or matrix containing sentiment information used for sentiment polarity judgment. Multi-information aggregated sentiment representation refers to a sentiment representation that aggregates multiple types of information. Here, it refers to a sentiment representation that aggregates at least multiple distance perception representations and semantic noise perception representations.

[0076] In an exemplary embodiment, the sentiment representation of the multi-information aggregation is obtained according to the following formula:

[0077] p af =σ(Waf [h sna ;h mda ]) (14)

[0078] h af =p af h sna +(1-p af )h mda (15)

[0079] h fm =[h sna ;h af ;h mda ] (16)

[0080] Among them, h sna represents the semantic noise perception representation; h mda represents the multi-distance perception representation; W af is the parameter of attribute-aware aggregate attention, which can be initialized first and learned during model training; σ represents the sigmoid activation function; p af represents the weight of attribute-aware aggregate attention on the semantic noise-aware representation; h af represents the output representation of attribute-aware aggregate attention; h f m is the emotional representation of the multi-information aggregation. That is, first h sna and h mda splicing, and then processed by the sigmoid activation function to obtain the semantic noise perception representation h sna The weight of h mda The weight is (1-p af );Then based on the weights obtained above, h sna and h mda Perform weighted summation to obtain h af ; Then h sna 、h af and h mda Splice and obtain the emotional representation h of multi-information aggregation fm .

[0081] In S250, the sentiment polarity of the attribute word is obtained according to the sentiment representation of the multi-information aggregation.

[0082] The attribute-level sentiment classification method provided by the embodiment of the present disclosure, on the one hand, obtains a multi-distance perception adjacency matrix according to the context distance, syntactic dependency distance and semantic perception distance between each word in the text and the attribute word, and obtains a multi-distance perception representation according to the multi-distance perception adjacency matrix, and obtains a multi-information aggregated sentiment representation according to the multi-distance perception representation and the semantic noise perception representation, and simultaneously integrates multiple distances and multiple information, so that the sentiment polarity judgment of the attribute word is more accurate; on the other hand, according to the context representation of the text and the attribute word in the text, the context distance representation, syntactic dependency distance representation and semantic perception distance representation between each word in the text and the attribute word, a semantic noise perception representation is obtained, so that it can be identified which words in the text are more important for the sentiment polarity judgment of the attribute word, which are unimportant or even noise words, thereby further improving the accuracy of the sentiment polarity classification of the attribute word.

[0083] In related technologies, before performing sentiment prediction, only useful information related to attribute words is extracted from the sentence context to identify the sentiment polarity of attribute words, ignoring the explicit identification and elimination of redundant and interfering information in the context. Noise words introduce redundant and interfering information, which confuses the model and makes it unable to accurately identify the sentiment polarity of attribute words. Secondly, these methods use dependency tree aggregation node representation to judge the sentiment polarity of attribute words, while the syntactic dependency tree only reveals whether there is a correlation between the context and the attribute words, ignoring the different importance of each word with respect to the attribute word, making it difficult to partition which words are more valuable for judging the sentiment polarity of the attribute word.

[0084] Based on the above problems, the method proposed in the embodiment of the present disclosure is an attribute-level sentiment classification method that integrates multi-syntactic and semantic distance perception and aggregation, by explicitly identifying and eliminating the redundant and interfering information of noise words so that they have not yet been effectively integrated into the model. Removing noise information can provide more useful information to the model and improve the sentiment modeling ability of the model. By considering the importance of different words in the text, such as a sentence, to the attribute word, the model can better capture the relevant emotional expression of the attribute word. In addition, the graph convolutional network can use multiple distances such as the context distance, syntactic dependency distance, and semantic perception distance of the sentence to further improve the performance of attribute-level sentiment classification.

[0085] Figure 3 It is a flowchart of another attribute-level sentiment classification method provided by an embodiment of the present disclosure. Figure 3 The embodiments are all described by taking the text to be classified as sentences, but the present disclosure is not limited thereto. Figure 3 The embodiments may be executed by any computer device. Figure 3 As shown, the method provided by the embodiment of the present disclosure may include the following steps.

[0086] In S310, the attribute word, the sentence context (i.e., the sentence containing the attribute word), and the corresponding context distance, syntactic dependency distance, and semantic perception distance between the attribute word and the attribute word are input into the multi-semantic and distance embedding module (refer to Figures 4 to 6 Any of the embodiments), obtain the context representation of the attribute word and the context representation, context distance representation, syntactic dependency distance representation, and semantic perception distance representation corresponding to each word in the sentence.

[0087] refer to Figure 5 or Figure 6 Embodiment, according to the attribute word {a 1 ,...,a m} and sentences Obtain contextual distance, syntactic dependency distance, and semantic perception distance. 1 Indicates the first word in the attribute word, a m Indicates the mth word in the attribute word. 1 Indicates the first word in a sentence. Indicates the mth s words. Among them, m represents the length of the attribute word, m s Indicates the length of the sentence.

[0088] When inputting the model, special characters “CLS” (i.e., the first character) and “SEP” (i.e., the second character) are concatenated at both ends of the attribute word and the sentence context, respectively, so as to obtain the concatenated sequence I of the attribute word. asp ={a cls ,a 1 ,...,a m ,a sep} and the concatenation sequence of the sentence context

[0089] In the embodiment of the present disclosure, the context distance between each word in the sentence and the attribute word is calculated to obtain the corresponding context distance sequence I con ={con 1 ,con 2 ,...,con n-1 ,con n}, where n = m s +2, the calculation formula of context distance can refer to the above formula (1). asp_left Indicates the position of the leftmost word that makes up the attribute word in the sentence, l asp_right Indicates the position of the rightmost word that makes up the attribute word in the sentence, l i Indicates the position of the i-th word in the sentence. If the attribute word is composed of a single word, then l asp_left = l asp_right .

[0090] In the embodiment of the present disclosure, the syntactic dependency distance between each word in the sentence and the attribute word is calculated to obtain the corresponding syntactic dependency distance sequence I. dep ={dep 1 ,dep 2 ,...,dep n-1 ,dep n}. The calculation formula of the syntactic dependency distance can refer to the above formula (2). At this time, It represents the shortest syntactic distance between the i-th word in the sentence and the attribute word in the syntactic dependency tree. If the attribute word is composed of multiple words, it is the shortest syntactic distance between the i-th word and the rightmost word that composes the attribute word.

[0091] In the embodiment of the present disclosure, the semantic perception distance between each word in the sentence and the attribute word is calculated to obtain the corresponding semantic perception distance sequence I dis ={dis 1 ,dis 2 ,...,dis n-1 ,dis n}. The calculation formula of semantic perception distance can refer to the above formula (3). At this time, C i =1 indicates that there is a direct syntactic connection between the attribute word and the i-th word in the sentence, that is, if there is a syntactic connection between the current word and the attribute word, the semantic perception distance at this time is explicitly set to 1. i = 0 means that there is an indirect syntactic connection between the attribute word and the i-th word in the sentence or there is no syntactic connection. i Indicates the position of the i-th word in the sentence, l asp_right Indicates the position of the rightmost word that makes up the attribute word in the sentence. If the attribute word is composed of a single word, the position of the rightmost word that makes up the attribute word is equivalent to the distance of the attribute word in the sentence.

[0092] In the disclosed embodiment, the dependency relationship between words in a sentence can be obtained through syntactic dependency analysis. If there is a dependency relationship between words, it is considered that there is a direct syntactic connection. If words can be connected through the dependency relationship between other words, it is considered that there is an indirect syntactic connection. If words cannot be connected through the dependency relationship between other words, there is no syntactic connection between words.

[0093] Take the sentence "This restaurant has a good location, but the service is not good" and the attribute word "service" as an example. The word that has a direct syntactic connection with the attribute word "service" is "not good", and the word that has an indirect grammatical connection is "but".

[0094] The semantic perceptual distance in the disclosed embodiment takes into account the information of the syntactic connection between each word in the sentence and the attribute word, as well as the semantic information of the relative distance between each word in the sentence and the attribute word, and the relative distance between the attribute word length and the sentence length. In this way, the perceptual distance between each word in the sentence and the attribute word, as well as the perceptual information of the attribute word length and the sentence length can be effectively injected into the model.

[0095] In the embodiment of the present disclosure, the concatenated sequence of attribute words and the concatenated sequence of sentences are input into the BERT embedding mapping unit 411 in the multi-semantic and distance embedding module 410 to obtain the context representation of the attribute words and the context representation corresponding to each word in the sentence. That is, in the multi-semantic and distance embedding module, BERT is used to obtain the context representation of the attribute words and each word in the sentence, respectively, and the calculation process is as follows:

[0096]

[0097]

[0098] Among them, BERT(·) represents all the operations of BERT, thereby obtaining the contextual representation of each word / term in the concatenated sequence of the sentence context

[0099] a q represents the qth word in the concatenated sequence of attribute words, Represents the context representation of the qth word in the concatenated sequence of attribute words, where q is a positive integer greater than or equal to 1 and less than or equal to m+2. The context representation of each word in the concatenated sequence of attribute words is obtained. By averaging the context representation of each word in the attribute word, we can get the overall representation h of the attribute word. asp After a specific attribute word concatenation sequence passes through BERT, the context representation of each word in the attribute word will be obtained. After average pooling, the overall representation of the attribute word will be obtained. This is because an attribute word may be composed of multiple words. Therefore, when modeling an attribute word, the context representation of each word that makes up the attribute word will be obtained. Pooling the context representations of these words will obtain the overall representation of the attribute word.

[0100] In the embodiment of the present disclosure, the context distance representation of each word in the concatenated sequence of the sentence context is obtained by embedding the context distance matrix. Through the syntactic dependency distance embedding matrix, the syntactic dependency distance representation of each word in the concatenated sequence of the sentence context is obtained. Through the semantic-aware distance embedding matrix, the semantic-aware distance representation of each word in the concatenated sequence of the sentence context is obtained.

[0101] In the disclosed embodiment, the representations of context distance, syntactic dependency distance and semantic perception distance are obtained by mapping, that is, initializing an embedding matrix, such as random initialization, Gaussian or uniform distribution initialization, and then adjusting it through a training process, which is learnable.

[0102] In the disclosed embodiment, a context distance embedding matrix is ​​obtained by initialization (for example, initialization by Gaussian distribution, average distribution, etc.), which will be updated during the training process. The specific context distance is obtained through the context distance embedding matrix to obtain the context distance representation corresponding to this context distance. That is, the context distance representation of each word in the sentence is obtained, that is, a vector, which is used for subsequent calculations and modeling. For example, if the context distance is 1, the first row of the context distance embedding matrix is ​​used as the corresponding context distance representation.

[0103] In the disclosed embodiment, a syntactic dependency distance embedding matrix is ​​obtained by initialization (for example, by Gaussian distribution, average distribution, etc.), which is updated during the training process. The specific syntactic dependency distance is embedded in the syntactic dependency distance matrix to obtain the syntactic dependency distance representation corresponding to the syntactic dependency distance. That is, the syntactic dependency distance representation of each word in the sentence is obtained, that is, a vector, which is used for subsequent calculations and modeling.

[0104] In the disclosed embodiment, a semantically perceptual distance embedding matrix is ​​obtained by initialization (for example, by Gaussian distribution, average distribution, etc.), which is updated during the training process. The specific semantically perceptual distance is embedded in the semantically perceptual distance matrix to obtain the semantically perceptual distance representation corresponding to the semantically perceptual distance. That is, the semantically perceptual distance representation of each word in the sentence is obtained, that is, a vector, which is used for subsequent calculations and modeling.

[0105] In S320, the context distance, syntactic dependency distance, and semantic perception distance between each word in the sentence and the attribute word are input into a multi-distance-aware adjacency matrix module to obtain a multi-distance-aware adjacency matrix.

[0106] In the disclosed embodiment, the distance-aware adjacency matrix module receives context distance, syntactic dependency distance, and semantic perception distance as inputs, and after calculation, the connection weights between words in the sentence can be obtained, thereby obtaining the connection weights between all words, and thus constructing a multi-distance-aware adjacency matrix. The multi-distance-aware adjacency matrix can effectively integrate and utilize multiple distance information such as context distance, syntactic dependency distance, and semantic perception distance, so that the model can capture more semantic and grammatical information.

[0107] Through the multi-distance sensing adjacency matrix module, the multi-distance sensing adjacency matrix is ​​obtained. The calculation process can refer to the above formulas (4) and (5). At this time, A i,j represents the association weight between the i-th word and the j-th word in the multi-distance-aware adjacency matrix, k d is a hyperparameter of the multi-distance-aware adjacency matrix. k d There is no specific range of values, it is a preset value. The value can be adjusted according to the actual application effect. The role of this value is to play a filtering role. When the distance value is too large, it may damage the performance of the model. Through this threshold, all values ​​greater than this value will be changed to this threshold. For example, it is equal to 5.

[0108] In S330, the obtained context representation of the attribute word, the context representation of each word in the sentence, the context distance representation, the syntactic dependency distance representation, and the semantic-aware distance representation are input into the semantic and syntactic-aware module, and the semantic noise-aware attention and the guidance information from the attribute words are used to generate a semantic noise-aware representation containing more useful information, and the multi-distance-aware GCN and the guidance information from the multi-distance-aware adjacency matrix are used to generate a multi-distance-aware representation containing more useful information.

[0109] The semantic and syntactic perception module mainly includes semantic noise-aware attention unit and multi-distance-aware GCN. Semantic noise-aware attention is used to generate semantic noise-aware representations with more useful information in combination with guidance information from attribute words. Multi-distance-aware GCN is used to generate multi-distance-aware representations with more useful information in combination with guidance information from multi-distance-aware adjacency matrices.

[0110] In the disclosed embodiment, the semantic noise-aware attention unit receives the context representation of each word in the sentence, the overall representation of the attribute word, the context distance representation of each word in the sentence, the syntactic dependency distance representation, and the semantic perception distance representation as input, and obtains the attention weight of the word in the sentence (i.e., the normalized semantic noise-aware attention weight) after intermediate representation calculation, and then multiplies the attention weight of each word by the obtained context representation of the corresponding word, and weightedly adds all the words in the sentence to obtain the semantic noise-aware representation. When calculating the attention weight, the disclosed embodiment takes into account a variety of distance information and semantic information, so that the calculated attention weight is more accurate.

[0111] In the embodiment of the present disclosure, in the calculation of semantic noise-aware attention, in the calculation process of the attention weight of each word in the sentence, after being spliced ​​with the overall representation of the attribute word, the non-normalized attention weight is obtained through a nonlinear function, so there is guidance information from the attribute word. Through the calculation of attention, the weights of some words that are not related to the attribute word to be classified will become relatively low, and the weights of related words will be relatively high, so they contain more useful information.

[0112] In the disclosed embodiment, noise words refer to words that are irrelevant to the judgment of the attribute words to be classified into sentiment polarity. For example, judging the sentiment polarity of "location" in the sentence "This restaurant has a good location, but the service is average", the attention weights of the two words "very good" are higher in the generated weights, and the attention weights of other words (such as "average") are relatively low.

[0113] In the multi-distance-aware adjacency matrix of the disclosed embodiment, the value at each position reflects the connection weight or association weight between the i-th word and the j-th word in the sentence. GCN obtains a multi-distance aggregation representation of each word in the sentence by performing a convolution operation on the multi-distance-aware adjacency matrix. This method of constructing a multi-distance-aware adjacency matrix is ​​an intuitive construction method. Through this method, multiple distances are integrated together to construct an adjacency matrix that integrates multiple distances.

[0114] The disclosed embodiment can use multiple types of distance representations and interactions between attribute words to calculate attention weights. Compared with a single distance, multiple types of distances can provide more useful information. After this calculation, context words related to the attribute words will be assigned higher attention weights, and irrelevant context words will be assigned lower attention weights. The calculation method of semantic noise-aware attention can refer to the above formulas (10) to (13). However, the present disclosure is not limited to this. Although a single-head attention mechanism is used as an example in the disclosed embodiment, a multi-head attention mechanism can actually be used. Among them, in the above formula, BiLSTM(·) represents all operations of BiLSTM, Represents the contextual semantic representation of the semantic perception modeling of the i-th word in the concatenated sequence of the sentence (which can be understood as an intermediate vector in the semantic noise perception attention calculation process). sn is the parameter of semantic noise-aware attention, which can be obtained by initialization (such as Gaussian distribution, average distribution, etc.) and updated through model training. The attention weight can be understood as reflecting the importance of the i-th word to the attribute word.

[0115] In the disclosed embodiment, the calculation method of the multi-distance-aware GCN can refer to the above formula (8) and formula (9). Through GCN, the multi-distance-aware representation of each word in the sentence can be modeled through the specific structural information in the adjacency matrix. For example, when t=1, that is, the first layer, when calculating the multi-distance aggregation representation of the i-th word in the sentence, the connection weight of each word in the i-th row of the adjacency matrix is ​​multiplied by its corresponding multi-distance set representation of the previous layer, and the sum is accumulated to obtain the multi-distance-aware representation of this word at this layer. The value of t can be 2 or 3, which is a preset value. It can be adjusted according to the actual effect. It represents the multi-distance aggregation representation of the i-th word in the sentence output by the last layer of the multi-distance perception GCN. At layer 0, the value of the multi-distance perception representation of each word in the sentence (that is, the initial value of the multi-distance perception representation of each word in the sentence), that is, the initial representation of each word in the sentence is

[0116] mean_pool(·) represents the average pooling operation. Because what is obtained is the contextual representation of each word in the context of the sentence, not the overall representation of the sentence. After the average pooling operation, it can be regarded as a sentence representation related to multiple distances and attribute words, that is, a multi-distance-aware representation. When GCN calculates the representation of each word, it captures the different degrees of importance of the word and other words in the sentence through the adjacency matrix, thereby obtaining the representation of each word in the sentence. At this time, the representation of each word contains useful information, and this useful information is integrated through average pooling. The multi-distance-aware representation obtained by average pooling is input into the subsequent module to calculate the sentiment polarity of the attribute word.

[0117] In S340, the semantic noise perception representation and the multi-distance perception representation are input into the attribute perception aggregation module, and the attribute perception aggregation attention mechanism is used to aggregate different sentiment representations, so that the multi-information aggregation sentiment representation modeled by the model can contain more information and more accurate sentiment knowledge.

[0118] In the disclosed embodiment, the attribute-aware aggregation module receives the semantic noise-aware representation and the multi-distance-aware representation as input, and obtains the gating value through sigmoid function calculation. The gating value can be regarded as the amount of information input to control the semantic noise-aware representation. 1 minus the gating value represents the amount of information input to control the multi-distance-aware representation.

[0119] In the disclosed embodiment, the attribute-aware aggregation module mainly includes an attribute-aware aggregation attention unit 441. The attribute-aware aggregation attention mechanism aggregates different sentiment representations so that the multi-information aggregated sentiment representation modeled by the model can contain more information and more accurate sentiment knowledge. The calculation method of the attribute-aware aggregation attention can refer to the above formula (14) and formula (15). The attribute-aware aggregation attention can automatically calculate how much information of the semantic noise perception representation and the multi-distance perception representation is needed for aggregation, which is used for the subsequent calculation of sentiment polarity. The sigmoid function can be understood as a gating mechanism that controls the amount of information flow.

[0120] Then, it is spliced ​​with the semantic noise perception representation and the multi-distance perception representation to obtain the multi-information aggregated sentiment representation output by the splicing operation unit 442 of the attribute perception aggregation module. The calculation process is as follows:

[0121] h fm =[h sna ;h af ;h mda ] (19)

[0122] The symbol “;” indicates the splicing operation, h fm The sentiment representation of multi-information aggregation output by the attribute-aware aggregation module contains more information and more accurate sentiment information.

[0123] In the disclosed embodiment, the semantic noise-aware attention can effectively use the guidance information of attribute words and multi-type distances to model semantic noise-aware representations. The multi-distance-aware GCN can effectively use the guidance information of the multi-distance-aware adjacency matrix to model multi-distance-aware representations. The above two representations are emotion-related representations modeled by different methods. The attribute-aware aggregation module can effectively aggregate the above two emotion representations to obtain an aggregated representation, which is also an emotion-related representation. By splicing these three representations, the emotion representation contains more information and more accurate emotion information, making the prediction of emotion polarity more accurate.

[0124] In S350, the obtained multi-information aggregated sentiment representation is input into a sentiment prediction module, and the sentiment polarity of the specific attribute word in the sentence is output.

[0125] The sentiment representation of multi-information aggregation is input into the sentiment prediction module, and the sentiment polarity of the sentence context about the attribute word is output. In the embodiment of the present disclosure, the calculation process of the sentiment prediction module is as follows:

[0126] h sc =W sc h fm +b sc (20)

[0127]

[0128] Among them, W sc and b sc is the learnable parameter of the model, h fm h represents the sentiment representation of multi-information aggregation output by the attribute perception module, which contains more information and more accurate sentiment information. sc represents the sentiment representation of multi-information aggregation after the linear layer 451, It represents the sentiment polarity distribution of attribute words output by the model after softmax 452. Softmax is a normalized exponential function.

[0129] The disclosed embodiment provides an overall framework, structure and model composition of an attribute-level sentiment classification method that integrates multi-syntactic and semantic distance perception and aggregation. The solution provided by the disclosed embodiment can explicitly identify and eliminate redundant information and interference information in the context of a sentence, and can integrate context distance, syntactic dependency distance, and semantic perception distance into the sentence representation encoding and sentiment polarity prediction related to attribute words, overcoming the problem that the attribute-level sentiment classification method has not yet effectively identified and eliminated the redundant and interference information of noise words, enabling the model to learn more useful information, and improving the model's modeling ability and expression ability of sentiment information. Secondly, the different importance levels of each word in a sentence with respect to a specific attribute word in a graph structure are explicitly considered, and the different importance levels of each word in a sentence with respect to a specific attribute word can be effectively modeled and encoded, overcoming the problem that the different importance levels of each word in a sentence with respect to a specific attribute word have not yet been effectively modeled, enabling the model to better capture the relevant sentiment expressions of the attribute words, and further improving the expression ability of the model.

[0130] In the disclosed embodiments, context distance calculation can effectively inject the relative position distance between each word and the attribute word into the model, syntactic dependency distance can effectively inject the dependency syntactic relative distance between each word and the attribute word into the model, and semantic perception distance calculation can effectively inject the perceived distance between each word and the attribute word and the perceived information of the attribute word length and sentence length into the model. The multi-distance-aware adjacency matrix module can effectively integrate and utilize information such as context distance, syntactic dependency distance, and semantic perception distance to model a multi-distance-aware adjacency matrix. Semantic noise-aware attention can effectively utilize the guiding information of attribute words and multi-type distances to model semantic noise-aware representations. Multi-distance-aware GCN can effectively utilize the guiding information of multi-distance-aware adjacency matrices to model multi-distance-aware representations. The attribute-aware aggregation module can effectively aggregate different emotional representations, enabling the model to model more informative and accurate emotional knowledge, further enhancing the model's emotional understanding ability.

[0131] Figure 4 is a schematic diagram of an attribute-level sentiment classification device provided by an embodiment of the present disclosure. Figure 4 As shown, the attribute-level sentiment classification device 400 provided by the embodiment of the present disclosure may include a multi-semantic and distance embedding module 410, a multi-distance-aware adjacency matrix module 420, a semantic and syntactic-aware module 430, an attribute-aware aggregation module 440, and a sentiment prediction module 450.

[0132] The multi-semantic and distance embedding module 410 is used to obtain the contextual representation of the text and the attribute words in the text, the contextual distance between each word in the text and the attribute word and its contextual distance representation, the syntactic dependency distance and its syntactic dependency distance representation, and the semantically-perceptual distance and its semantically-perceptual distance representation.

[0133] The multi-distance-aware adjacency matrix module 420 is used to obtain a multi-distance-aware adjacency matrix according to the context distance, syntactic dependency distance and semantic perception distance between each word in the text and the attribute word.

[0134] The semantic and syntactic perception module 430 includes a semantic noise perception attention submodule and a multi-distance perception graph convolutional network. The semantic noise perception attention submodule obtains a semantic noise perception representation based on the contextual representation of the text and the attribute words in the text, the contextual distance representation between each word in the text and the attribute words, the syntactic dependency distance representation, and the semantic perception distance representation; the multi-distance perception graph convolutional network is used to obtain a multi-distance perception representation based on the multi-distance perception adjacency matrix.

[0135] The attribute-aware aggregation module 440 is used to obtain a multi-information-aggregated sentiment representation based on the multi-distance-aware representation and the semantic noise-aware representation.

[0136] The sentiment prediction module 450 is used to obtain the sentiment polarity of the attribute word according to the sentiment representation of the multi-information aggregation.

[0137] Figure 4 The other contents of the embodiment can be referred to in the other embodiments mentioned above, which will not be described in detail here.

[0138] Figure 5 is a schematic diagram of another attribute-level sentiment classification device provided by an embodiment of the present disclosure. Figure 5 As shown, the attribute-level sentiment classification device mainly includes a multi-semantic and distance embedding module 410, a multi-distance-aware adjacency matrix module 420, a semantic and syntactic awareness module 430, an attribute-aware aggregation module 440 and a sentiment prediction module 450.

[0139] The multi-semantic and distance embedding module 410 may further include a BERT embedding mapping unit 411, a contextual distance embedding mapping unit 412, a syntactic dependency distance embedding mapping unit 413, and a semantic-aware embedding mapping unit 414. The BERT embedding mapping unit 411 is used to obtain a contextual representation of an attribute word and a contextual representation of each word in a sentence. The contextual distance embedding mapping unit 412, the syntactic dependency distance embedding mapping unit 413, and the semantic-aware embedding mapping unit 414 are used to obtain a syntactic dependency distance representation, a semantic-aware distance representation, and a contextual distance representation of each word in a sentence, respectively.

[0140] The multi-distance-aware adjacency matrix module 420 is used to integrate context distance, syntactic dependency distance and semantic-aware distance to generate a multi-distance-aware adjacency matrix.

[0141] The semantic and syntactic perception module 430 mainly includes a semantic noise perception attention unit 431 and a multi-distance perception GCN 432, which are used to eliminate noise information irrelevant to attribute words and better model the association relationship between different words about attribute words, and generate semantic noise perception representations and multi-distance perception representations containing more useful information.

[0142] The attribute-aware aggregation module 440 is mainly composed of an attribute-aware aggregation attention unit 441 and a splicing operation unit 442, which generates an attribute-aware aggregation representation, thereby obtaining a multi-information aggregation sentiment representation with a larger amount of information.

[0143] The sentiment prediction module 450 may further include a linear layer 451 and a Softmax 452 for outputting the sentiment polarity about a specific attribute word in the sentence.

[0144] The present disclosure proposes an attribute-level sentiment classification method that integrates multi-syntactic and semantic distance perception and aggregation, which can effectively integrate context distance, syntactic dependency distance, and semantic perception distance into the model, effectively alleviate the redundant and noise information in the sentence context, and effectively learn and model the importance of each word in the sentence about the attribute word in the graph structure. The proposed multi-semantic and distance embedding module and semantic noise-aware attention can effectively integrate context distance, syntactic dependency distance, and semantic perception distance into the sentence representation encoding related to the attribute word and the sentiment polarity prediction, overcoming the problem that the attribute-level sentiment classification method has not yet effectively identified and eliminated the redundant and interfering information of the noise word, enabling the model to learn more useful information, and improving the model's modeling ability and expression ability of the sentiment information. Furthermore, an attribute-aware aggregation module is also proposed, which can effectively and dynamically select semantic noise-aware representations, multi-distance-aware representations, and output representations with attribute-aware aggregated attention for aggregation, so that the model can model more information and more accurate sentiment knowledge, further improving the model's sentiment understanding ability. The performance of the attribute-level sentiment classification model has been improved, and it can more accurately determine the sentiment polarity of attribute words in sentences.

[0145] Figure 6 It is a schematic diagram of another attribute-level sentiment classification device provided in an embodiment of the present disclosure. Figure 6 Example and Figure 5 The difference of the embodiment is that the input of the attribute-aware aggregate attention unit 441 includes not only the semantic noise perception input by the semantic noise perception attention unit 431 and the multi-distance perception representation input by the multi-distance perception GCN432, but also the representation of the attribute word input by the BERT embedding mapping unit 411. For example, the above formula (14) can be modified as follows:

[0146] p af =σ(W af [h sna ;h mda ;h af ]) (twenty two)

[0147] In the present disclosure, Figures 4 to 6 The modules in any embodiment are trained together. For example, the specific training process is to input the sentences and attribute words in the training sample, obtain the predicted value of the model (i.e., the sentiment polarity of the attribute words in the training sample predicted by the model output) through the above series of calculations, and calculate the loss function by cross entropy between the predicted value and the true value (i.e., the true sentiment polarity of the attribute words in the training sample). For example, the calculation formula of the loss function is as follows:

[0148]

[0149] In the above formula, D represents all training samples in the training data set. It represents the sentiment polarity distribution of specific attribute words in the sentences of the training samples output by the model, and y represents the true sentiment polarity of the attribute word in the training samples.

[0150] The scheme proposed in the embodiment of the present disclosure, after the model is trained, can better identify the sentiment polarity of a specific attribute word in a sentence. The semantic noise-aware attention can effectively use the guidance information of attribute words and multi-type distances to model semantic noise-aware representations; the proposed multi-distance-aware GCN can effectively use the guidance information of multi-distance-aware adjacency matrices to model multi-distance-aware representations; the proposed attribute-aware aggregation module can effectively aggregate different sentiment representations, so that the model can model more information and more accurate sentiment knowledge, further improving the model's sentiment understanding ability. The scheme proposed in the embodiment of the present disclosure can be widely used in text classification. For example, it can be applied to customer service scenarios, comment analysis scenarios, smart judicial scenarios, smart education scenarios, etc.

[0151] Figures 4 to 6 Any of the attribute-level sentiment classification devices shown may be a computer program (including program code, etc.) running in a computer device. For example, the attribute-level sentiment classification device may be an application software. The device may be used to execute the corresponding steps in the method provided in the embodiment of the present disclosure. Figures 4 to 6 Any of the attribute-level sentiment classification devices 400 shown can be used to Figure 4 The computer device in the corresponding embodiment.

[0152] See also Figure 7 , Figure 7 Schematic diagram of a computer device provided by an embodiment of the present disclosure. Figure 7 As shown, the computer device in the embodiment of the present disclosure may include: one or more processors 701, a memory 702, and an input-output interface 703. The processor 701, the memory 702, and the input-output interface 703 are connected via a bus 704. The memory 702 is used to store a computer program, which includes program instructions, and the input-output interface 703 is used to receive data and output data, such as for data interaction between a host machine and a computer device, or for data interaction between various virtual machines in a host machine; the processor 701 is used to execute the program instructions stored in the memory 702. Among them, the processor 701 can execute each step of the method shown in any of the above embodiments.

[0153] In some feasible implementations, the processor 701 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0154] The memory 702 may include a read-only memory and a random access memory, and provide instructions and data to the processor 701 and the input / output interface 703. A portion of the memory 702 may also include a non-volatile random access memory. For example, the memory 702 may also store information about the device type.

[0155] In a specific implementation, the computer device can execute the implementation methods provided by the various steps in any of the above method embodiments through its built-in functional modules and / or units. For details, please refer to the implementation methods provided by the various steps in the figures shown in the above method embodiments, which will not be repeated here.

[0156] The embodiments of the present disclosure provide a computer device, including a processor, an input / output interface, and a memory. The processor obtains a computer program in the memory to execute each step of the method shown in any of the above embodiments.

[0157] The embodiments of the present disclosure also provide a computer-readable storage medium, which stores a computer program. The computer program is suitable for being loaded by the processor and executing the attribute-level sentiment classification method provided in each step of any of the above embodiments. For details, please refer to the implementation method provided in each step of any of the above embodiments, which will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in the present disclosure, please refer to the description of the method embodiment of the present disclosure. As an example, the computer program can be deployed to be executed on one computer device, or on multiple computer devices located in one location, or on multiple computer devices distributed in multiple locations and interconnected by a communication network.

[0158] The computer-readable storage medium may be the attribute-level sentiment classification device provided in any of the aforementioned embodiments or the internal storage unit of the computer device, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (smart media card, SMC), a secure digital (secure digital, SD) card, a flash card (flash card), etc. equipped on the computer device. Further, the computer-readable storage medium may also include both the internal storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0159] The present disclosure also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in any of the above-mentioned optional modes.

[0160] The terms "first", "second", etc. in the specification, claims, and drawings of the embodiments of the present disclosure are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product, or equipment that includes a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules that are not listed, or may optionally include other step units inherent to these processes, methods, devices, products, or equipment.

[0161] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in this description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0162] The method and related apparatus provided by the embodiments of the present disclosure are described with reference to the method flow chart and / or structural diagram provided by the embodiments of the present disclosure. Specifically, each process and / or box in the method flow chart and / or structural diagram, as well as the combination of the processes and / or boxes in the flow chart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable application display device to generate a machine, so that the instructions executed by the processor of the computer or other programmable application display device generate instructions for implementing the process in the process. Figure 1 A process or multiple processes and / or structures Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable application display device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or structures Figure 1 These computer program instructions can also be loaded onto a computer or other programmable application display device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A flow or multiple flows and / or structures illustrate the steps of the functions specified in one block or multiple blocks.

[0163] The above disclosure is only a preferred embodiment of the present disclosure, which certainly cannot be used to limit the scope of rights of the present disclosure. Therefore, equivalent changes made according to the claims of the present disclosure are still within the scope of the present disclosure.

Claims

1. An attribute-level sentiment classification method, characterized in that: include: Obtaining a text and a context representation of an attribute word in the text, a context distance between each word in the text and the attribute word and its context distance representation, a syntactic dependency distance and its syntactic dependency distance representation, and a semantic perception distance and its semantic perception distance representation; Obtaining a multi-distance-aware adjacency matrix according to the context distance, syntactic dependency distance, and semantic perception distance between each word in the text and the attribute word; Obtaining a multi-distance-aware representation according to the multi-distance-aware adjacency matrix; Obtaining a semantic noise perception representation according to the context representation of the text and the attribute words in the text, the context distance representation, the syntactic dependency distance representation and the semantic perception distance representation between each word in the text and the attribute words; Obtaining a multi-information aggregated sentiment representation according to the multi-distance perception representation and the semantic noise perception representation; The sentiment polarity of the attribute word is obtained according to the sentiment representation of the multi-information aggregation.

2. The method according to claim 1, characterized in that Obtaining the context distance and context distance representation, syntactic dependency distance and syntactic dependency distance representation, and semantic perception distance and semantic perception distance representation between each word in the text and the attribute word, including: The first character and the second character are concatenated at both ends of the attribute word and the text to obtain a concatenated sequence of the attribute word and a concatenated sequence of the text. The length of the concatenated sequence of the attribute word is m+2, where m is the length of the attribute word; the length of the concatenated sequence of the text is n=m s +2, m s is the length of the text, m and m s are all positive integers greater than or equal to 1; Calculating the context distance between each word in the concatenated sequence of the text and the attribute word; obtaining the context distance representation of each word in the concatenated sequence of the text according to the context distance between each word in the concatenated sequence of the text and the attribute word and the context distance embedding matrix; Calculating the syntactic dependency distance between each word in the concatenated sequence of the text and the attribute word; obtaining the syntactic dependency distance representation of each word in the concatenated sequence of the text according to the syntactic dependency distance between each word in the concatenated sequence of the text and the attribute word and the syntactic dependency distance embedding matrix; The semantic perception distance between each word in the concatenated sequence of the text and the attribute word is calculated; and the semantic perception distance representation of each word in the concatenated sequence of the text is obtained according to the semantic perception distance between each word in the concatenated sequence of the text and the attribute word and the semantic perception distance embedding matrix.

3. The method according to claim 2, characterized in that The context distance con between the i-th word in the text and the attribute word is obtained according to the following formula: i : Among them, l asp_left Indicates the position of the leftmost word constituting the attribute word in the text; l asp_right Indicates the position of the rightmost word constituting the attribute word in the text; l i represents the position of the i-th word in the text, where i is a positive integer greater than or equal to 1 and less than or equal to n.

4. The method according to claim 2, characterized in that The syntactic dependency distance dep between the i-th word in the text and the attribute word is obtained according to the following formula: i : in, It represents the shortest syntactic distance between the i-th word in the text and the attribute word in the syntactic dependency tree. If the attribute word includes multiple words, it represents the shortest syntactic distance between the i-th word in the text and the rightmost word constituting the attribute word. k is a hyperparameter of the syntactic dependency distance, and k is an integer greater than 0 and less than n.

5. The method according to claim 2, characterized in that The semantic perception distance dis between the i-th word in the text and the attribute word is obtained according to the following formula: i : Among them, C i =1 indicates that there is a direct syntactic connection between the attribute word and the i-th word in the text; C i =0 indicates that there is an indirect syntactic connection or no syntactic connection between the attribute word and the i-th word in the text; l i represents the position of the i-th word in the text, l asp_right Indicates the position of the rightmost word constituting the attribute word in the text.

6. The method according to claim 2, characterized in that The multi-distance-aware adjacency matrix is ​​obtained according to the following formula: Among them, con i and con j dep respectively represent the context distance between the i-th word and the j-th word in the concatenated sequence of the text and the attribute word; i and dep j respectively represent the syntactic dependency distances between the i-th word and the j-th word in the concatenated sequence of the text and the attribute word; i and dis j Respectively represent the semantic perception distance between the i-th word and the j-th word in the concatenated sequence of the text and the attribute word; represents the initial association weights of the i-th word and the j-th word in the concatenated sequence of the text; A i,j represents the association weights of the i-th word and the j-th word in the concatenated sequence of the text in the multi-distance-aware adjacency matrix; i and j are both positive integers greater than or equal to 1 and less than or equal to n; k d is a hyperparameter of the multi-distance-aware adjacency matrix.

7. The method according to claim 6, characterized in that The multi-distance perception representation is obtained by the following formula: Among them, W mda 、b mda are the parameters of the multi-distance-aware graph convolutional network; and Respectively represent the multi-distance aggregation representation of the i-th word in the concatenated sequence of the text output by the t-th layer and the t-1-th layer of the multi-distance-aware graph convolutional network, where t is a positive integer greater than 1; represents the multi-distance aggregation representation of the i-th word in the concatenated sequence of the text output by the last layer of the multi-distance-aware graph convolutional network; h mda Represents a multi-distance-aware representation of the multi-distance-aware graph convolutional network output.

8. The method according to claim 2, characterized in that Obtaining a context representation of a text and an attribute word in the text, including: Obtaining a contextual representation of each word in the concatenated sequence of the text through a pre-trained language model; Obtaining a context representation of each word in the concatenated sequence of attribute words through a pre-trained language model; The context representation of each word in the concatenated sequence of the attribute words is average pooled to obtain the overall representation of the attribute word.

9. The method according to claim 8, characterized in that The semantic noise perception representation is obtained according to the following formula: in, Respectively represent the context representation, context distance representation, syntactic dependency distance representation, and semantic perception distance representation of the i-th word in the concatenated sequence of the text; h asp Represents the overall representation of the attribute word; W represents the contextual semantic representation of the i-th word in the concatenated sequence of the text; sn Parameters for semantic noise-aware attention; and Respectively represent the unnormalized semantic noise-aware attention weights of the i-th word and the j-th word in the concatenated sequence of the text; represents the normalized semantic noise-aware attention weight of the ith word in the concatenated sequence of the text, h sna The semantic noise-aware representation representing the output of the semantic noise-aware attention; i and j are both positive integers greater than or equal to 1 and less than or equal to n.

10. The method according to claim 1, characterized in that The sentiment representation of the multi-information aggregation is obtained according to the following formula: p af =σ(W af [h sna ;h mda ]) h af =p af h sna +(1-p af )h mda h fm =[h sna ;h af ;h mda ] Among them, h sna represents the semantic noise perception representation; h mda represents the multi-distance perception representation; W af is the parameter of attribute-aware aggregate attention; p af represents the weight of attribute-aware aggregate attention on the semantic noise-aware representation; h af represents the output representation of attribute-aware aggregate attention; h fm It is the sentiment representation of the multi-information aggregation.

11. An attribute-level sentiment classification device, characterized in that: include: A multi-semantic and distance embedding module, used to obtain a contextual representation of a text and an attribute word in the text, a contextual distance between each word in the text and the attribute word and its contextual distance representation, a syntactic dependency distance and its syntactic dependency distance representation, and a semantic perception distance and its semantic perception distance representation; A multi-distance-aware adjacency matrix module, used for obtaining a multi-distance-aware adjacency matrix according to the context distance, syntactic dependency distance and semantic perception distance between each word in the text and the attribute word; A semantic and syntactic perception module, comprising a semantic noise perception attention submodule and a multi-distance perception graph convolutional network, wherein the semantic noise perception attention submodule obtains a semantic noise perception representation according to the context representation of the text and the attribute words in the text, the context distance representation between each word in the text and the attribute words, the syntactic dependency distance representation, and the semantic perception distance representation; the multi-distance perception graph convolutional network is used to obtain a multi-distance perception representation according to the multi-distance perception adjacency matrix; An attribute perception aggregation module, used to obtain a multi-information aggregated sentiment representation based on the multi-distance perception representation and the semantic noise perception representation; The sentiment prediction module is used to obtain the sentiment polarity of the attribute word according to the sentiment representation of the multi-information aggregation.

12. A computer device, characterized in that: Includes processor, memory, input and output interfaces; The processor is connected to the memory and the input / output interface respectively, wherein the input / output interface is used to receive and output data, the memory is used to store a computer program, and the processor is used to call the computer program so that the computer device executes the method described in any one of claims 1-10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded and executed by a processor, so that a computer device having the processor executes the method according to any one of claims 1 to 10.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images