Sentiment analysis device, method, computing device and readable storage medium

Abstract

The present invention discloses a sentiment analysis device, comprising: an input processing unit adapted to map each word in a target text into a first vector corresponding to each word based on a predetermined aspect; a recurrent neural network processing unit adapted to receive the first vector of each word and obtain a second vector corresponding to each word using a recurrent neural network model; a position decay unit adapted to map the second vector of each word into a third vector based on the positional relationship between each word and the predetermined aspect; an attention processing unit adapted to calculate the attention distribution of each word based on the third vector of each word; a weighted combination unit adapted to perform weighted combination of the third vectors of each word based on the attention distribution; and a classification output unit adapted to determine the sentiment polarity of the target text with respect to the predetermined aspect based on the output of the weighted combination unit. The present invention also discloses a corresponding sentiment analysis method, a computing device, and a readable storage medium.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing technology, and in particular to a sentiment analysis device, method, computing equipment and readable storage medium. Background Art

[0002] With the development and widespread use of internet technology, various social and shopping platforms generate massive amounts of text data. This text often contains reviews of various objects (such as products and services). These reviews have a certain emotional polarity, such as positive, negative, or neutral. The technology that mines this emotional polarity is called sentiment analysis.

[0003] Text usually contains sentiment polarity for one or more aspects. However, existing sentiment analysis techniques can only extract overall sentiment classification and cannot obtain sentiment polarity for each aspect, resulting in an inability to fully understand users' evaluation of the object.

[0004] Therefore, a more advanced sentiment analysis scheme is needed to classify sentiments in various aspects. Summary of the Invention

[0005] To this end, embodiments of the present invention provide a sentiment analysis apparatus, method, computing device, and readable storage medium in an effort to solve or at least alleviate at least one of the above problems.

[0006] According to one aspect of an embodiment of the present invention, a sentiment analysis device is provided, which is suitable for determining the sentiment polarity of a target text with respect to a predetermined aspect. The sentiment analysis device includes: an input processing unit, suitable for mapping each word in the target text into a first vector corresponding to each word based on the predetermined aspect; a recurrent neural network processing unit, suitable for receiving the first vector of each word, and obtaining a second vector corresponding to each word by using a recurrent neural network model; a position decay unit, suitable for receiving the second vector of each word, and mapping the second vector of each word into a third vector based on the positional relationship between each word and the predetermined aspect; an attention processing unit, suitable for receiving the third vector of each word, and calculating the attention distribution of each word based on the third vector of each word; a weighted combination unit, based on receiving the third vector of each word and the attention distribution, and performing weighted combination on the third vector of each word based on the attention distribution; and a classification output unit, suitable for receiving the output of the weighted combination unit, and determining the sentiment polarity of the target text with respect to the predetermined aspect based on the output of the weighted combination unit.

[0007] Optionally, in the device according to an embodiment of the present invention, the embedding processing unit includes: an entity identification embedding processing unit, suitable for mapping the identification relationship between each word in the target text and the predetermined aspect into an identification vector corresponding to each word; a word embedding processing unit, suitable for mapping each word in the target text into a word vector corresponding to each word; and an embedding combination unit, suitable for combining the word vector and the identification vector corresponding to each word to obtain a first vector corresponding to each word.

[0008] Optionally, in the apparatus according to the embodiment of the present invention, the identification relationship indicates whether a word in the target text is an entity word corresponding to a predetermined aspect.

[0009] Optionally, in the device according to an embodiment of the present invention, the recurrent neural network used by the recurrent neural network processing unit is a bidirectional long short-term memory network.

[0010] Optionally, in the device according to an embodiment of the present invention, the attention processing unit is suitable for adopting a structured attention mechanism to calculate the attention distribution of each word based on the third vector of each word.

[0011] Optionally, in the device according to an embodiment of the present invention, the structured attention mechanism includes a multi-head structured attention mechanism, and the attention processing unit is suitable for calculating multiple attention distributions corresponding to multiple heads for each word based on the third vector of each word.

[0012] Optionally, the device according to an embodiment of the present invention further includes a conditional random field processing unit connected to the recurrent neural network processing unit, and the attention processing unit is suitable for calculating the attention distribution based on the conditional random field processing unit.

[0013] Optionally, in an apparatus according to an embodiment of the present invention, the attention processing unit is adapted to calculate the attention distribution based on predetermined labels of each word, a third vector, an emission score output by the recurrent neural network processing unit, and a transfer score of the conditional random field processing unit.

[0014] Optionally, in the device according to an embodiment of the present invention, the weighted combination unit is suitable for performing weighted combination of the third vectors of each word based on the attention distribution for each attention distribution to obtain multiple fourth vectors; and combining the multiple fourth vectors to obtain and output the fifth vector of the target text.

[0015] Optionally, in the device according to an embodiment of the present invention, the classification output unit includes a fully connected neural network processing unit and a Softmax processing unit.

[0016] Optionally, in the apparatus according to the embodiment of the present invention, entity words corresponding to the predetermined aspects are extracted from the target text.

[0017] According to another aspect of an embodiment of the present invention, a sentiment analysis method is provided, including: mapping each word in a target text into a first vector corresponding to each word based on a predetermined aspect; obtaining a second vector corresponding to each word using a recurrent neural network model; mapping the second vector of each word into a third vector based on a positional relationship between each word and the predetermined aspect; calculating an attention distribution of each word based on the third vector of each word; weightedly combining the third vectors of each word based on the attention distribution; and determining the sentiment polarity of the target text with respect to the predetermined aspect based on the vector obtained by the weighted combination.

[0018] Optionally, in the method according to an embodiment of the present invention, the step of mapping each word in the target text into a first vector corresponding to each word based on a predetermined aspect includes: mapping the identification relationship between each word in the target text and the predetermined aspect into an identification vector corresponding to each word; mapping each word in the target text into a word vector corresponding to each word; and combining the word vector corresponding to each word and the identification vector to obtain a first vector corresponding to each word.

[0019] Optionally, in the method according to the embodiment of the present invention, the identification relationship indicates whether a word in the target text is an entity word corresponding to a predetermined aspect.

[0020] Optionally, in the method according to an embodiment of the present invention, the recurrent neural network is a bidirectional long short-term memory network.

[0021] Optionally, in the method according to an embodiment of the present invention, the step of calculating the attention distribution of each word based on the third vector of each word includes: using a structured attention mechanism to calculate the attention distribution of each word based on the third vector of each word.

[0022] Optionally, in the method according to an embodiment of the present invention, the step of calculating the attention distribution of each word based on the third vector of each word includes: adopting a multi-head structured attention mechanism, and calculating multiple attention distributions corresponding to multiple heads for each word based on the third vector of each word.

[0023] Optionally, in the method according to an embodiment of the present invention, the step of calculating the attention distribution includes: using a conditional random field to calculate the attention distribution.

[0024] Optionally, in the method according to an embodiment of the present invention, the step of weightedly combining the third vectors of each word based on the attention distribution includes: for each attention distribution, weightedly combining the third vectors of each word based on the attention distribution to obtain multiple fourth vectors; and combining the multiple fourth vectors to obtain the fifth vector of the target text.

[0025] According to another aspect of an embodiment of the present invention, a computing device is provided, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing any one of the sentiment analysis methods according to an embodiment of the present invention.

[0026] According to another aspect of an embodiment of the present invention, a readable storage medium storing a program is provided, wherein the program includes instructions that, when executed by a computing device, enable the computing device to perform any one of the sentiment analysis methods according to an embodiment of the present invention.

[0027] The sentiment analysis scheme according to an embodiment of the present invention can obtain the sentiment polarity of the target text for predetermined aspects through embedding processing, recurrent neural network processing, decay processing, attention mechanism processing and classification processing of each word in the target text.

[0028] By combining word identification vectors and word vectors, we can obtain identification information for the predetermined aspect, improving the accuracy of sentiment analysis. By performing decay based on the positional relationship between words and the predetermined aspect, we reduce the influence of words that are far from the predetermined aspect on the sentiment polarity of the predetermined aspect, further improving the accuracy of sentiment analysis.

[0029] Among them, the use of structured attention mechanisms (such as conditional random fields) can effectively extract structural information (such as words and phrases) from the target text, compensating for the loss in previous feature extraction and further improving the accuracy of sentiment analysis. Furthermore, the use of multi-head structured attention mechanisms can extract features of different dimensions in parallel, improving the problem of insufficient accuracy caused by insufficient feature extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] To achieve the above and related purposes, certain illustrative aspects are described herein in conjunction with the following description and accompanying drawings, which indicate various ways in which the principles disclosed herein may be practiced, and all aspects and their equivalents are intended to fall within the scope of the claimed subject matter. The above and other objects, features, and advantages of the present disclosure will become more apparent by reading the following detailed description in conjunction with the accompanying drawings. Throughout this disclosure, the same reference numerals generally refer to the same parts or elements.

[0031] Figure 1 1 shows a schematic diagram of a sentiment analysis system 100 according to an embodiment of the present invention;

[0032] Figure 2 shows a schematic diagram of a computing device 200 according to one embodiment of the present invention;

[0033] Figure 3 1 shows a schematic diagram of a sentiment analysis device 300 according to an embodiment of the present invention;

[0034] Figure 4 A schematic diagram showing a sentiment analysis model 400 according to one embodiment of the present invention; and

[0035] Figure 5 A flowchart of a sentiment analysis method 500 according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0036] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0037] Figure 1 FIG. 1 shows a schematic diagram of a sentiment analysis system 100 according to an embodiment of the present invention. Figure 1 As shown, the sentiment analysis system 100 includes a processing front end 110 and a sentiment analysis device 300.

[0038] The processing front end 110 is any requesting party that needs to perform sentiment analysis. For example, in one embodiment, the processing front end 110 can be part of a shopping platform. The shopping platform needs to determine customer evaluations of various aspects of a product or service in order to improve the product or service. Therefore, the processing front end 110 can obtain customer reviews of the product or service and send them to the sentiment analysis device 300 for sentiment classification.

[0039] The processing front end 110 can also be part of a social platform. The social platform can perform sentiment analysis on articles published on the platform regarding a certain event or opinion, so that other readers can understand the evaluation of the event or opinion. In this case, the processing front end 110 can send the article to the sentiment analysis device 200 for sentiment classification.

[0040] The embodiments of the present invention are not limited to the specific form of the processing front end 110. The sentiment analysis device 200 can receive requests from the processing front end 110 in various ways. For example, the sentiment analysis device 200 can provide an application programming interface (API) with a predetermined format definition to facilitate the processing front end 110 to organize the sentiment analysis request according to the definition and send it to the sentiment analysis device 300.

[0041] The sentiment analysis device 300 receives a sentiment analysis request, obtains a target text and / or a predetermined aspect of the target text from the request, and classifies the target text according to the sentiment polarity based on the predetermined aspect.

[0042] Typically, the target text includes multiple words, among which there are one or more aspect terms, corresponding to one or more different aspects. According to an embodiment of the present invention, the predetermined aspect may be an entity term contained in the target text. Taking the target text "The service at this restaurant is good, but the food is not" as an example, the predetermined aspect may be the entity term "service" or "food" in the target text. In the present disclosure, unless otherwise specified, a predetermined aspect and the entity term corresponding to the predetermined aspect are equivalent concepts.

[0043] Entity words can be obtained in various ways. For example, any existing or future technology for extracting words from text can be used to extract entity words from the target text. Alternatively, entity words can be prepared in advance and then provided to the sentiment analysis device 300.

[0044] Sentiment polarity refers to the sentiment expressed by the target text. The target text may have different sentiment polarities for different aspects. Generally speaking, sentiment polarity can be categorized into positive and negative. Of course, neutrality can also be included. If necessary, even more categories can be included, such as very good, good, fair, poor, very poor, and so on. The embodiments of the present invention do not impose any restrictions on the number of categories of sentiment polarity.

[0045] For example, consider the target text "This toy has a nice design, but the quality is really bad." This target text contains two entity words: "design" and "quality." The sentiment for the entity word "design" is positive, while the sentiment for the entity word "quality" is negative. In other words, the predetermined aspect can be either the entity word "design" or "quality." The target text has a positive sentiment for the design aspect and a negative sentiment for the quality aspect.

[0046] It should be understood that sentiment analysis device 300 has a large number of computational parameters, which need to be adjusted through training to achieve better computational results in actual use. Therefore, sentiment analysis device 300 can be trained using labeled training samples in dataset 120, that is, text labeled with the sentiment polarity of predetermined aspects.

[0047] Any suitable loss function and training algorithm known in the art may be used to train the sentiment analysis device 300. For example, a maximum likelihood estimation, negative log-likelihood estimation, cross entropy, or mean square error function may be used as a loss function, and a training algorithm such as Adam, AdaGrad, or SGD may be used for training. The embodiments of the present invention do not limit the specific loss function and training algorithm.

[0048] The specific structures of the various units and devices mentioned above and the corresponding processing methods will be described below with reference to the accompanying drawings.

[0049] According to an embodiment of the present invention, various components in the above-mentioned sentiment analysis system 100, such as various units, devices, etc., can be implemented by the computing device 200 described below.

[0050] Figure 2 FIG. 2 shows a schematic diagram of a computing device 200 according to an embodiment of the present invention. Figure 2 As shown, in a basic configuration 202, computing device 200 typically includes system memory 206 and one or more processors 204. A memory bus 208 may be used for communication between processor 204 and system memory 206.

[0051] Depending on the desired configuration, the processor 204 can be any type of processor, including, but not limited to, a microprocessor (μP), a microcontroller (μC), a digital signal processing unit (DSP), or any combination thereof. The processor 204 can include one or more levels of cache, such as a level 1 cache 210 and a level 2 cache 212, a processor core 214, and registers 216. An example processor core 214 can include an arithmetic logic unit (ALU), a floating point unit (FPU), a digital signal processing core (DSP core), or any combination thereof. An example memory controller 218 can be used with the processor 204, or in some implementations, the memory controller 218 can be an internal part of the processor 204.

[0052] Depending on the desired configuration, system memory 206 can be any type of memory, including but not limited to volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.), or any combination thereof. System memory 206 can include an operating system 220, one or more applications 222, and program data 224. In some embodiments, application 222 can be arranged to execute instructions on the operating system by one or more processors 204 using program data 224.

[0053] The computing device 200 may also include an interface bus 240 that facilitates communication from various interface devices (e.g., output devices 242, peripheral interfaces 244, and communication devices 246) to the basic configuration 202 via the bus / interface controller 230. Example output devices 242 include a graphics processing unit 248 and an audio processing unit 250. These can be configured to facilitate communication with various external devices such as a display or speakers via one or more A / V ports 252. Example peripheral interfaces 244 may include a serial interface controller 254 and a parallel interface controller 256, which can be configured to facilitate communication with external devices such as input devices (e.g., a keyboard, mouse, pen, voice input device, touch input device) or other peripherals (e.g., a printer, scanner, etc.) via one or more I / O ports 258. Example communication devices 246 may include a network controller 260, which can be arranged to facilitate communication with one or more other computing devices 262 via a network communication link via one or more communication ports 264.

[0054] A network communication link can be an example of a communication medium. Communication media can generally be embodied as computer-readable instructions, data structures, program modules in a modulated data signal such as a carrier wave or other transmission mechanism, and can include any information delivery medium. A "modulated data signal" can be a signal in which one or more of a data set or a change thereof can be performed in a manner that encodes information in the signal. As non-limiting examples, communication media can include wired media such as a wired network or a dedicated line network, and various wireless media such as sound, radio frequency (RF), microwave, infrared (IR) or other wireless media. The term computer-readable medium as used herein can include both storage media and communication media.

[0055] The computing device 200 can be implemented as a server, such as a database server, an application server, a web server, etc., or as a personal computer including a desktop computer and a notebook computer. Of course, the computing device 200 can also be implemented as a part of a small-sized portable (or mobile) electronic device.

[0056] In an embodiment according to the present invention, the computing device 200 is implemented as a sentiment analysis apparatus 300 and is configured to execute the sentiment analysis method 500 according to an embodiment of the present invention. The application 222 of the computing device 200 includes multiple instructions for executing the sentiment analysis method 500 according to an embodiment of the present invention, and the program data 224 may also store configuration information of the sentiment analysis system 100.

[0057] Figure 3 FIG. 3 shows a structural block diagram of a sentiment analysis device 300 according to an embodiment of the present invention. Figure 3 As shown, the sentiment analysis device 300 is suitable for determining the sentiment polarity of the target text for a predetermined aspect, which includes an input processing unit 310, a recurrent neural network processing unit 320, a position decay unit 330, an attention processing unit 340, a weighted combination unit 350 and a classification output unit 360.

[0058] The input processing unit 310 may map each word in the target text to a first vector corresponding to each word based on a predetermined aspect. Specifically, the input processing unit 310 may include an entity identifier embedding processing unit 311 , a word embedding processing unit 312 , and an embedding combination unit 313 .

[0059] The entity identifier embedding processing unit 311 can perform entity identifier embedding processing (Aspect Indicator Embedding), that is, mapping the identification relationship between each word in the target text and the predetermined aspect into an identification vector corresponding to each word. The identification relationship between each word and the predetermined aspect indicates whether the word is an entity word corresponding to the predetermined aspect. Accordingly, the identification vector can be used to identify which word is the entity word corresponding to the predetermined aspect.

[0060] For example, for the target text W = {w1,w2,...,w n}, word w in the target text t The identification relationship with the predetermined aspect can be expressed as the following formula:

[0061]

[0062] i and j represent the starting position and ending position of the entity word corresponding to the predetermined aspect in the target text, respectively, and t represents the tth position.

[0063] After obtaining the identification relationship between the word and the predetermined aspect, the identification relationship is embedded and mapped to obtain a corresponding identification vector. In some embodiments, the identification vector can be obtained by random mapping, for example, obtaining a random identification vector based on a standard normal distribution N(0,1).

[0064] The word embedding processing unit 312 can perform word embedding, that is, mapping each word in the target text into a corresponding word vector. Various word embedding methods can be used, such as Glove, Word2Vec, FastText, etc. The embodiments of the present invention do not limit the specific method for performing word embedding. Any method that can embed each word in the target text to obtain a corresponding word vector is within the scope of protection of the embodiments of the present invention.

[0065] The embedding combination unit 313 is connected to the entity identifier embedding processing unit 311 and the word embedding processing unit 312, respectively, and can combine the word vector and the identifier vector corresponding to each word to obtain a first vector corresponding to each word. The combination method may include, for example, directly concatenating the word vector and the identifier vector. The embodiment of the present invention does not limit the specific combination method.

[0066] The recurrent neural network processing unit 320 receives the first vector of each word output by the embedding combination unit 313 and uses a recurrent neural network (RNN) model to calculate a second vector corresponding to each word. The second vector of each word is typically an implicit expression vector (hidden state) of the word based on a predetermined aspect, which includes the influence factors of the surrounding context words on the word.

[0067] In some embodiments, the recurrent neural network model used by the recurrent neural network processing unit 320 includes a bidirectional long short-term memory network (Bi-LSTM). Of course, other RNN network structures, such as a long short-term memory network (LSTM), may also be used. The embodiments of the present invention do not limit the specific network structure used by the recurrent neural network processing unit 320.

[0068] The position decay unit 330 is connected to the recurrent neural network processing unit 320 and can receive the second vector of each word output by the recurrent neural network processing unit 320. Based on the positional relationship between each word and the predetermined aspect, the second vector of each word is mapped into a third vector. The third vector of each word in the target text forms a decaying representation sequence of the target text.

[0069] It is understandable that the information about a predetermined aspect in the target text can greatly affect the sentiment polarity of the predetermined aspect. Similarly, context words (i.e., words other than entity words in the target text, context words) that are closer to the predetermined aspect tend to have a greater impact on the sentiment polarity judgment of the predetermined aspect. Although the identity vector can distinguish the entity words corresponding to the predetermined aspect from the context words, the identity vector does not provide any relative position information of the context words with respect to the predetermined aspect.

[0070] Therefore, the second vector can be decayed based on the positional relationship between each word and the entity word corresponding to the predetermined aspect, so as to reduce the influence of context words that are far away from the entity word on the sentiment polarity judgment of the predetermined aspect.

[0071] In some embodiments, the position decay unit 330 may use the following formula to map the second vector to obtain a third vector:

[0072] r t =f(t)h t

[0073]

[0074] h t is the second vector, r t is the third vector, f(t) is the mapping function, i and j represent the starting and ending positions of the entity word corresponding to the predetermined aspect in the target text, respectively, and t represents the tth position. L is a constant whose value is greater than the maximum text length in dataset 120. γ is a hyperparameter; a larger value increases the influence weight of context words closer to the entity word corresponding to the predetermined aspect on the predetermined aspect.

[0075] The attention processing unit 340 receives the third vector of each word output by the position decay unit 330 and uses the attention mechanism to calculate the attention distribution of each word in the target text.

[0076] A structured attention mechanism can be used to combine the structural information of the target text (such as multi-level information such as words, phrases, sentences, etc.) to make better information selection. In some embodiments, the structured attention mechanism used includes a multi-head structured attention mechanism, that is, multiple attention distributions corresponding to multiple heads for each word in the target text can be calculated. In this way, features can be extracted from subspaces with different positions and representations in parallel, that is, features of different dimensions can be extracted from multiple directions, which greatly improves the problem of low accuracy of sentiment analysis due to insufficient extraction of rich features.

[0077] Of course, other attention mechanisms may also be used, such as a key-value pair attention mechanism, a soft attention mechanism, etc. The embodiment of the present invention does not limit the specific attention mechanism adopted by the recurrent neural network processing unit 320.

[0078] Furthermore, according to some embodiments, a conditional random field (CRF) can be used to calculate the attention distribution in the structured attention mechanism / multi-head structured attention mechanism. In this way, the use of the conditional random field can incorporate the structural information of the target text, thereby compensating for the loss of features extracted by the previous processing unit.

[0079] For example, the sentiment analysis device 300 may further include a conditional random field processing unit 370 ( Figure 3(Not shown) Predetermined labels, for example, z∈{Yes,No}, are introduced into the processing of the recurrent neural network processing unit 320 and the conditional random field processing unit 370. Subsequently, a predetermined label corresponding to each word in the target text is assigned, indicating whether the corresponding word has a semantic association relationship with the predetermined aspect. The predetermined labels of each word in the target text form a label sequence for the target text. The implicit expression vector of each word output by the recurrent neural network processing unit 320 can be used as the emission score of the predetermined label corresponding to the word.

[0080] The attention processing unit 340 may calculate each attention distribution based on the conditional random field processing unit 370. In some embodiments, the attention distribution (also referred to as probability distribution) of each word in the target text may be calculated according to the following formula:

[0081]

[0082]

[0083] Where score(z,R)score(z,R) is the relevance score, which is equal to the sum of the transition scores and the emission scores of the bidirectional long short-term memory network output. z is the label sequence of the target text, z=[z1,z2,...,z n ]. R is the decay representation sequence of the target text, R=[r1,r2,...,r n ]. A is the transfer score matrix, Indicates the predetermined label z t Go to the pre-set tag z t+1 The transfer fraction z0 and z n+1 are the start tag and the end tag respectively. E is the emission score matrix obtained from the long short-term memory network, Represents the predetermined label z at the i-th position t The transfer score matrix between predetermined labels can be obtained by training the recurrent neural network processing unit 320 and the conditional random field processing unit 370 (using texts marked with correct label sequences as training samples).

[0084] The weighted combination unit 350 is connected to the attention processing unit 340 and the position decay unit 330 respectively, and can receive the attention distribution of the target text calculated by the attention processing unit 340 and the third vector of each word output by the position decay unit 330, and perform weighted combination of the third vectors of each word based on the attention distribution to obtain the fourth vector of the target text.

[0085] It is understood that if a predetermined tag indicates that the word corresponding to the predetermined tag has a semantic association with the predetermined aspect, it can be considered that the word has an impact on the sentiment polarity of the predetermined aspect, and this impact can be represented by marginal probabilities. Marginal probabilities can be efficiently calculated using a forward-backward algorithm and dynamic programming.

[0086] In some embodiments, weighted combination may be performed according to the following formula:

[0087]

[0088] Among them, s is the fourth vector of the target text, z t =Yes indicates that the word corresponding to the predetermined tag has a semantic association relationship with the predetermined aspect. t =Yes|R) is the marginal probability.

[0089] Furthermore, it should be noted that when a multi-head structured attention mechanism is employed, the attention processing unit 340 can repeat the process of calculating the attention distribution of the target text and performing weighted combination of the third vectors of each word based on the attention distribution in parallel, thereby obtaining multiple attention distributions corresponding to multiple heads and multiple fourth vectors. The number of repetitions is the number of heads employed. In this way, more information on predetermined aspects can be extracted.

[0090] In this case, the weighted combination unit 350 may further combine (for example, directly concatenate) multiple fourth vectors corresponding to multiple heads to obtain a fifth vector of the target text:

[0091] q=[s1,s2,...,s a ],

[0092] Among them, q is the fifth vector of the target text, s1,s2,...,s a are the fourth vectors corresponding to each head respectively. a is the number of heads used in the multi-head structured attention mechanism, and its value can be set according to actual needs, for example, a = 4.

[0093] The classification output unit 360 is connected to the weighted combination unit 350, and is suitable for receiving the fifth vector (when multiple heads are used) or the fourth vector (when multiple heads are not used) of the target text output by the weighted combination unit 350, and determining the sentiment polarity of the target text for a predetermined aspect based on the fifth vector or the fourth vector.

[0094] Specifically, the classification output unit 360 may include a fully connected neural network processing unit 361 and a Softmax processing unit 362. The fully connected neural network processing unit 361 receives the fifth vector or the fourth vector and outputs the sixth vector of the target text, thereby reducing the feature dimension of the fifth vector or the fourth vector for subsequent classification. The Softmax processing unit 362 receives the sixth vector and determines the sentiment polarity of the target text with respect to a predetermined aspect based on the sixth vector. For example, it outputs a probability distribution of the sentiment polarity with respect to the predetermined aspect and selects the sentiment polarity with the highest probability as the sentiment polarity of the target text with respect to the predetermined aspect.

[0095] In some embodiments, the probability distribution of sentiment polarity for a predetermined aspect may be obtained according to the following formula:

[0096] p(y|q)=Softmax(W c q+b c ),

[0097] Among them, W c and b c They are respectively the weight parameters and bias parameters of the classification output unit 360, both of which can be obtained through training.

[0098] The sentiment analysis device 300 constructs a deep learning-based sentiment analysis model 400 through the collaboration of the above-mentioned multiple computing units to obtain the sentiment polarity of the target text with respect to predetermined aspects.

[0099] Figure 4 FIG. 4 is a schematic diagram of a sentiment analysis model 400 according to an embodiment of the present invention. Figure 4 The sentiment analysis model 400 shown is composed of Figure 3 The sentiment analysis device 300 shown is constructed.

[0100] exist Figure 4 In the example, the sentiment analysis model 400 includes an entity identification embedding layer 410, a word embedding layer 420, a bidirectional long short-term memory network layer 430, a position decay layer 440, a multi-head structured attention layer 450, a combination layer 460, and a sentiment classification layer 470. These layers are respectively composed of Figure 3 It is implemented by the entity identification embedding processing unit 311, the word embedding processing unit 312, the embedding combination unit 313, the recurrent neural network processing unit 320, the position decay unit 330, the attention processing unit 340, the weighted combination unit 350, the fully connected neural network processing unit 361 and the Softmax processing unit 362.

[0101] The entity identifier embedding layer 410 generates an identifier vector for each word in the target text and the identifier relationship of the predetermined aspect. The word embedding layer 420 generates a word vector for each word in the target text. The corresponding identifier vector and word vector are concatenated to obtain the first vector x1, x2, ..., x t ,...x n The bidirectional long short-term memory network layer 430 receives the first vector x1, x2, ..., x of each word in the target text. t ,...x n , output the second vector h1,h2,...,h of each word in the target text t ,...,h n The position decay layer 440 receives the second vectors h1, h2, ..., h of each word in the target text. t ,...,h n , output the corresponding third vector r1,r2,...,r after decay t ,...,r n .

[0102] The multi-head structured attention layer 450 receives the third vectors r1, r2, ..., r of each word output by the position decay layer 440 t ,...,r n Since the multi-head structured attention mechanism with four heads is adopted, the layer 450 uses the conditional random field to calculate the corresponding four heads ( Figure 4 Attention Distribution 1, Attention Distribution 2, Attention Distribution 3, and Attention Distribution 4 for each word in the target text (the number of heads shown is only an example).

[0103] The combination layer 460 receives the third vectors r1, r2, ..., r of each word output by the position decay layer 440. t ,...,r n , and also receives the attention distributions Attention Distribution 1, Attention Distribution 2, Attention Distribution 3, and Attention Distribution 4 calculated by the multi-head structured attention layer 450, and for each attention distribution, the third vector r1, r2, ..., r of each word is calculated based on the attention distribution. t ,...,r nPerform weighted combination to obtain four fourth vectors s1, s2, s3, and s4. The combination layer 460 combines these four fourth vectors to obtain a fifth vector q.

[0104] The sentiment classification layer 470 includes a fully connected layer and a Softmax layer ( Figure 4 (not shown). The fully connected layer receives the fifth vector q and outputs a sixth vector. The softmax layer receives the sixth vector and outputs the sentiment polarity of the target text with respect to a predetermined aspect.

[0105] Figure 5 FIG. 5 shows a flow chart of a sentiment analysis method 500 according to an embodiment of the present invention. Figure 5 As shown, sentiment analysis method 500 begins at step S510. In step S510, each word in the target text is mapped to a first vector corresponding to each word based on a predetermined aspect. For example, the identification relationship between each word in the target text and the predetermined aspect is mapped to an identification vector corresponding to each word, each word in the target text is mapped to a word vector corresponding to each word, and the word vector corresponding to each word and the identification vector are combined to obtain a first vector corresponding to each word.

[0106] Then, in step S520, a recurrent neural network model is used to obtain a second vector corresponding to each word based on the first vector of each word. The recurrent neural network model may be a bidirectional long short-term memory network.

[0107] Then, in step S530, based on the positional relationship between each word and the predetermined aspect, the second vector of each word is mapped to a third vector. In step S540, based on the third vector of each word, the attention distribution of each word in the target text is calculated. In some embodiments, a structured attention mechanism is used to calculate the attention distribution of each word in the target text. In other embodiments, a multi-head structured attention mechanism can be used to calculate multiple attention distributions corresponding to multiple heads for each word in the target text. Among them, a conditional random field can be used to calculate the attention distribution.

[0108] In step S550, based on the attention distribution of each word in the target text, the third vectors of each word are weighted and combined. In some embodiments, if a structured attention mechanism is used, the third vectors of each word are weighted and combined based on the attention distribution of each word to obtain a fourth vector. In other embodiments, if a multi-head structured attention mechanism is used, for each attention distribution corresponding to each head, the third vectors of each word are weighted and combined based on the attention distribution to obtain a fourth vector corresponding to that head. Subsequently, the multiple fourth vectors corresponding to the multiple heads are combined to obtain a fifth vector.

[0109] In step S560 , based on the obtained fourth vector or fifth vector, the sentiment polarity of the target text with respect to the predetermined aspect is determined.

[0110] The above combined Figures 1 to 4 The corresponding processing in each step of the sentiment analysis method 500 has been explained in detail in the specific description of the sentiment analysis system 100 and the sentiment analysis device 300, and the repeated content will not be repeated here.

[0111] Traditional sentiment analysis solutions cannot determine the sentiment polarity of a predetermined aspect, or the accuracy of determining the sentiment polarity of a predetermined aspect is low. The sentiment analysis solution according to an embodiment of the present invention can obtain the sentiment polarity of a target text with respect to a predetermined aspect by performing embedding, recurrent neural network, decay, attention mechanism, and classification processing on each word in the target text.

[0112] By combining word identification vectors and word vectors, we can obtain identification information for the predetermined aspect, improving the accuracy of sentiment analysis. By performing decay based on the positional relationship between words and the predetermined aspect, we reduce the influence of words that are far from the predetermined aspect on the sentiment polarity of the predetermined aspect, further improving the accuracy of sentiment analysis.

[0113] Among them, the use of structured attention mechanisms (such as conditional random fields) can effectively extract structural information (such as words and phrases) from the target text, compensating for the loss in previous feature extraction and further improving the accuracy of sentiment analysis. Furthermore, the use of multi-head structured attention mechanisms can extract features of different dimensions in parallel, improving the problem of insufficient accuracy caused by insufficient feature extraction.

[0114] It should be understood that the various techniques described herein can be implemented in conjunction with hardware or software, or a combination thereof. Thus, the methods and apparatus of the present invention, or certain aspects or portions of the methods and apparatus of the present invention, can take the form of program code (i.e., instructions) embedded in a tangible medium, such as a floppy disk, CD-ROM, hard drive, or any other machine-readable storage medium, wherein when the program is loaded into a machine such as a computer and executed by the machine, the machine becomes an apparatus for practicing the present invention.

[0115] When program code is executed on a programmable computer, the computing device generally includes a processor, a storage medium readable by the processor (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. The memory is configured to store the program code; and the processor is configured to execute the various methods of the present invention according to the instructions in the program code stored in the memory.

[0116] By way of example and not limitation, computer-readable media include computer storage media and communication media. Computer-readable media include computer storage media and communication media. Computer storage media stores information such as computer-readable instructions, data structures, program modules, or other data. Communication media generally embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and includes any information delivery media. Combinations of any of the above are also included within the scope of computer-readable media.

[0117] It should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Accordingly, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of the invention.

[0118] Those skilled in the art will appreciate that the modules, units, or components of the devices in the examples disclosed herein may be arranged in the device described in the embodiment, or alternatively may be located in one or more devices different from the devices in the examples. The modules in the foregoing examples may be combined into one module or further divided into multiple submodules.

[0119] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition may be divided into multiple submodules or subunits or subcomponents. All features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed herein may be combined in any combination, except that at least some of such features and / or processes or units are mutually exclusive. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0120] Furthermore, those skilled in the art will appreciate that although some embodiments described herein include certain features included in other embodiments but not other features, combinations of features from different embodiments are intended to be within the scope of the present invention and to form different embodiments. For example, in the claims below, any of the claimed embodiments may be used in any combination.

[0121] In addition, some of the embodiments are described herein as methods or combinations of method elements that can be implemented by a processor of a computer system or by other devices that perform the functions described. Thus, a processor having the necessary instructions for implementing the method or method element forms a device for implementing the method or method element. Furthermore, the elements described herein of the device embodiments are examples of devices for implementing the functions performed by the elements for the purpose of implementing the invention.

[0122] As used herein, unless otherwise specified, the use of ordinal numbers "first," "second," "third," etc. to describe common objects merely indicates that different instances of similar objects are involved and are not intended to imply that the objects so described must have a given order in time, space, ranking, or in any other manner.

[0123] Although the present invention has been described with respect to a limited number of embodiments, it will be apparent to those skilled in the art, having benefit of the foregoing description, that other embodiments are contemplated within the scope of the invention thus described. Furthermore, it should be noted that the language used in this specification has been selected primarily for readability and didactic purposes, rather than for the purpose of explaining or limiting the subject matter of the present invention. Consequently, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. The disclosure of the present invention is intended to be illustrative rather than restrictive of the scope of the invention, which is defined by the appended claims.

Claims

1. A sentiment analysis device, adapted to determine the sentiment polarity of a target text with respect to a predetermined aspect, the sentiment analysis device comprising: an input processing unit, adapted to map each word in the target text into a first vector corresponding to each word based on the predetermined aspect, comprising: mapping an identification relationship between each word in the target text and the predetermined aspect into an identification vector corresponding to each word, and combining the word vector corresponding to each word with the identification vector to obtain the first vector corresponding to each word; a recurrent neural network processing unit adapted to receive the first vector of each word and obtain a second vector corresponding to each word using a recurrent neural network model; a position decay unit adapted to receive the second vector of each word and map the second vector of each word into a third vector based on a positional relationship between each word and the predetermined aspect; an attention processing unit, adapted to receive the third vector of each word and calculate an attention distribution of each word based on the third vector of each word using a structured attention mechanism; a weighted combination unit, adapted to receive the third vectors and attention distribution of the respective words, and perform weighted combination on the third vectors of the respective words based on the attention distribution; and The classification output unit is adapted to receive the output of the weighted combination unit and determine the sentiment polarity of the target text with respect to the predetermined aspect based on the output of the weighted combination unit.

2. The device according to claim 1, wherein The input processing unit includes: an entity identifier embedding processing unit, adapted to map the identifier relationship between each word in the target text and the predetermined aspect into an identifier vector corresponding to each word; A word embedding processing unit, adapted to map each word in the target text into a word vector corresponding to each word; and an embedding combination unit, adapted to combine the word vector corresponding to each word and the identification vector to obtain the first vector corresponding to each word.

3. The device according to claim 2, wherein The identification relationship indicates whether a word in the target text is an entity word corresponding to the predetermined aspect.

4. The device according to claim 1, wherein The recurrent neural network used by the recurrent neural network processing unit is a bidirectional long short-term memory network.

5. The device according to claim 1, wherein The structured attention mechanism includes a multi-head structured attention mechanism, and the attention processing unit is suitable for calculating multiple attention distributions corresponding to multiple heads of each word based on the third vector of each word.

6. The apparatus according to claim 1 or 5, further comprising a conditional random field processing unit connected to the recurrent neural network processing unit, wherein the attention processing unit is adapted to The attention distribution is calculated based on the conditional random field processing unit.

7. The device according to claim 6, wherein The attention processing unit is adapted to The attention distribution is calculated based on the predetermined labels of the respective words, the third vector, the emission scores output by the recurrent neural network processing unit, and the transfer scores of the conditional random field processing unit.

8. The device according to claim 5, wherein The weighted combination unit is suitable for For each attention distribution, performing a weighted combination on the third vectors of each word based on the attention distribution to obtain a plurality of fourth vectors; The multiple fourth vectors are combined to obtain and output a fifth vector of the target text.

9. The device according to claim 1, wherein The classification output unit includes a fully connected neural network processing unit and a Softmax processing unit.

10. The device of claim 1, wherein The entity words corresponding to the predetermined aspects are extracted from the target text.

11. A sentiment analysis method, comprising: Mapping each word in the target text to a first vector corresponding to each word based on the predetermined aspect, including: mapping an identification relationship between each word in the target text and the predetermined aspect to an identification vector corresponding to each word, and combining the word vector corresponding to each word with the identification vector to obtain the first vector corresponding to each word; Use the recurrent neural network model to obtain the second vector corresponding to each word; mapping the second vectors of the respective words into third vectors based on a positional relationship between the respective words and the predetermined aspect; Using a structured attention mechanism, based on the third vector of each word, calculate the attention distribution of each word; Performing a weighted combination on the third vectors of the respective words based on the attention distribution; and Based on the vector obtained by weighted combination, the sentiment polarity of the target text with respect to the predetermined aspect is determined.

12. The method of claim 11, wherein: Based on the predetermined aspect, the step of mapping each word in the target text into a first vector corresponding to each word includes: Mapping the identification relationship between each word in the target text and the predetermined aspect into an identification vector corresponding to each word; Mapping each word in the target text into a word vector corresponding to each word; The word vector and the identification vector corresponding to each word are combined to obtain the first vector corresponding to each word.

13. The method of claim 12, wherein: The identification relationship indicates whether a word in the target text is an entity word corresponding to the predetermined aspect.

14. The method of claim 11, wherein: The recurrent neural network is a bidirectional long short-term memory network.

15. The method of claim 11, wherein: The step of calculating the attention distribution of each word based on the third vector of each word includes: A multi-head structured attention mechanism is adopted to calculate multiple attention distributions corresponding to multiple heads of each word based on the third vector of each word.

16. The method according to claim 11 or 15, wherein The step of calculating the attention distribution includes: using a conditional random field to calculate the attention distribution.

17. The method of claim 15, wherein: The step of weightedly combining the third vectors of the respective words based on the attention distribution includes: For each attention distribution, performing a weighted combination on the third vectors of each word based on the attention distribution to obtain a plurality of fourth vectors; The multiple fourth vectors are combined to obtain a fifth vector of the target text.

18. A computing device comprising: one or more processors; Memory; as well as One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing any one of the sentiment analysis methods according to claims 11-17.

19. A readable storage medium storing a program, the program comprising instructions, which, when executed by a computing device, causes the computing device to perform any one of the sentiment analysis methods according to claims 11-17.

Images