Emotion analysis method and device, electronic equipment and computer readable storage medium

Abstract

The invention provides an emotion analysis method and device, electronic equipment and a computer readable storage medium. The method comprises the steps of obtaining a feature sequence corresponding to a to-be-analyzed text, wherein the feature sequence comprises a plurality of coding features; performing attention processing on each coding feature in the feature sequence to obtain an attention feature of the to-be-analyzed text; performing spatial conversion processing on the attention features to obtain spatial conversion features of the to-be-analyzed text; and performing entity attribute recognition processing based on the spatial conversion features to obtain entity attributes corresponding to the to-be-analyzed text, and performing emotion mapping processing based on the spatial conversion features to obtain emotion polarities of the entity attributes. According to the invention, the accuracy and reliability of sentiment analysis can be improved.

Description

Technical Field

[0001] The present application relates to artificial intelligence technology, and in particular to a sentiment analysis method, device, electronic device and computer-readable storage medium. Background Art

[0002] Artificial Intelligence (AI) is a comprehensive technology in computer science. By studying the design principles and implementation methods of various intelligent machines, machines are given the functions of perception, reasoning and decision-making. Artificial intelligence technology is an interdisciplinary subject that covers a wide range of fields, such as natural language processing technology and machine learning / deep learning. With the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0003] Fine-grained sentiment analysis is one of the important tasks in the field of natural language processing. It aims to accurately identify and understand subtle sentiment tendencies and sentiment polarity from texts. This technology has a wide range of applications in social media mining, user comment analysis, market research, etc. However, traditional sentiment analysis methods usually fail to capture fine-grained sentiment information in texts, resulting in low sentiment analysis accuracy. Summary of the invention

[0004] The embodiments of the present application provide a sentiment analysis method, device, electronic device and computer-readable storage medium, which can improve the accuracy and reliability of sentiment analysis.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] The present application embodiment provides a sentiment analysis method, including:

[0007] Obtaining a feature sequence corresponding to the text to be parsed, wherein the feature sequence includes multiple encoding features;

[0008] Performing attention processing on each encoded feature in the feature sequence to obtain an attention feature of the text to be parsed;

[0009] Performing spatial transformation processing on the attention feature to obtain the spatial transformation feature of the text to be parsed;

[0010] Entity attribute recognition processing is performed based on the spatial conversion feature to obtain the entity attribute corresponding to the text to be parsed, and sentiment mapping processing is performed based on the spatial conversion feature to obtain the sentiment polarity of the entity attribute.

[0011] The present application provides a sentiment analysis device, including:

[0012] An acquisition module, used to acquire a feature sequence corresponding to the text to be parsed, wherein the feature sequence includes a plurality of encoding features;

[0013] An attention module, used for performing attention processing on each encoded feature in the feature sequence to obtain an attention feature of the text to be parsed;

[0014] A conversion module, used for performing spatial conversion processing on the attention feature to obtain the spatial conversion feature of the text to be parsed;

[0015] The prediction module is used to perform entity attribute recognition processing based on the spatial conversion feature to obtain the entity attribute corresponding to the text to be parsed, and perform sentiment mapping processing based on the spatial conversion feature to obtain the sentiment polarity of the entity attribute.

[0016] An embodiment of the present application provides an electronic device, including:

[0017] A memory for storing computer executable instructions or computer programs;

[0018] The processor is used to implement the sentiment analysis method provided in the embodiment of the present application when executing the computer executable instructions or computer program stored in the memory.

[0019] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions or a computer program for implementing the sentiment analysis method provided in the embodiment of the present application when executed by a processor.

[0020] An embodiment of the present application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, the sentiment analysis method provided in the embodiment of the present application is implemented.

[0021] The embodiments of the present application have the following beneficial effects:

[0022] Applying the embodiments of this application,

[0023] By obtaining the feature sequence corresponding to the text to be parsed and performing attention processing, the contextual dependencies of the text to be parsed can be fully learned; by performing spatial transformation processing on the attention features to extract and combine spatial transformation features of different levels in the multi-layer sentiment space, the text to be parsed can be represented and abstracted more deeply, and entity attribute recognition processing can be performed based on the spatial transformation features, and sentiment mapping processing can be performed on the identified entity attributes to obtain the sentiment polarity of the entity attributes, which can improve the accuracy and reliability of sentiment analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1is a structural diagram of a sentiment analysis system provided in an embodiment of the present application;

[0025] Figure 2 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0026] Figure 3A It is a flowchart of the sentiment analysis method provided in the embodiment of the present application;

[0027] Figure 3B It is a flowchart of the sentiment analysis method provided in the embodiment of the present application;

[0028] Figure 3C It is a flowchart of the sentiment analysis method provided in the embodiment of the present application;

[0029] Figure 3D It is a flowchart of the sentiment analysis method provided in the embodiment of the present application;

[0030] Figure 4 is a schematic diagram of the structure of the space conversion layer provided in an embodiment of the present application;

[0031] Figure 5 It is a structural diagram of the sentiment analysis model provided in the embodiment of the present application;

[0032] Figure 6 It is a flowchart of a method for training a sentiment analysis model provided in an embodiment of the present application;

[0033] Figure 7 It is a flowchart of a model parameter updating method provided in an embodiment of the present application;

[0034] Figure 8 It is a schematic diagram of the architecture of the sentiment analysis model provided in the embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0036] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0037] In the following description, the terms "first\second..." are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second..." can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0038] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0039] 1) Entity attributes refer to the objects of sentiment analysis, including entities and aspects of entities.

[0040] 2) Sentiment polarity refers to the user’s positive / negative / neutral sentiment towards entity attributes (such as towards the entity or towards a specific aspect of the entity), which is used to describe the tendency of sentiment towards entity attributes and usually includes positive, negative, and neutral.

[0041] During the implementation of the embodiments of the present application, the applicant found that the related technology, when performing fine-grained sentiment classification, first constructs a feature extraction model based on a semi-supervised learning method, and uses a supervised learning method to construct a fine-grained sentiment analysis model, and uses the constructed fine-grained sentiment model to perform aspect topic classification and fine-grained sentiment classification of the comment text. Since this technology relies on a semi-supervised pre-training model, it has the disadvantage of requiring a large amount of unlabeled data and is highly dependent on the performance of the pre-training model. When the performance of the pre-training model is not high, the accuracy of the fine-grained sentiment classification is low. In addition, some related technologies, when performing fine-grained sentiment classification, extract topics through a topic model in the data preprocessing stage, and use a more complex model structure (such as a combination of multiple models) for sentiment classification. Since this method relies on a topic model for topic extraction in the data preprocessing stage, the performance and effect of the topic model will be affected by the data set. If the topic distribution of the data set is uneven or irrelevant to the actual sentiment analysis task, it may affect the accuracy of the final result.

[0042] To this end, the embodiments of the present application provide a sentiment analysis method, device, electronic device and computer-readable storage medium. When performing sentiment analysis on a text to be analyzed, by acquiring a feature sequence corresponding to the text to be analyzed and performing attention processing, the contextual dependencies of the text to be analyzed can be fully learned; by performing spatial transformation processing on the attention features to extract and combine spatial transformation features of different levels in the multi-layer sentiment space, the text to be analyzed can be more deeply represented and abstracted, and entity attribute recognition processing is performed based on the spatial transformation features, and sentiment mapping processing is performed on the identified entity attributes to obtain the sentiment polarity of the entity attributes, which can improve the accuracy and reliability of the sentiment analysis.

[0043] The sentiment analysis provided in the embodiments of the present application can be implemented by various electronic devices, for example, it can be implemented by a terminal device or a server alone, or it can be implemented by a terminal and a server in collaboration. The following describes an exemplary application when the electronic device is implemented as a server in a sentiment analysis system, see Figure 1 , Figure 1 It is a structural diagram of the sentiment analysis system provided in an embodiment of the present application. The terminal 400 is connected to the server 200 via the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0044] In some embodiments, the server 200 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal 400 may include but is not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, etc. The terminal and the server may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present application.

[0045] In some embodiments, the function of the sentiment analysis system is implemented based on the server 200. The server 200 obtains the text to be parsed from the terminal 400. The server 200 extracts features from the text to be parsed to obtain a feature sequence corresponding to the text to be parsed; performs attention processing on each encoded feature in the feature sequence to obtain an attention feature corresponding to the text to be parsed; performs spatial conversion processing on the attention feature to obtain a spatial conversion feature of the text to be parsed; performs entity attribute recognition processing based on the spatial conversion feature to obtain the entity attributes corresponding to the text to be parsed, and performs sentiment mapping processing based on the spatial conversion feature to obtain the sentiment polarity for the entity attributes, and sends the sentiment polarity for the entity attributes to the terminal 400.

[0046] In other embodiments, the embodiments of the present application can be implemented with the aid of cloud technology. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or a local area network to achieve data computing, storage, processing, and sharing.

[0047] Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, and application technology based on the cloud computing business model. It can form a resource pool that can be used on demand and is flexible and convenient. Cloud computing technology will become an important support. The background services of the technical network system require a large amount of computing and storage resources.

[0048] Next, the structure of the electronic device for implementing the sentiment analysis method provided in the embodiment of the present application is described. As mentioned above, the electronic device provided in the embodiment of the present application can be Figure 1 Server 200 in . Figure 2 , Figure 2 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, Figure 2 The server 200 shown includes: at least one processor 210, a memory 250, and at least one network interface 220. The various components in the server 200 are coupled together via a bus system 240. It is understood that the bus system 240 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 240 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus system 240 is not described in detail. Figure 2 Various buses are labeled as bus system 240 .

[0049] The processor 210 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0050] The memory 250 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 250 may optionally include one or more storage devices that are physically remote from the processor 210.

[0051] The memory 250 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 250 described in the embodiments of the present application is intended to include any suitable type of memory.

[0052] In some embodiments, memory 250 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0053] The operating system 251 includes system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic businesses and processing hardware-based tasks; a network communication module 252 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 220. Exemplary network interfaces 220 include: Bluetooth, Wireless Compatibility Certification (WiFi), and Universal Serial Bus (USB), etc.

[0054] In some embodiments, the sentiment analysis device provided in the embodiments of the present application can be implemented in software. Figure 2 A sentiment analysis device 255 stored in a memory 250 is shown, which may be software in the form of a program and a plug-in, etc., and includes the following software modules: an acquisition module 2551, an attention module 2552, a conversion module 2553, and a prediction module 2554. These modules are logical, and therefore may be arbitrarily combined or further split according to the functions implemented. The functions of each module will be described below.

[0055] In some embodiments, the terminal or server can implement the sentiment analysis method provided in the embodiments of the present application by running a computer program. For example, the computer program can be a native program (e.g., a dedicated sentiment analysis program) or a software module in the operating system; it can be a native application (APP, Application), that is, a program that needs to be installed in the operating system to run; it can also be a small program that can be embedded in any APP, that is, a program that can be run only by downloading it to a browser environment. In short, the above-mentioned computer program can be any form of application, module or plug-in.

[0056] The sentiment analysis method provided by the embodiment of the present application will be described in conjunction with the exemplary application and implementation of the server 200 provided by the embodiment of the present application. Figure 3A , Figure 3A: is a flow chart of a sentiment analysis method provided in an embodiment of the present application, the method comprising:

[0057] In step 101, a feature sequence corresponding to the text to be parsed is obtained, wherein the feature sequence includes a plurality of encoding features.

[0058] In some embodiments, the server may obtain the feature sequence corresponding to the text to be parsed in the following manner: encode each word segment in the text to be parsed to obtain the encoding features of each word segment; encode the emotion category identifier in the text to be parsed to obtain the encoding features of the emotion category identifier; based on the encoding features of each word segment and the encoding features of the emotion category identifier, generate a feature sequence of the text to be parsed; wherein the encoding features of the emotion category identifier are located at the first position of the feature sequence, and the order of the encoding features of each word segment in the feature sequence is the same as the order of the corresponding word segment in the text to be parsed.

[0059] In actual applications, after the server obtains the text to be parsed, it can perform text preprocessing on the text to be parsed. The text preprocessing includes truncation processing and filling processing. Among them, truncation processing refers to removing stop characters in the text to be parsed. Among them, stop characters include at least one of the following: modal particles, redundant punctuation marks, these characters have no effect on the semantics of the parsed text and are redundant characters. Removing these characters can refine the parsed text and reduce the encoding complexity of the parsed text and the complexity of machine learning. The filling process refers to filling in the sentiment category identifier [CLS], text segmentation character [SEP] and text end character [SEP]. For example, when the text to be parsed includes sentence 1 and sentence 2, the text to be parsed after preprocessing can be expressed as "[CLS], Tok1, ..., TokN, [SEP1], Tok1, ..., TokM, [SEP2]", wherein [SEP1] is a text segmentation character used to separate sentence 1 and sentence 2, [SEP2] is a text end character used to indicate the end of the text to be parsed, and [CLS] is a sentiment category identifier, which does not have semantic information. Therefore, when feature extraction is performed on the text to be parsed, it will not affect the feature extraction of the text to be parsed, but these symbols without obvious semantic information will more "fairly" integrate the semantic information of each character in the text to be parsed, thereby better representing the semantics of the text to be parsed.

[0060] Encode each word (or character) and sentiment category identifier of the preprocessed text to be parsed to obtain the corresponding encoding features, for example, T [CLS] , T 1 , …, T N , T [STP1] , T 1 ', ..., T M ', T [STP2] , where T[CLS] is the sentiment category of the text to be parsed, which is used to characterize the sentiment intention of the text to be parsed. [STP1] It is the feature corresponding to the text segmentation character in the text to be parsed, which is used to segment different sentences. T[STP2] is the feature corresponding to the text terminator, which is used to indicate that the text to be parsed ends here.

[0061] In some embodiments, the server may encode each word segment in the text to be parsed in the following manner to obtain the encoding features of each word segment: perform the following processing on each word segment: vectorize the word segment to obtain the word vector corresponding to the word segment; perform position encoding on the word segment according to the position of the word segment in the text to be parsed to obtain the position vector of the word segment; determine the attribution vector of the word segment according to the sentence to which the word segment belongs in the text to be parsed; obtain the encoding features of the word segment according to the word vector of the word segment, the position vector of the word segment and the attribution vector of the word segment.

[0062] Here, in actual implementation, when encoding each word in the text to be parsed, it can be achieved through a feature extraction model based on a neural network, such as inputting the text to be parsed after preprocessing into the feature extraction model for feature extraction, and extracting the word vector, position vector and attribution vector of each word respectively, wherein the word vector can be obtained by performing word embedding processing on the word (that is, the word embedding obtained by the word embedding processing is used as the word vector), and word embedding refers to dividing words into a limited set of common sub-word units, which can achieve a compromise between the effectiveness of words and the flexibility of characters; the position vector refers to encoding the position information of the word segment into a feature vector, which is a crucial link in introducing the position relationship of the word segment in the text to be parsed into the feature extraction model; the attribution vector is used to distinguish two sentences, and the attribution vectors corresponding to the words in the same sentence are the same.

[0063] After obtaining the word vector, position vector and belonging vector of each word segment, the word vector, position vector and belonging vector of each word segment are simply summed, and the sum result is the encoding feature of the word segment; or, the weights of the word vector, position vector and belonging vector are obtained respectively, and the word vector, position vector and belonging vector of the word segment are weighted and summed according to the weights to obtain the encoding feature of the word segment. Similarly, after obtaining the encoding feature of the sentiment category identifier, the encoding feature of the sentiment category identifier is concatenated with the encoding feature of each word segment, such as placing the encoding feature of the sentiment category identifier in the first place, and arranging the corresponding encoding features of each word segment according to the order of their positions in the text to be parsed, so as to obtain the feature sequence of the text to be parsed, that is, the encoding feature of the sentiment category identifier is located at the first place of the feature sequence, and the order of the encoding features of each word segment in the feature sequence is the same as the order of the corresponding word segment in the text to be parsed.

[0064] After obtaining the feature sequence corresponding to the text to be parsed, the features in the feature sequence can also be processed for dimensionality reduction. For example, the feature extraction model used to extract the coding features is based on a neural network model including multiple neurons. During forward propagation, the activation value of a certain neuron stops working with a certain probability (which can be set according to actual needs, such as 0.3 or 0.5). The generally hidden neurons in the network are randomly deleted to achieve dimensionality reduction, and the remaining coding features after deletion are used for subsequent processing. This can improve the generalization ability of the feature extraction model. Since it will not rely too much on certain local features in the text to be parsed, it can solve the overfitting problem of the model.

[0065] In step 102, attention processing is performed on each encoded feature in the feature sequence to obtain an attention feature of the text to be parsed.

[0066] In some embodiments, see Figure 3B , Figure 3B is a flow chart of the sentiment analysis method provided in the embodiment of the present application, Figure 3A Step 102 in the above example can be accomplished by Figure 3B Steps 1021 to 1023 are shown to achieve this: In step 1021, each coding feature in the feature sequence is linearly transformed to obtain the query feature, key feature, and value feature corresponding to each coding feature.

[0067] Taking the linear transformation of a coding feature as an example, assuming that the coding feature of word segment A is X, the linear transformation matrices are W Q , W K , W V , then the encoding feature X of word segment A is linearly transformed, and the query feature Q = X*W is obtained Q , key feature K = X*W K , value feature V = X*W V .

[0068] In step 1022, association processing is performed based on the query features and key features corresponding to each coding feature to obtain the attention weight of each coding feature.

[0069] In some embodiments, the server can perform association processing based on the query feature and the key feature corresponding to each coding feature to obtain the attention weight of each coding feature in the following manner: perform the following processing for each coding feature: perform association processing based on the query feature and the key feature to obtain the first influencing factor of the coding feature; obtain the maximum prediction length of the feature sequence and the scaling factor for controlling the scaling degree of the maximum prediction length, and determine the second influencing factor of the coding feature based on the scaling factor, the maximum prediction length and the first influencing factor; perform maximum likelihood processing based on the first influencing factor and the second influencing factor to obtain the attention weight of the coding feature.

[0070] In practical applications, when performing association processing based on query features and key features, firstly, dot product processing is performed on the transpose of the query feature and the key feature to obtain a first dot product result, square root processing is performed on the length of the word segmentation to obtain a square root result, and the first dot product result and the square root result are divided, and the obtained division result is the first influencing factor of the encoding feature, from which it can be seen that the first influencing factor is positively correlated with the transpose of the query feature and the key feature, and negatively correlated with the length of the encoding feature; then, the maximum predicted length of the feature sequence and the scaling factor for controlling the scaling degree of the maximum predicted length are obtained, and the scaling factor and the logarithm of the maximum predicted length are dot product processed to obtain a second dot product result; the first influencing factor is dot product processed with the second dot product result, and the third dot product result is the second influencing factor of the encoding feature; finally, maximum likelihood processing is performed based on the first influencing factor and the second influencing factor, such as adding the first influencing factor and the second influencing factor to obtain the sum result, and the sum result is subjected to maximum likelihood processing to obtain the attention weight of the word segmentation.

[0071] Continuing with the above example, when calculating the attention weight of the encoded feature of word segment A, the query feature (Q) corresponding to the encoded feature of word segment A and the transpose of the key feature (K T ) is processed by dot product to obtain the first dot product result, recorded as QK T ; Take the square root of the length of the word to get the square root result For the first dot product result (QK T ) and the square root result The division result is obtained by dividing, that is, the first influencing factor, which is recorded as Get the maximum prediction length of the feature sequence (assuming it is n), and perform dot product processing on the scaling factor (assuming it is α, which is a learnable adjustment factor) and the logarithm of the maximum prediction length (log(n+1)) to obtain the second dot product result, recorded as α·log(n+1); The third dot product result, i.e. the second impact factor, is obtained by performing a dot product with the second dot product result (α·log(n+1)), which is recorded as Second impact factor The first impact factor The summation is performed and the summation result is processed by maximum likelihood to obtain the attention weight corresponding to the encoding feature of word segmentation A, which is recorded as

[0072] In step 1023, based on the attention weight of each coding feature, the value feature of each coding feature is weighted and summed to obtain the attention feature corresponding to the text to be parsed.

[0073] Continuing with the above example, the attention weight corresponding to the encoding feature of word segmentation A is multiplied by the value feature of the encoding feature of word segmentation A to obtain a multiplication result, which is the attention feature of the encoding feature of word segmentation A, recorded as:

[0074] According to the above method, the attention feature corresponding to each encoding feature in the feature sequence can be calculated. After obtaining the attention feature corresponding to each encoding feature, the attention features corresponding to all encoding features can be summed up, and the sum result is the attention feature corresponding to the text to be parsed.

[0075] In the above way, the dependency between each encoded feature in the feature sequence can be learned through the self-attention mechanism, so as to mine the important features in the text to be parsed for subsequent sentiment prediction processing, which can improve the accuracy of sentiment analysis.

[0076] In step 103, the attention features are spatially transformed to obtain the spatial transformation features of the text to be parsed.

[0077] Here, by performing spatial transformation processing on the attention features of the text to be parsed, the spatial transformation features of the text to be parsed in the multi-layer emotional space can be obtained, wherein the number of layers of the emotional space is consistent with the categories of emotional polarity based on entity attributes. For example, the number of layers of the emotional space is the product of the number of entity attributes and the number of categories of emotional polarity. For example, the categories of emotional polarity corresponding to the text to be parsed include negative, neutral, and positive (that is, there are 3 emotional categories). If the number of entity attributes is one, assuming that the entity attribute is the entity of mobile phone, then the categories of emotional polarity are divided into 3 categories according to the entity, namely: mobile phone-positive, mobile phone-neutral, mobile phone-negative, and the number of layers of the emotional space is 3; if the number of entity attributes is two, assuming that the entity attributes are customer service and manual roles, then the categories of emotional polarity are divided into 6 categories according to the role aspects, namely: user-positive, user-neutral, user-negative, customer service-positive, customer-neutral, customer-negative, and the number of layers of the emotional space is 6.

[0078] In some embodiments, see Figure 3C , Figure 3C is a flow chart of the sentiment analysis method provided in the embodiment of the present application, Figure 3A Step 103 in Figure 3C Steps 1031-1032 shown are implemented as follows:

[0079] In step 1031, at least one spatial transformation layer is obtained; in step 1032, the attention features are spatially transformed through at least one spatial transformation layer to obtain the spatial transformation features of the text to be parsed.

[0080] In some embodiments, when the number of spatial transformation layers is 1, the spatial transformation layer includes a linear transformation layer and a nonlinear transformation layer. The above step 1032 can be implemented as follows: the attention feature is linearly transformed through the linear transformation layer in the spatial transformation layer to obtain an intermediate transformation feature, and the intermediate conversion feature is nonlinearly mapped through the nonlinear transformation layer in the spatial transformation layer to obtain the spatial transformation feature of the text to be parsed in the multi-layer emotional space.

[0081] In some embodiments, when the number of spatial transformation layers is N, each spatial transformation layer includes a linear transformation layer and a nonlinear transformation layer, and the above step 1032 can be implemented as follows: the attention feature is linearly transformed by the linear transformation layer in the first spatial transformation layer to obtain the first intermediate transformation feature, and the first intermediate transformation feature is nonlinearly mapped by the nonlinear transformation layer in the first spatial transformation layer to obtain the first spatial transformation feature; the i-1th spatial transformation feature output by the i-1th spatial transformation layer is linearly transformed by the linear transformation layer in the i-th spatial transformation layer to obtain the i-th intermediate transformation feature, and the i-th intermediate transformation feature is nonlinearly mapped by the nonlinear transformation layer in the i-th spatial transformation layer to obtain the i-th spatial transformation feature; the Nth spatial transformation feature output by the Nth spatial transformation layer is determined as the spatial transformation feature of the text to be parsed; wherein N is a positive integer greater than 2, and i is a positive integer greater than 1 and less than N-1.

[0082] As an example, see Figure 4 , Figure 4 This is a structural diagram of the spatial conversion layer provided in an embodiment of the present application. Assuming that the number of spatial conversion layers is two, the attention feature is linearly transformed through the linear transformation layer in the first spatial conversion layer to obtain the first intermediate conversion feature, and the first intermediate conversion feature is nonlinearly mapped through the nonlinear transformation layer in the first spatial conversion layer to obtain the first spatial conversion feature; the first spatial conversion feature is linearly transformed through the linear transformation layer in the second spatial conversion layer to obtain the second intermediate conversion feature, and the second intermediate conversion feature is nonlinearly mapped through the nonlinear transformation layer in the second spatial conversion layer to obtain the spatial conversion feature of the text to be parsed in the emotional space; in this way, by performing spatial conversion processing on the attention feature to extract and combine spatial conversion features of different levels in the multi-layer emotional space, the text to be parsed can be more deeply represented and abstracted for subsequent emotional prediction processing, which can improve the accuracy of emotional analysis.

[0083] In step 104, entity attribute recognition processing is performed based on the spatial conversion feature to obtain entity attributes corresponding to the text to be parsed, and sentiment mapping processing is performed based on the spatial conversion feature to obtain sentiment polarity for the entity attributes.

[0084] In some embodiments, see Figure 3D , Figure 3D is a flow chart of the sentiment analysis method provided in the embodiment of the present application, Figure 3A Step 104 in the above example can be accomplished by Figure 3D Steps 1041 to 1045 are shown to achieve:

[0085] In step 1041, the spatial conversion feature is subjected to entity attribute mapping processing to obtain the corresponding first mapping feature; in step 1042, the first mapping feature is subjected to bias processing to obtain the entity attribute corresponding to the text to be parsed; in step 1043, the spatial conversion feature is subjected to sentiment polarity mapping processing to obtain the corresponding second mapping feature; in step 1044, the second mapping feature is subjected to bias processing to obtain the sentiment category corresponding to the text to be parsed; in step 1045, the entity attribute is subjected to sentiment mapping processing based on the sentiment category to obtain the sentiment polarity of the entity attribute.

[0086] Here, in practical applications, the spatial transformation features corresponding to the text to be parsed can be respectively input into the entity recognition model and the sentiment classification model to perform entity attribute recognition and sentiment category recognition respectively, wherein the entity recognition model and the sentiment classification model can both be fully connected layers (i.e., neural network layers). For example, when performing entity attribute recognition, the spatial transformation features (which are multidimensional feature vectors or feature matrices) corresponding to the text to be parsed are input into the entity recognition model, and entity attribute mapping processing (or linear transformation processing) is performed on the spatial transformation features to obtain the corresponding first mapping features, such as multiplying each feature in the spatial transformation features with the weight matrix (i.e., a set of learnable weights) corresponding to the entity recognition model to obtain the first mapping features, and then bias processing is performed on the first mapping features, such as adding the first mapping features to the bias vector to obtain the entity attributes corresponding to the text to be parsed.

[0087] When performing sentiment classification, the spatial transformation feature (i.e., the T [CLS] The corresponding spatial conversion features) are input into the sentiment classification model, and the spatial conversion features are subjected to sentiment classification mapping processing (or linear transformation processing) to obtain the corresponding second mapping features, such as multiplying each feature in the spatial conversion features with the weight matrix (i.e., a set of learnable weights) corresponding to the sentiment classification model to obtain the second mapping features, and then the second mapping features are biased, such as adding the second mapping features to the bias vector to obtain the sentiment category corresponding to the text to be parsed.

[0088] Then, the entity attributes and sentiment categories corresponding to the text to be parsed are parsed, that is, sentiment mapping is performed on the entity attributes based on the sentiment categories to obtain the sentiment polarity of the entity attributes. For example, if the text to be parsed is "The sound quality of this phone is great, but the battery is not durable", the entity attributes identified by the above method include: sound quality, battery, and the sentiment categories include: positive (great), negative (not durable). It can be seen that the result of sentiment analysis is: the sentiment polarity for sound quality is positive, and the sentiment polarity for battery is negative.

[0089] In some embodiments, the sentiment analysis method provided in the embodiments of the present application is implemented by calling a sentiment analysis model, see Figure 5 , Figure 5 is a schematic diagram of the structure of the sentiment analysis model provided in the embodiment of the present application. The sentiment analysis model includes: a feature extraction layer, an attention layer, a space conversion layer, an entity recognition layer and a sentiment classification layer; Figure 6 right Figure 5 The training process of the sentiment analysis model shown is explained. Figure 6 : is a flow chart of a method for training a sentiment analysis model provided in an embodiment of the present application, the training method comprising:

[0090] In step 201, a feature sequence corresponding to a training sample is extracted through a feature extraction layer, wherein the feature sequence includes a plurality of coding features, and the training sample is annotated with an entity attribute label and a sentiment category label; in step 202, an attention layer is used to perform attention processing on each coding feature in the feature sequence corresponding to the training sample to obtain an attention feature corresponding to the training sample; in step 203, a spatial conversion layer is used to perform spatial conversion processing on the attention feature of the training sample to obtain a spatial conversion feature of the training sample; in step 204, an entity recognition layer is used to perform entity attribute recognition processing based on the spatial conversion feature to obtain an entity attribute recognition result corresponding to the training sample, and a sentiment mapping processing is performed based on the spatial conversion feature through a sentiment classification layer to obtain a sentiment classification result corresponding to the training sample; in step 205, based on the entity attribute recognition result and the entity attribute label corresponding to the training sample, and the sentiment classification result and the sentiment category label corresponding to the training sample, the model parameters of the sentiment analysis model are updated.

[0091] In practical applications, each training sample in the training sample set is input into the sentiment analysis model. The training sample set includes multiple training samples. Since the execution process of each training sample in the training sample set is the same, the processing of a training sample is taken as an example to illustrate. First, feature extraction is performed on the training sample through the feature extraction layer, such as encoding each word segment in the training sample to obtain the encoding feature of each word segment in the training sample; encoding the sentiment category identifier in the training sample to obtain the encoding feature of the sentiment category identifier; based on the encoding feature of each word segment and the encoding feature of the sentiment category identifier, a feature sequence corresponding to the training sample is generated.

[0092] Then, through the attention layer, attention processing is performed on each coding feature in the feature sequence corresponding to the training sample, such as performing linear transformation processing on each coding feature in the feature sequence corresponding to the training sample to obtain the query feature, key feature and value feature corresponding to each coding feature; association processing is performed based on the query feature and key feature corresponding to each coding feature to obtain the attention weight of each coding feature; based on the attention weight of each coding feature, weighted sum processing is performed on the value feature of each coding feature to obtain the attention feature corresponding to the training sample.

[0093] Then, the attention features are spatially transformed through a spatial transformation layer, such as through at least one spatial transformation layer, to obtain spatial transformation features of the training samples in the multi-layer emotion space.

[0094] Then, through the entity recognition layer, entity attribute recognition processing is performed based on the spatial transformation features to obtain the entity attribute recognition results corresponding to the training samples, and through the sentiment classification layer, sentiment mapping processing is performed based on the spatial transformation features to obtain the sentiment classification results corresponding to the training samples, wherein the entity attribute recognition results are used to characterize the probability that the entity attributes corresponding to the training samples belong to the annotated entity attribute labels, and the sentiment classification results are used to characterize the probability that the training samples belong to the annotated sentiment category labels.

[0095] Finally, based on the entity attribute recognition results corresponding to the training samples and the annotated entity attribute labels, the loss function of the entity recognition layer is constructed, and based on the sentiment classification results corresponding to the training samples and the annotated sentiment category labels, the loss function of the sentiment classification layer is constructed. The loss function of the entity recognition layer and the loss function of the sentiment classification layer are combined to construct the loss function of the sentiment analysis model, and the model parameters of the sentiment analysis model are updated based on the loss function of the sentiment analysis model.

[0096] In some embodiments, see Figure 7 , Figure 7 is a flow chart of a model parameter updating method provided in an embodiment of the present application, Figure 6 Step 205 in the above example can be accomplished by Figure 7 Steps 2051 to 2054 shown implement:

[0097] In step 2051, one-hot encoding is performed on the entity attribute labels and sentiment category labels corresponding to the training samples to obtain entity attribute label encoding features and sentiment category label encoding features corresponding to the training samples.

[0098] Here, after the entity attribute labels and sentiment category labels corresponding to the training samples are one-hot encoded, the obtained entity attribute label encoding features and sentiment category label encoding features corresponding to the training samples are both relatively sparse features, which are convenient for multiplication with subsequent entity attribute recognition results and sentiment classification results.

[0099] In step 2052, a first loss function of the entity recognition layer is determined based on the entity attribute recognition result and the entity attribute label encoding feature.

[0100] Here, when constructing the first loss function, the logarithm of the entity attribute recognition result and the product of the entity attribute label encoding feature can be added to obtain a sub-loss function of the entity recognition layer corresponding to each training sample, and the sub-loss function of the entity recognition layer corresponding to each training sample can be averaged to obtain the first loss function of the entity recognition layer.

[0101] As an example, the first loss function can be expressed as Among them, N (a positive integer) is the number of training samples in the training sample set, C represents the number of emotion categories (C is a positive integer), y ik is the entity attribute label encoding feature (one-hot encoding) of training sample i, p jk It is the probability that the entity attribute predicted by training sample i (1≤i≤N, and i is a positive integer) belongs to the annotated entity attribute label k (1≤k≤C, and k is a positive integer) (i.e., the entity attribute recognition result).

[0102] In step 2053, the second loss function of the sentiment classification layer is determined according to the sentiment classification result and the sentiment category label encoding features.

[0103] Here, when constructing the second loss function, the product of the logarithm of the sentiment classification result and the sentiment category label encoding feature can be summed to obtain a sub-loss function of the sentiment classification layer corresponding to each training sample, and the sub-loss function of the sentiment classification layer corresponding to each training sample is averaged to obtain the second loss function of the sentiment classification layer;

[0104] As an example, the second loss function can be expressed as Where N is the number of training samples in the training sample set, C is the number of emotion categories (C is a positive integer), and y icis the emotional category label encoding feature (one-hot encoding) of training sample i, p ic is the probability that the emotion category predicted by training sample i belongs to the annotated emotion category label c (1≤c≤C and c is a positive integer) (i.e., the emotion classification result).

[0105] In step 2054, a third loss function of the sentiment analysis model is determined according to the first loss function and the second loss function, and model parameters of the sentiment analysis model are updated based on the third loss function.

[0106] Here, when constructing the third loss function, the first weight coefficient of the first loss function and the second weight coefficient of the second loss function can be obtained, and based on the first weight coefficient and the second weight coefficient, the second loss function and the first loss function are weighted summed to obtain the third loss function of the sentiment analysis model, and the model parameters of the sentiment analysis model are updated based on the third loss function.

[0107] As an example, the third loss function can be expressed as:

[0108]

[0109] Among them, λ 1 is the first weight coefficient of the first loss function, λ 2 are the second weight coefficients of the second loss function. These weight coefficients can be adaptively adjusted according to the training process of the sentiment analysis model to dynamically adjust the entity attribute recognition task and the sentiment classification task. The sentiment analysis model can allocate training resources in an optimal way according to the importance of different learning tasks autonomously according to the current training status. By adaptively adjusting the weight coefficients, the loss contribution of the sentiment classification and entity recognition tasks can be better balanced to avoid excessive interference of one task on the learning of another task.

[0110] After constructing the third loss function, determine whether the value of the third loss function exceeds a preset threshold based on the value of the third loss function. When the value of the third loss function exceeds the preset threshold, determine the error signal of the sentiment analysis model based on the third loss function, back-propagate the error signal in the sentiment analysis model, and update the model parameters of each layer during the propagation process.

[0111] Here, back propagation is explained. The feature sequence of the training sample is input into the input layer of the neural network model, passes through the hidden layer, and finally reaches the output layer and outputs the result. This is the forward propagation process of the neural network model. Since there is an error between the output result of the neural network model and the actual result, the error between the output result and the actual value is calculated, and the error is backpropagated from the output layer to the hidden layer until it propagates to the input layer. In the process of back propagation, the value of the model parameter is adjusted according to the error; the above process is continuously iterated until convergence. Among them, the sentiment analysis model belongs to the neural network model.

[0112] Through the above method, by obtaining the feature sequence corresponding to the text to be parsed, the overfitting caused by too many invalid features can be reduced; by performing attention processing on the feature sequence, the context dependency of the text to be parsed can be fully learned; by performing spatial transformation processing on the attention features to extract and combine spatial transformation features of different levels in the multi-layer sentiment space, the text to be parsed can be represented and abstracted more deeply, and entity attribute recognition processing is performed based on the spatial transformation features, and sentiment mapping processing is performed on the identified entity attributes to obtain the sentiment polarity for the entity attributes, which can improve the accuracy and reliability of sentiment analysis.

[0113] Below, an exemplary application of the embodiment of the present application in an actual application scenario will be described. In the actual post-loan collection business scenario, the emotional changes between customer service and users are mined for dialogue scenarios, customer service violations and customer emotions can be detected, and customer complaint rates can be reduced. The text to be parsed in this scenario is the call text converted by automatic speech recognition technology (ASR, Automatic Speech Recognition). The effective information is relatively scattered and the call text is mostly long. Therefore, the introduction of an adaptive attention mechanism can effectively improve the accuracy of sentiment analysis in this business scenario.

[0114] In practical applications, the sentiment analysis method provided in the embodiment of the present application can be implemented based on the sentiment analysis model, see Figure 8 , Figure 8 This is a schematic diagram of the architecture of the sentiment analysis model provided in the embodiment of the present application. The sentiment analysis model includes a feature extraction layer, a feature removal layer, an attention layer, a spatial conversion layer, and a prediction layer (including a sentiment classification layer and an entity recognition layer). Each layer will be explained one by one below.

[0115] 1. Feature extraction layer

[0116] like Figure 8As shown in the figure, the text to be parsed (such as a conversation between customer service and users in the actual post-loan collection business scenario shown on the left side of Table 1) is input into the sentiment analysis model, and each word in the text to be parsed is encoded through the feature extraction layer to obtain the encoding features of the word, and the sentiment category identifier in the text to be parsed is encoded to obtain the encoding features of the sentiment category identifier; based on the encoding features of each word and the encoding features of the sentiment category identifier, a feature sequence of the text to be parsed is generated; wherein the encoding features of the sentiment category identifier are at the first position of the feature sequence, and the order of the encoding features of each word in the feature sequence is the same as the order of the corresponding word in the text to be parsed.

[0117] When encoding each word segment, the following processing is performed on each word segment: the word segment is vectorized to obtain the word vector corresponding to the word segment; according to the position of the word segment in the text to be parsed, the word segment is positionally encoded to obtain the position vector of the word segment; according to the sentence to which the word segment belongs in the text to be parsed, the attribution vector of the word segment is determined; the word vector of the word segment, the position vector of the word segment and the attribution vector of the word segment are summed to obtain the encoding feature of the word segment.

[0118] 2. Feature Removal Layer

[0119] Through the feature elimination layer, the dimensionality reduction processing is performed on the encoding features of the feature sequence of the text to be parsed, such as randomly deleting some word segmentation encoding features to reduce overfitting, and the eliminated encoding features are used for subsequent processing.

[0120] 3. Attention Layer

[0121] The attention layer is used to perform attention processing on each encoding feature in the feature sequence of the text to be parsed, and the attention feature corresponding to the text to be parsed is obtained. For example, the following enhanced attention formula is used to calculate the attention feature of each encoding feature, and based on the attention weight of each encoding feature, the attention feature of each encoding feature is weighted summed to obtain the attention feature corresponding to the text to be parsed.

[0122]

[0123] Among them, Q is the query feature of the encoding feature, K can be the key feature of the encoding feature, V is the value feature of the encoding feature, d k is the length of the encoded feature, n is the maximum predicted length of the feature sequence corresponding to the text to be parsed, and α is a learnable adjustment factor used to control the degree of scaling of the maximum prediction length.

[0124] Since the above enhanced attention formula introduces α and terms related to the input sequence length (maximum prediction length), it can better adapt to different datasets and tasks and improve the model's ability to pay attention to and understand the semantics of the text to be parsed.

[0125] 4. Space conversion layer

[0126] Through the spatial transformation layer, the attention features are spatially transformed to obtain the spatial transformation features of the text to be parsed in the multi-layer sentiment space, where the number of layers in the sentiment space is consistent with the category of sentiment polarity based on entity attributes, such as Figure 8 As shown in the figure, the categories of sentiment polarity corresponding to the text to be parsed include negative, neutral, and positive, and the entity attributes include two role aspects: customer service and manual. The categories of sentiment polarity are divided into 6 types according to the role aspects, namely: user-positive, user-neutral, user-negative, customer service-positive, customer-neutral, and customer-negative. The number of layers in the sentiment space is 6.

[0127] The number of spatial transformation layers can be multiple, and each spatial transformation layer includes a linear transformation layer and a nonlinear transformation layer. The linear transformation layer is used to perform linear transformation processing. It receives the input sentence vector (when the linear transformation layer is located in the first spatial transformation layer, the sentence vector is the attention feature of the text to be parsed obtained above, and when the linear transformation layer is located in the i-th spatial transformation layer, the sentence vector is the i-th spatial transformation feature output by the i-1-th spatial transformation layer), and performs linear transformation through matrix multiplication and bias term addition. This linear transformation maps the sentence vector to an intermediate feature space, so that the subsequent nonlinear transformation can better capture and express the features.

[0128] The nonlinear transformation layer is used for nonlinear mapping processing. For example, the nonlinear transformation layer is arranged with a nonlinear activation function, such as ReLU, sigmoid or tanh. The nonlinear properties are introduced through the nonlinear transformation layer to help the network learn more complex and abstract features.

[0129] When there are multiple spatial transformation layers, a multi-layer feature extraction structure is formed by stacking multiple linear transformation layers and nonlinear transformation layers. Each layer takes the output of the previous layer as input and performs linear and nonlinear transformations on it. Such a multi-layer structure can gradually extract and combine features at different levels, enabling the network to represent and abstract the input more deeply.

[0130] 5. Prediction layer

[0131] The prediction layer includes a sentiment classification layer and an entity recognition layer, wherein the entity recognition layer can be a fully connected layer (i.e., a neural network layer). For example, when performing entity attribute recognition, the spatial transformation features (which are multidimensional feature vectors or feature matrices) corresponding to the text to be parsed are input into the entity recognition layer, and entity attribute mapping processing (or linear transformation processing) is performed on the spatial transformation features to obtain the corresponding first mapping features, such as multiplying each feature in the spatial transformation features with the weight matrix (i.e., a set of learnable weights) corresponding to the entity recognition model to obtain the first mapping features, and then bias processing is performed on the first mapping features, such as adding the first mapping features to the bias vector to obtain the entity attributes corresponding to the text to be parsed.

[0132] Similarly, the sentiment classification layer can be regarded as a fully connected neural network, whose input is the spatial transformation features corresponding to the encoding features of the sentiment category identifier in the text to be parsed. When performing sentiment classification, the spatial transformation features corresponding to the encoding features of the sentiment category identifier in the feature sequence corresponding to the text to be parsed (i.e., the above T [CLS] The corresponding spatial conversion features) are input into the sentiment classification model, and the spatial conversion features are subjected to sentiment classification mapping processing (or linear transformation processing) to obtain the corresponding second mapping features, such as multiplying each feature in the spatial conversion features with the weight matrix (i.e., a set of learnable weights) corresponding to the sentiment classification model to obtain the second mapping features, and then the second mapping features are subjected to bias processing, such as adding the second mapping features to the bias vector to obtain the sentiment category corresponding to the text to be parsed. Then, the entity attributes and sentiment categories corresponding to the text to be parsed are parsed, that is, sentiment mapping processing is performed on the entity attributes based on the sentiment categories to obtain the sentiment polarity for the entity attributes.

[0133] like Figure 8 As shown, after the text to be parsed (such as the dialogue shown in Table 1) is input into the sentiment analysis model, the corresponding entity attributes are identified to include: customer service-deferred repayment, user-able to repay, and the output sentiment categories include: customer service-positive (corresponding to the entity attribute of "customer service-deferred repayment"), user-positive (corresponding to the entity attribute of "user-able to repay").

[0134] For example, see Table 1. The left side of Table 1 shows a conversation between a customer service representative and a user in an actual post-loan collection business scenario (i.e. Figure 8 The text to be parsed is shown in the figure), and the sentiment analysis results obtained by using the sentiment analysis method provided in the embodiment of the present application are shown on the right. For example, the entity attributes identified include: customer service-Aspect: deferred repayment, user-Aspect: able to repay; sentiment polarity includes sentiment polarity for customer service and sentiment polarity for users, from Figure 8It can be concluded from Table 1 that the sentiment polarity for the entity attribute “customer service-deferred repayment” is “positive”, and the sentiment polarity for the entity attribute “user-able to repay” is also “positive”.

[0135] Table 1

[0136]

[0137]

[0138] Refer to Table 2, which shows another conversation between customer service and the user in the actual post-loan collection business scenario on the left (i.e., the text to be parsed mentioned above), and the sentiment analysis results obtained by using the sentiment analysis method provided in the embodiment of the present application are shown on the right. For example, the entity attributes identified include: Customer Service-Aspect: Village Secretary, User-Aspect: Refusal to Repay; the sentiment polarity includes the sentiment polarity for customer service and the sentiment polarity for users. For example, if the sentiment polarity of customer service is negative, the sentiment polarity of the user is also negative.

[0139] Table 2

[0140]

[0141] Next, the training of the sentiment analysis model is explained. Considering that the sentiment analysis method provided in the embodiment of the present application involves two tasks, namely entity attribute recognition and sentiment classification, multi-task joint training can be performed when training the sentiment analysis model. For example, a feature extraction layer is used to extract a feature sequence corresponding to each training sample in the training sample set, wherein the feature sequence includes multiple coding features, and each training sample is annotated with an entity attribute label and a sentiment category label; a feature elimination layer is used to eliminate some training samples in the training sample set, or some coding features in the feature sequence corresponding to each training sample are eliminated to solve the overfitting problem of the model.

[0142] Through the attention layer, attention processing is performed on each coding feature in the feature sequence corresponding to each training sample to obtain the attention feature corresponding to each training sample; through the spatial transformation layer, spatial transformation processing is performed on the attention feature of each training sample to obtain the spatial transformation feature of each training sample; through the entity recognition layer, entity attribute recognition processing is performed based on the spatial transformation feature to obtain the entity attribute recognition result corresponding to each training sample, and sentiment mapping processing is performed based on the spatial transformation feature to obtain the sentiment classification result corresponding to each training sample; based on the entity attribute recognition result and entity attribute label corresponding to each training sample, and the sentiment classification result and sentiment category label corresponding to each training sample, the model parameters of the sentiment analysis model are updated.

[0143] When updating the model parameters of the sentiment analysis model, the loss function of the sentiment analysis model can be constructed in the following way: First, based on the entity attribute recognition results and entity attribute labels corresponding to each training sample, the first loss function of the entity recognition layer is constructed. As an example, the first loss function can be expressed as Among them, N (a positive integer) is the number of training samples in the training sample set, C represents the number of emotion categories (C is a positive integer), y ik is the entity attribute label encoding feature (one-hot encoding) of training sample i, p jk It is the probability that the entity attribute predicted by training sample i (1≤i≤N, and i is a positive integer) belongs to the labeled entity label k (1≤k≤C, and k is a positive integer) (i.e., the entity attribute recognition result).

[0144] Then, based on the sentiment classification results and sentiment category labels corresponding to the training samples, the second loss function of the sentiment classification layer is constructed. As an example, the second loss function can be expressed as Where N is the number of training samples in the training sample set, C is the number of emotion categories (C is a positive integer), and y ic is the emotional category label encoding feature (one-hot encoding) of training sample i, p ic is the probability that the emotion category predicted by training sample i belongs to the annotated emotion category label c (1≤c≤C and c is a positive integer) (i.e., the emotion classification result).

[0145] Finally, the first weight coefficient of the first loss function and the second weight coefficient of the second loss function are obtained, and based on the first weight coefficient and the second weight coefficient, the second loss function and the first loss function are weighted summed to obtain the third loss function of the sentiment analysis model, and the model parameters of the sentiment analysis model are updated based on the third loss function. As an example, the third loss function can be expressed as:

[0146]

[0147] Among them, λ1 is the first weight coefficient of the first loss function, λ 2 are the second weight coefficients of the second loss function. These weight coefficients can be adaptively adjusted according to the training process of the sentiment analysis model to dynamically adjust the entity attribute recognition task and the sentiment classification task. The sentiment analysis model can allocate training resources in an optimal way according to the importance of different learning tasks autonomously according to the current training status. By adaptively adjusting the weight coefficients, the loss contribution of the sentiment classification and entity recognition tasks can be better balanced to avoid excessive interference of one task on the learning of another task.

[0148] Through the above-mentioned method, for texts to be parsed that are long, have scattered important information, and have a high dependence on context, the sentiment analysis method provided in the embodiment of the present application can be used to more accurately parse the emotional changes of customer service and users; in addition, joint task training can effectively reduce the complexity of application processes, reduce link request time, and save computing resources.

[0149] The following is a description of an exemplary structure of the sentiment analysis device 255 provided in the embodiment of the present application implemented as a software module. In some embodiments, Figure 2 As shown, the software modules stored in the sentiment analysis device 255 of the memory 250 may include:

[0150] An acquisition module 2551 is used to acquire a feature sequence corresponding to the text to be parsed, wherein the feature sequence includes multiple encoding features; an attention module 2552 is used to perform attention processing on each encoding feature in the feature sequence to obtain the attention feature of the text to be parsed; a conversion module 2553 is used to perform spatial conversion processing on the attention feature to obtain the spatial conversion feature of the text to be parsed; a prediction module 2554 is used to perform entity attribute recognition processing based on the spatial conversion feature to obtain the entity attributes corresponding to the text to be parsed, and perform sentiment mapping processing based on the spatial conversion feature to obtain the sentiment polarity of the entity attributes.

[0151] In some embodiments, the acquisition module 2551 is also used to encode each word segment in the text to be parsed to obtain the encoding features of each word segment; encode the emotion category identifier corresponding to the text to be parsed to obtain the encoding features of the emotion category identifier; based on the encoding features of each word segment and the encoding features of the emotion category identifier, generate a feature sequence of the text to be parsed; wherein the encoding features of the emotion category identifier are located at the first position of the feature sequence, and the order of the encoding features of the word segments in the feature sequence is the same as the order of the word segments in the text to be parsed.

[0152] In some embodiments, the acquisition module 2551 is also used to perform the following processing on each of the word segments: vectorize the word segment to obtain the word vector corresponding to the word segment; perform position encoding processing on the word segment according to the position of the word segment in the text to be parsed to obtain the position vector of the word segment; determine the attribution vector of the word segment according to the sentence to which the word segment belongs in the text to be parsed; obtain the encoding feature of the word segment according to the word vector of the word segment, the position vector of the word segment and the attribution vector of the word segment.

[0153] In some embodiments, the attention module 2552 is also used to perform linear transformation processing on each encoded feature in the feature sequence to obtain query features, key features and value features corresponding to each encoded feature; perform association processing based on the query features and key features corresponding to each encoded feature to obtain the attention weight of each encoded feature; based on the attention weight of each encoded feature, perform weighted sum processing on the value features of each encoded feature to obtain the attention feature of the text to be parsed.

[0154] In some embodiments, the attention module 2552 is also used to perform the following processing for each of the coding features: performing association processing based on the query feature and the key feature to obtain a first influencing factor of the coding feature, wherein the first influencing factor is positively correlated with the transpose of the query feature and the key feature, respectively, and negatively correlated with the length of the coding feature; obtaining the maximum predicted length of the feature sequence and a scaling factor for controlling the degree of scaling of the maximum predicted length, and determining the second influencing factor of the coding feature based on the scaling factor, the maximum predicted length and the first influencing factor; performing maximum likelihood processing based on the first influencing factor and the second influencing factor to obtain the attention weight of the coding feature.

[0155] In some embodiments, the conversion module 2553 is further used to obtain at least one spatial conversion layer; through the at least one spatial conversion layer, the attention feature is spatially converted to obtain the spatial conversion feature of the text to be parsed.

[0156] In some embodiments, each of the spatial conversion layers includes a linear conversion layer and a nonlinear conversion layer. The conversion module 2553 is also used to, when the number of the spatial conversion layers is N, perform linear conversion processing on the attention feature through the linear conversion layer in the first spatial conversion layer to obtain the first intermediate conversion feature, and perform nonlinear mapping processing on the first intermediate conversion feature through the nonlinear conversion layer in the first spatial conversion layer to obtain the first spatial conversion feature; perform linear conversion processing on the i-1th spatial conversion feature output by the i-1th spatial conversion layer through the linear conversion layer in the i-th spatial conversion layer to obtain the i-th intermediate conversion feature, and perform nonlinear mapping processing on the i-th intermediate conversion feature through the nonlinear transformation layer in the i-th spatial conversion layer to obtain the i-th spatial conversion feature; determine the Nth spatial conversion feature output by the Nth spatial conversion layer as the spatial conversion feature of the text to be parsed in the multi-layer emotional space; wherein N is a positive integer greater than 2, and i is a positive integer greater than 1 and less than N-1.

[0157] In some embodiments, the prediction module 2554 is further used to perform entity attribute mapping processing on the spatial conversion feature to obtain a corresponding first mapping feature; and perform bias processing on the first mapping feature to obtain an entity attribute corresponding to the text to be parsed.

[0158] In some embodiments, the prediction module 2554 is also used to perform sentiment polarity mapping processing on the spatial conversion feature to obtain a corresponding second mapping feature; perform bias processing on the second mapping feature to obtain a sentiment category corresponding to the text to be parsed; and perform sentiment mapping processing on the entity attribute based on the sentiment category to obtain the sentiment polarity of the entity attribute.

[0159] In some embodiments, the sentiment analysis method is implemented by calling a sentiment analysis model, and the sentiment analysis model includes: a feature extraction layer, an attention layer, a spatial conversion layer, an entity recognition layer and a sentiment classification layer; the device also includes: a training module, which is used to extract a feature sequence corresponding to a training sample through the feature extraction layer, and the training sample is annotated with an entity attribute label and a sentiment category label; through the attention layer, the feature sequence corresponding to the training sample is subjected to attention processing to obtain an attention feature corresponding to the training sample; through the spatial conversion layer, the attention feature of the training sample is subjected to spatial conversion processing to obtain a spatial conversion feature of the training sample in a multi-layer sentiment space; through the entity recognition layer, entity attribute recognition processing is performed based on the spatial conversion feature to obtain an entity attribute recognition result corresponding to the training sample, and sentiment mapping processing is performed based on the spatial conversion feature to obtain a sentiment classification result corresponding to the training sample; based on the entity attribute recognition result and the entity attribute label corresponding to the training sample, and the sentiment classification result and the sentiment category label corresponding to the training sample, the model parameters of the sentiment analysis model are updated.

[0160] In some embodiments, the training module is also used to perform one-hot encoding processing on the entity attribute labels and the sentiment category labels corresponding to the training samples, respectively, to obtain the entity attribute label encoding features and the sentiment category label encoding features corresponding to the training samples; determine the first loss function of the entity recognition layer according to the entity attribute recognition results and the entity attribute label encoding features; determine the second loss function of the sentiment classification layer according to the sentiment classification results and the sentiment category label encoding features; determine the third loss function of the sentiment analysis model according to the first loss function and the second loss function, and update the model parameters of the sentiment analysis model based on the third loss function.

[0161] The embodiment of the present application provides a computer program product, which includes a computer program or a computer executable instruction, and the computer program or the computer executable instruction is stored in a computer-readable storage medium. The processor of the electronic device reads the computer executable instruction from the computer-readable storage medium, and the processor executes the computer executable instruction, so that the electronic device performs the above-mentioned sentiment analysis method of the embodiment of the present application.

[0162] The embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions or computer programs are stored. When the computer-executable instructions or computer programs are executed by a processor, the processor will be caused to execute the sentiment analysis method provided by the embodiment of the present application, for example, Figure 3A The sentiment analysis method is shown.

[0163] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0164] In some embodiments, computer executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0165] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file storing other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0166] As an example, computer executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.

[0167] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A sentiment analysis method, characterized in that: The method comprises: Obtaining a feature sequence corresponding to the text to be parsed, wherein the feature sequence includes multiple encoding features; Performing attention processing on each encoded feature in the feature sequence to obtain an attention feature of the text to be parsed; Performing spatial transformation processing on the attention feature to obtain the spatial transformation feature of the text to be parsed; Entity attribute recognition processing is performed based on the spatial conversion feature to obtain the entity attribute corresponding to the text to be parsed, and sentiment mapping processing is performed based on the spatial conversion feature to obtain the sentiment polarity of the entity attribute.

2. The method according to claim 1, characterized in that The step of obtaining a feature sequence corresponding to the text to be parsed includes: Encoding each word in the text to be parsed to obtain encoding features of each word; Encoding the emotion category identifier in the text to be parsed to obtain the encoding features of the emotion category identifier; Generate a feature sequence of the text to be parsed based on the encoding features of each of the word segments and the encoding features of the sentiment category identifier; Among them, the coding feature of the emotion category identifier is located at the first place of the feature sequence, and the order of the coding feature of each word segment in the feature sequence is the same as the order of the corresponding word segment in the text to be parsed.

3. The method according to claim 2, characterized in that The encoding process is performed on each word in the text to be parsed to obtain the encoding features of each word, including: The following processing is performed for each of the word segments: Performing vectorization processing on the word segmentation to obtain a word vector corresponding to the word segmentation; According to the position of the segmented word in the text to be parsed, the segmented word is subjected to position encoding processing to obtain a position vector of the segmented word; Determine the belonging vector of the word according to the sentence to which the word belongs in the text to be parsed; The encoding feature of the word segmentation is obtained according to the word vector of the word segmentation, the position vector of the word segmentation and the belonging vector of the word segmentation.

4. The method according to claim 1, characterized in that The performing attention processing on each encoded feature in the feature sequence to obtain the attention feature corresponding to the text to be parsed includes: Performing linear transformation processing on each coding feature in the feature sequence to obtain a query feature, a key feature, and a value feature corresponding to each coding feature; Performing association processing based on the query feature and the key feature corresponding to each of the coding features to obtain an attention weight for each of the coding features; Based on the attention weight of each of the coding features, a weighted summation process is performed on the value feature of each of the coding features to obtain the attention feature corresponding to the text to be parsed.

5. The method according to claim 4, characterized in that The associating process is performed based on the query feature and the key feature corresponding to each of the coding features to obtain the attention weight of each of the coding features, including: The following processing is performed for each of the coding features: performing association processing based on the query feature and the key feature to obtain a first impact factor of the coding feature, wherein the first impact factor is positively correlated with the transposition of the query feature and the key feature, respectively, and is negatively correlated with the length of the coding feature; Acquire a maximum prediction length of the feature sequence and a scaling factor for controlling a scaling degree of the maximum prediction length, and determine a second influencing factor of the coding feature based on the scaling factor, the maximum prediction length and the first influencing factor; Maximum likelihood processing is performed based on the first influencing factor and the second influencing factor to obtain an attention weight of the encoding feature.

6. The method according to claim 1, characterized in that The performing spatial transformation processing on the attention feature to obtain the spatial transformation feature of the text to be parsed includes: Obtain at least one spatial transformation layer; The at least one spatial transformation layer is used to perform spatial transformation processing on the attention feature to obtain the spatial transformation feature of the text to be parsed.

7. The method according to claim 6, characterized in that The spatial transformation layer includes a linear transformation layer and a nonlinear transformation layer. The at least one spatial transformation layer is used to perform spatial transformation processing on the attention feature to obtain the spatial transformation feature of the text to be parsed, including: When the number of the spatial transformation layers is N, the attention feature is linearly transformed by a linear transformation layer in the first spatial transformation layer to obtain a first intermediate transformation feature, and the first intermediate transformation feature is nonlinearly mapped by a nonlinear transformation layer in the first spatial transformation layer to obtain a first spatial transformation feature; Performing linear transformation processing on the i-1th spatial transformation feature output by the i-1th spatial transformation layer through the linear transformation layer in the i-th spatial transformation layer to obtain an i-th intermediate transformation feature, and performing nonlinear mapping processing on the i-th intermediate transformation feature through the nonlinear transformation layer in the i-th spatial transformation layer to obtain an i-th spatial transformation feature; Determine the Nth spatial conversion feature output by the Nth spatial conversion layer as the spatial conversion feature of the text to be parsed; Wherein, N is a positive integer greater than 2, and i is a positive integer greater than 1 and less than N-1.

8. The method according to claim 1, characterized in that The entity attribute recognition process is performed based on the spatial conversion feature to obtain the entity attribute corresponding to the text to be parsed, including: Performing entity attribute mapping processing on the space conversion feature to obtain a corresponding first mapping feature; The first mapping feature is subjected to bias processing to obtain entity attributes corresponding to the text to be parsed.

9. The method according to claim 8, characterized in that The performing sentiment mapping processing based on the spatial conversion feature to obtain the sentiment polarity of the entity attribute includes: Performing sentiment polarity mapping processing on the spatial conversion feature to obtain a corresponding second mapping feature; Performing bias processing on the second mapping feature to obtain a sentiment category corresponding to the text to be parsed; Performing sentiment mapping processing on the entity attribute based on the sentiment category to obtain the sentiment polarity of the entity attribute.

10. The method according to any one of claims 1 to 9, characterized in that: The sentiment analysis method is implemented by calling a sentiment analysis model, wherein the sentiment analysis model includes: a feature extraction layer, an attention layer, a space conversion layer, an entity recognition layer and a sentiment classification layer; the method also includes: Extracting a feature sequence corresponding to a training sample through the feature extraction layer, wherein the feature sequence includes a plurality of coding features, and the training sample is annotated with an entity attribute label and a sentiment category label; Through the attention layer, attention processing is performed on each encoded feature in the feature sequence corresponding to the training sample to obtain an attention feature corresponding to each training sample; Performing spatial transformation processing on the attention features of the training samples through the spatial transformation layer to obtain spatial transformation features of each of the training samples; By means of the entity recognition layer, entity attribute recognition processing is performed based on the spatial conversion features to obtain entity attribute recognition results corresponding to the training samples, and by means of the sentiment classification layer, sentiment mapping processing is performed based on the spatial conversion features to obtain sentiment classification results corresponding to the training samples; Based on the entity attribute recognition result and the entity attribute label corresponding to the training sample, and the sentiment classification result and the sentiment category label corresponding to the training sample, the model parameters of the sentiment analysis model are updated.

11. The method according to claim 10, characterized in that The updating of the model parameters of the sentiment analysis model based on the entity attribute recognition result and the entity attribute label corresponding to the training sample, and the sentiment classification result and the sentiment category label corresponding to the training sample, comprises: Performing one-hot encoding processing on the entity attribute labels and the sentiment category labels corresponding to the training samples respectively, to obtain entity attribute label encoding features and sentiment category label encoding features corresponding to the training samples; Determining a first loss function of the entity recognition layer according to the entity attribute recognition result and the entity attribute label encoding feature; Determining a second loss function of the sentiment classification layer according to the sentiment classification result and the sentiment category label encoding feature; According to the first loss function and the second loss function, a third loss function of the sentiment analysis model is determined, and model parameters of the sentiment analysis model are updated based on the third loss function.

12. A sentiment analysis device, characterized in that: The device comprises: An acquisition module, used to acquire a feature sequence corresponding to the text to be parsed, wherein the feature sequence includes a plurality of encoding features; An attention module is used to perform attention processing on each encoded feature in the feature sequence to obtain an attention feature corresponding to the text to be parsed; A conversion module, used for performing spatial conversion processing on the attention feature to obtain the spatial conversion feature of the text to be parsed; The prediction module is used to perform entity attribute recognition processing based on the spatial conversion feature to obtain the entity attribute corresponding to the text to be parsed, and perform sentiment mapping processing based on the spatial conversion feature to obtain the sentiment polarity of the entity attribute.

13. An electronic device, characterized in that: include: A memory for storing computer executable instructions or computer programs; A processor, used to implement the sentiment analysis method described in any one of claims 1 to 11 when executing the computer executable instructions or computer programs stored in the memory.

14. A computer-readable storage medium, characterized in that: Computer executable instructions or computer programs are stored, and when the computer executable instructions or computer programs are executed by a processor, the sentiment analysis method according to any one of claims 1 to 11 is implemented.

Images