Emotion understanding model training method and device, emotion understanding method and equipment

Abstract

The invention discloses an emotion understanding model training method and device, an emotion understanding method and equipment, and the method comprises the steps: obtaining a text sample from a training sample set, and enabling the text sample to comprise an emotion label; inputting the text sample into a preset network model to enable the preset network model to map the text sample into word vectors, capturing a local dependency relationship and a global dependency relationship between words by using the word vectors through a self-attention mechanism, and generating emotion prediction information corresponding to the text sample based on the local dependency relationship and the global dependency relationship, the word vector comprises position information of a word corresponding to the word vector in the text sample; calculating training loss based on the emotion prediction information and the emotion label, and updating a preset network model by using the training loss to obtain an updated preset network model; and when a training ending condition is satisfied, determining the currently updated preset network model as an emotion understanding model. Therefore, the emotion of the text can be accurately understood.

Description

Technical Field

[0001] The present application relates to the field of natural language processing technology, and in particular to a sentiment understanding model training method and device, and a sentiment understanding method and device. Background Art

[0002] NLP (Natural Language Processing) is an important direction in the fields of computer science and artificial intelligence. By studying natural language processing, we can realize the method of effective communication between people and computers using natural language. The main tasks of natural language processing include machine translation, sentiment analysis, question-answering system, text classification, information extraction, semantic understanding, text generation, etc. Among them, how to accurately understand the emotions contained in the text is an important problem faced in the sentiment analysis task. Summary of the invention

[0003] In view of this, the purpose of this application is to provide a sentiment understanding model training method and device, sentiment understanding method and equipment, which can ensure accurate understanding of the sentiment of the text. The specific scheme is as follows:

[0004] In a first aspect, the present application discloses a method for training a sentiment understanding model, comprising:

[0005] Acquire a text sample from a training sample set, wherein the text sample contains a sentiment label;

[0006] Inputting the text sample into a preset network model so that the preset network model maps the text sample into a word vector, using the word vector to capture local dependencies and global dependencies between words through a self-attention mechanism, and generating sentiment prediction information corresponding to the text sample based on the local dependencies and the global dependencies, wherein the word vector includes position information of the word corresponding to the word vector in the text sample;

[0007] Calculating a training loss based on the emotion prediction information and the emotion label, and using the training loss to update the preset network model to obtain an updated preset network model;

[0008] When the training end condition is met, the currently updated preset network model is determined as the emotion understanding model.

[0009] Optionally, also include:

[0010] Acquire an initial text, and perform word segmentation on the initial text;

[0011] Using a preset mask and random words to mask the text after word segmentation, and adding sentiment labels to obtain the text sample;

[0012] The preset mask is used according to a first preset probability, and the random word is used according to a second preset probability.

[0013] Optionally, before segmenting the initial text, the method further includes:

[0014] Performing a preprocessing operation on the initial text;

[0015] The preprocessing operation includes a deduplication operation and / or a unified format operation.

[0016] Optionally, the word vector is used to capture the global dependency between words through a self-attention mechanism, including:

[0017] Generate a corresponding word representation based on the word vector and the local dependency;

[0018] The word representations are utilized through a self-attention mechanism to capture the global dependencies between words.

[0019] Optionally, generating the sentiment prediction information corresponding to the text sample based on the local dependency and the global dependency includes:

[0020] Processing the word vector using a feedforward neural network in the preset network model to obtain an output representation of the feedforward neural network;

[0021] generating a first target representation of a word based on the output representation, the local dependencies, and the global dependencies;

[0022] Processing the first target representation using a residual network to obtain a second target representation of the word;

[0023] Generate sentiment prediction information corresponding to the text sample based on the second target representation.

[0024] Optionally, the calculating the training loss based on the emotion prediction information and the emotion label includes:

[0025] The training loss is calculated based on the sample weight, the focal loss factor, the emotion prediction information, and the emotion label.

[0026] In a second aspect, the present application discloses a method for emotion understanding, comprising:

[0027] Obtain the target text to be understood;

[0028] Performing word segmentation on the target text to obtain a model input text;

[0029] Inputting the model input text into the emotion understanding model to obtain the emotion understanding result output by the emotion understanding model;

[0030] Wherein, the emotion understanding model is trained by the aforementioned emotion understanding model training method.

[0031] In a third aspect, the present application discloses a sentiment understanding model training device, comprising:

[0032] A text sample acquisition module, used to acquire text samples from a training sample set, wherein the text samples contain sentiment labels;

[0033] A network model prediction module, used for inputting the text sample into a preset network model so that the preset network model maps the text sample into a word vector, using the word vector to capture local dependencies and global dependencies between words through a self-attention mechanism, and generating sentiment prediction information corresponding to the text sample based on the local dependencies and the global dependencies, wherein the word vector includes position information of the word corresponding to the word vector in the text sample;

[0034] A training loss calculation module, used to calculate the training loss based on the emotion prediction information and the emotion label, and update the preset network model using the training loss to obtain an updated preset network model;

[0035] The training end control module is used to determine the currently updated preset network model as the emotion understanding model when the training end condition is met.

[0036] In a fourth aspect, the present application discloses an electronic device, including a memory and a processor, wherein:

[0037] The memory is used to store the computer program;

[0038] The processor is used to execute the computer program to implement the aforementioned emotion understanding model training method and / or emotion understanding method.

[0039] In a fifth aspect, the present application discloses a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned emotion understanding model training method and / or emotion understanding method.

[0040] It can be seen from the above scheme that the present application provides a method for training a sentiment understanding model, including: obtaining a text sample from a training sample set, wherein the text sample contains a sentiment label; inputting the text sample into a preset network model so that the preset network model maps the text sample into a word vector, using the word vector to capture local dependencies and global dependencies between words through a self-attention mechanism, and generating sentiment prediction information corresponding to the text sample based on the local dependencies and the global dependencies, wherein the word vector contains position information of the word corresponding to the word vector in the text sample; calculating the training loss based on the sentiment prediction information and the sentiment label, and using the training loss to update the preset network model to obtain an updated preset network model; when the training end condition is met, the currently updated preset network model is determined as the sentiment understanding model.

[0041] It can be seen that the beneficial effects of the present application are: the preset network model is trained using text samples containing sentiment labels, the preset network model can map the text samples into word vectors, and the word vectors contain the position information of the words in the text samples, avoiding the problem that the model cannot recognize the vocabulary sequence due to changes in the word order, and capturing the local dependencies and global dependencies between words through the self-attention mechanism, and generating sentiment prediction information corresponding to the text samples based on the local dependencies and global dependencies. Through the self-attention mechanism, local information and global information can complement each other, which not only improves the efficiency of the model in processing local information, but also can capture the global semantic associations, and can ensure accurate understanding of the emotions of the text.

[0042] Correspondingly, the emotion understanding model training device, emotion understanding method, equipment and readable storage medium provided in this application also have the above-mentioned technical effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0044] Figure 1 A flow chart of a method for training an emotion understanding model provided in an embodiment of the present application;

[0045] Figure 2 A flow chart of an emotion understanding method provided in an embodiment of the present application;

[0046] Figure 3A schematic diagram of the structure of an emotion understanding model training device provided in an embodiment of the present application;

[0047] Figure 4 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0049] With the rapid development of AI (Artificial Intelligence) technology, LLM (Large Language Model) has become an important milestone in natural language processing. Large language models can be used to perform multiple tasks such as semantic understanding, translation, and generating coherent text based on understanding. Large language models are artificial intelligence models that use machine learning technology to understand and generate human language. They use neural network-based models and usually use natural language processing technology to process and calculate their output. NIP is a branch of artificial intelligence that focuses on enabling computers to understand, interpret, and generate text, so that LLM can perform multiple tasks such as text analysis, sentiment analysis, language translation, and speech recognition. However, existing technologies still have shortcomings in terms of adaptability to specific fields and processing efficiency.

[0050] See also Figure 1 As shown, the embodiment of the present application discloses a method for training a sentiment understanding model, comprising:

[0051] Step S11: Acquire a text sample from a training sample set, wherein the text sample contains a sentiment label.

[0052] The text sample can be understood as the text obtained by segmenting the initial text, that is, including each word in the initial text. The sentiment tag can be a sentiment score for each word. In this embodiment, a sentiment score can be assigned to each word. The sentiment score is determined based on the sentiment polarity of the word. The sentiment polarity can include positive sentiment, negative sentiment, and in some embodiments, neutral sentiment. For example, the more positive the sentiment, the higher the score.

[0053] In an optional implementation, an initial text may be obtained and the initial text may be segmented; the segmented text may be masked using a preset mask and random words, and sentiment tags may be added to obtain the text sample; wherein the preset mask is used according to a first preset probability, and the random words are used according to a second preset probability.

[0054] Among them, the initial text can be segmented using a word segmentation tool. Masking can be understood as replacing the original word with a preset mask or a random word. In addition, in the embodiment of the present application, the position of the masking can also be randomly selected. Through dynamic masking, not only the position of the mask can be randomly selected, but also the method of masking can be randomly selected. In this way, the diversity of the mask is increased, so that the model can be exposed to more variations during the training process, thereby improving the model performance.

[0055] Furthermore, before the segmentation of the initial text, the method further includes: performing a preprocessing operation on the initial text; wherein the preprocessing operation includes a deduplication operation and / or a format unification operation. The deduplication operation can be understood as an operation of removing repeated words from consecutive words. The format unification operation can include one or more operations such as converting proper nouns to standard writing, converting uppercase and lowercase letters, and removing spaces.

[0056] That is, the embodiment of the present application first collects and preprocesses data, then performs word segmentation and adds sentiment tags to the words to obtain a training sample set. The preprocessing operation may include deduplication and unified format. Among them, deduplication can remove duplicate content from the text data to ensure that each text segment is processed only once. For example: input the initial text "In recent years, with the rapid development of artificial intelligence technology, natural language processing (NATural LANguage Processing, NLP) has become a research hotspot. Many enterprises and companies are actively developing NLP applications, including but not limited to machine translation, sentiment analysis, text summarization, question-and-answer systems, etc. These applications have not only greatly promoted the convenience of information exchange, but also had a profound impact on multiple industries.", and obtain the result, and remove the repeated content "enterprise" and other parts in the initial text. The unified format can unify certain specific formats in the text into a consistent form, such as converting proper nouns to standard writing, converting uppercase and lowercase, removing spaces, etc. Further, for word segmentation, the jieba word segmentation tool can be used to split continuous text strings into basic unit words with practical meaning.

[0057] Furthermore, dynamic masking is performed. For example, 15% of the tokens (i.e., words) are selected for masking and replaced with [MASK] with a probability of 80%, with a probability of 10% being replaced with random words, and with a probability of 10% remaining unchanged. There are 48 tokens in the given list, so 15% is about 7 tokens. The processing results are: ["[MASK]", ",", "With", "artificial intelligence", "technology", "of", "[MASK]", ",", "Natural language processing", "(", "NATURAL", "LANGUAGE", "PROCESSING", ",", "NLP", ")", "has", "become", "research", "hotspot", ".", "many", "companies", "are", "enthusiastically", "developing", "NLP", "applications", ",", "including", "but", "not limited to", "machine translation", ",", "sentiment analysis", ",", "text summarization", ",", "question-answering system", "[MASK]", "。","these", "applications", "not only", "greatly", "promoted", "information exchange", "[MASK]", "convenience", ",", "but also", "in", "multiple", "industries", "produced", "far-reaching", "of", "impact","."]. Dynamic masking allows you to randomly select not only the position of the mask, but also the way the mask is applied. For example, in addition to the commonly used replacement with the [MASK] tag, you can also randomly choose to replace it with a random word or keep it as it is. This increases the diversity of the mask, allowing the model to be exposed to more variations during training.

[0058] Step S12: Input the text sample into a preset network model so that the preset network model maps the text sample into a word vector, uses the word vector to capture local dependencies and global dependencies between words through a self-attention mechanism, and generates sentiment prediction information corresponding to the text sample based on the local dependencies and the global dependencies, wherein the word vector contains position information of the word corresponding to the word vector in the text sample.

[0059] The preset network model is a deep learning model. In an optional implementation, it can be an initial network model. In another implementation, the preset network model can be a pre-trained model. A model can be pre-trained on a large-scale corpus so that the model can understand the relationship between words and capture semantic features at the vocabulary and sentence levels. By training on a large amount of unlabeled data, the model can learn a more general representation, which helps to improve its generalization ability on different tasks.

[0060] The preset network model includes a word vector model. The embodiment of the present application can map the text sample to a word vector using the word vector model. For example, the word vector mapping network structure is BERT (i.e., Bidirectional Encoder Representations from Transformers, a bidirectional encoding representation based on Transformer (i.e., a neural network model), which is a pre-trained language representation model). In this embodiment, pre-processed words can be input, and the pre-trained word vector model BERT can be used to map the words in the text to a vector in a high-dimensional space, such as: [[0.1, 0.2,..., 0.5], [0.2, 0.3, ..., 0.6], ..., [0.4, 0.5, ..., 0.9]]. The vector expresses the meaning information of the vocabulary through numericalization.

[0061] Furthermore, the word vector is positionally encoded, and the word position information is added to the word vector. For the word vector obtained in the previous step, the position information is added to the word vector, and the result is [[0.1, 0.2, ..., 0.5, pos1],[0.2, 0.3, ..., 0.6, pos2], ..., [0.4, 0.5, ..., 0.9, posn]]. posn represents the position information. Position encoding is a special vector that is attached to the word vector to help the model understand the relative position of the word in the sentence. It is particularly important for self-attention mechanisms (such as the Transformer model) because the self-attention mechanism itself does not contain any information about the position of the word in the sentence. Since the self-attention mechanism cannot directly obtain the position information of the word, this leads to a problem: for the same word sequence, if their order changes, the model will not be able to distinguish between the two situations. For example, in chess, the two sentences "rook eats pawn" and "pawn eats rook" have the same words, but the order is different, and the meanings are completely different. If the model does not have position information, it will not be able to distinguish between the two sentences.

[0062] In this embodiment, using the word vectors through the self-attention mechanism to capture local dependencies between words may include: using preset sliding windows to divide the word vector sequence corresponding to the text sample; using the self-attention mechanism, capturing local dependencies between words based on the word vectors in each preset sliding window. Further, in this embodiment, using the word vectors through the self-attention mechanism to capture global dependencies between words may include: generating corresponding word representations based on the word vectors and the local dependencies; and using the word representations through the self-attention mechanism to capture global dependencies between words. That is, when capturing global dependencies between words, the embodiment of the present application also takes local dependencies into consideration, which can improve the accuracy of global dependencies.

[0063] In this embodiment, generating the sentiment prediction information corresponding to the text sample based on the local dependency and the global dependency may include: processing the word vector using the feedforward neural network in the preset network model to obtain the output representation of the feedforward neural network; generating a first target representation of the word based on the output representation, the local dependency and the global dependency; processing the first target representation using a residual network to obtain a second target representation of the word; and generating the sentiment prediction information corresponding to the text sample based on the second target representation.

[0064] In a specific implementation, this embodiment can calculate the local attention score for each word in the sliding window. Specifically, the attention score is calculated based on the word vector in the preset sliding window to obtain the local attention score, the local attention score is normalized, and the normalized local attention score is used as the weight to perform weighted summation on the word vector in the window to obtain a weighted context representation. As a local dependency, this weighted context representation is combined with the original representation of the word, that is, the word vector with added position information, to obtain the corresponding word representation, which can be achieved by addition, splicing or more complex fusion mechanisms. Then, the word representation can be used to capture the global dependency between words through the self-attention mechanism.

[0065] That is, the present application implements local self-attention, which captures the dependencies between words through the self-attention mechanism, allowing the model to focus on different parts of the text when processing the text. The three different linear transformations of the input vector query, key and value respectively obtain the query vector, key vector and value vector. The query vector is dot-producted (inner product) with the key vector to obtain an attention score matrix, each element of which represents the degree of match between the query vector and the key vector. The attention score matrix is ​​softmax-normalized to obtain a normalized attention weight matrix, and the value vector is weighted and summed using the normalized attention weight matrix to obtain a weighted output vector. Add an attention weight vector (i.e., the aforementioned weighted output vector) to the position encoding vector. For example, it can be: [[0.1, 0.2, ..., 0.5, pos1,attn1], [0.2, 0.3, ..., 0.6, pos2, attn2], ..., [0.4, 0.5, ..., 0.9, posn,attnn]], where attnn represents the local dependency, i.e., the attention weight vector, which reflects the correlation between the current element and other elements. The local dependency can be obtained by sliding the window, and the sliding window is used to only care about the words within the window each time. The degree of overlap of adjacent words can be controlled by adjusting the step size of the window sliding. Through local self-attention, the model can more effectively capture the dependency between words. And by adjusting the window size and sliding step size, only the nearest words in the input sequence can be focused on, which reduces the amount of computation while also making the model more flexible in different tasks and data sets.

[0066] Further, for global information transfer, this embodiment captures the global information in the entire input sequence through a global attention mechanism and integrates it into the representation of each word. The purpose of this is to enable the model to better capture long-distance dependencies and enhance the expressiveness of the local self-attention mechanism. The global query vector and the global attention score are calculated by the vocabulary representation with position encoding and local attention score obtained in the previous step, and then the normalized global attention score is obtained. The global attention score is added to the representation of each word by weighted summing the global attention score with the original representation (i.e., the word representation containing local dependencies), and the result is [[0.1, 0.2, ..., 0.5, pos1, attn1, global_attn1], [0.2, 0.3, ..., 0.6, pos2, attn2, global_attn2],..., [0.4, 0.5, ..., 0.9, posn, attnn, global_attnn]]. global_attnn represents global dependencies. By complementing local self-attention with global information transfer, the model not only improves the efficiency of processing local information, but also captures global semantic associations.

[0067] Furthermore, a feedforward neural network is a neural network without feedback connections, where data is passed unidirectionally from the input layer to the output layer. In this step, the feedforward network is used to further extract features. The input layer of the feedforward neural network receives the pure vocabulary representation after removing the position encoding and attention score, and processes it through the feedforward network, adding the output to the original representation to form the final representation [[0.1, 0.2, ..., 0.5, pos1, attn1, global_attn1,ffn1], [0.2, 0.3, ..., 0.6, pos2, attn2, global_attn2, ffn2], ..., [0.4, 0.5,..., 0.9, posn, attnn, global_attnn, ffnn]]. ffnn represents the output obtained by the feedforward neural network processing.

[0068] After that, residual connection and batch normalization are performed. Residual connection is used to add direct connection between network layers to facilitate the training of deep networks. This connection helps alleviate the gradient vanishing problem. Batch normalization is used to speed up the training process and make the model more robust. This is achieved by normalizing small batches of data. The final representation obtained by the above feedforward neural network is used as input, and the residual structure is input to obtain the result of residual connection. Batch normalization is usually applied to the feature dimension (ie, the number of channels), so it is necessary to transpose the data first so that the feature dimension becomes the second dimension (ie, the input dimension of the batch normalization layer). The result of the residual connection is normalized through the batch normalization layer, and the data is transposed again to restore it to its original shape. After the above processing, the new representation is obtained [[0.1, 0.2, ..., 0.5, pos1, attn1,global_attn1, ffn1, norm1], [0.2, 0.3, ..., 0.6, pos2, attn2, global_attn2,ffn2, norm2], ..., [0.4, 0.5, ..., 0.9, posn, attnn, global_attnn, ffnn,normn]]. normn represents the normalization result, which is converted into sentiment prediction information to give a prediction score for each word.

[0069] Step S13: Calculate the training loss based on the emotion prediction information and the emotion label, and use the training loss to update the preset network model to obtain an updated preset network model.

[0070] The embodiment of the present application can calculate the training loss based on the sample weight, the focus loss factor, the emotion prediction information and the emotion label. And, in an optional implementation, the training loss is a regularized training loss. The focus loss factor is a modulation factor to reduce the weight of simple samples (usually samples that are easy to classify) and focus the training on difficult samples.

[0071] An adaptive loss function is used in the embodiment of the present application to improve the generalization ability of the model by dynamically adjusting the width of the classification boundary. First, the basic loss of each sample, such as the cross entropy loss, is calculated, and the sample weight is calculated according to the difficulty of the sample or other prior knowledge. The sample weight and the focal loss factor are combined to form an adaptive loss function. In addition, regularization can be used to prevent the model from overfitting during the training process. For example, L2 regularization can be used, and L2 regularization is achieved by adding the sum of the square values ​​of the parameters to the loss function. L2 regularization can prevent parameters from being too large, but will not cause the parameters to become zero, thereby helping the model to better generalize to new data.

[0072] Step S14: When the training end condition is met, the currently updated preset network model is determined as the emotion understanding model.

[0073] The training end condition may be that the training loss is less than a preset threshold, or the number of model iterations reaches a preset threshold.

[0074] In this embodiment, the emotion understanding model can be a large model language model, and this application implements an architecture design and training method that can significantly improve the performance and practicality of the large model. That is, the embodiment of this application can provide a new system for the large model language model, which is effectively optimized in terms of model performance, usage efficiency, multi-scenario practicality, and enhanced interpretability through various means such as local self-attention and global information transmission, emotion understanding and adaptability, making natural language processing more accurate and efficient.

[0075] It can be seen that the embodiment of the present application uses text samples containing sentiment labels to train the preset network model. The preset network model can map the text samples into word vectors, and the word vectors contain the position information of the words in the text samples, thereby avoiding the problem that the model cannot recognize the vocabulary sequence due to changes in the order of words. In addition, the local dependencies and global dependencies between words are captured through the self-attention mechanism, and the sentiment prediction information corresponding to the text samples is generated based on the local dependencies and the global dependencies. Through the self-attention mechanism, local information and global information can complement each other, which not only improves the efficiency of the model in processing local information, but also can capture the global semantic associations, and can ensure accurate understanding of the emotions of the text.

[0076] In addition, the emotional language understanding model provided in this embodiment can be used to process text generation, dialogue systems, question-answering systems and other fields in natural language. In addition, a multimodal fusion architecture can also be implemented, which can not only process text data, but also integrate other modal information such as speech and images. By using a shared embedding space and a cross-modal attention mechanism, the model can learn semantic associations between different modalities and enhance the depth and richness of language understanding. For example, visual features are used to assist in understanding the implicit meaning in the text, or the emotional understanding of the dialogue is enhanced through speech prosody. It is necessary to design a multilayer perceptron (MLP) to convert data of different modalities into a unified high-dimensional representation, and then use the attention mechanism to weightedly fuse these representations to ensure the effective transmission of key information. For example, a multilayer perceptron can be added to the emotional understanding model to convert text samples, image samples and / or speech samples into a unified representation, and then the emotional understanding model is subsequently processed. The specific performance indicators that the model needs to achieve are based on the BLEU (Bilingual Evaluation Understudy) score (used to evaluate machine translation), the ROUGE (Recall-Oriented Understudy for Gisting Evaluation) score (used to evaluate text summarization), and the accuracy rate (used for classification tasks).

[0077] See also Figure 2 As shown, the present application implements and discloses a method for emotion understanding, including:

[0078] Step S21: Obtain the target text to be understood.

[0079] Step S22: Segment the target text to obtain a model input text.

[0080] Step S23: inputting the model input text into the emotion understanding model to obtain the emotion understanding result output by the emotion understanding model;

[0081] The emotion understanding model is trained by the emotion understanding model training method disclosed in the aforementioned embodiment.

[0082] In the embodiment of the present application, the output layer of sentiment understanding can generate a category probability distribution from the sentiment prediction information, i.e., the predicted sentiment score for each word, so that the model can make the final classification decision. The output of the previous layer is mapped to the required number of categories through the fully connected layer in the output layer, and the output is converted to a probability value between 0 and 1 using the Sigmoid function, resulting in tensor ([[0.75]]), which means that the probability that the input text belongs to positive sentiment is 0.75, and the probability that it belongs to negative sentiment is 0.25.

[0083] See also Figure 3 As shown, the embodiment of the present application discloses a sentiment understanding model training device, comprising:

[0084] A text sample acquisition module 11 is used to acquire text samples from a training sample set, wherein the text samples contain sentiment labels;

[0085] A network model prediction module 12 is used to input the text sample into a preset network model so that the preset network model maps the text sample into a word vector, uses the word vector to capture local dependencies and global dependencies between words through a self-attention mechanism, and generates sentiment prediction information corresponding to the text sample based on the local dependencies and the global dependencies, wherein the word vector contains position information of the word corresponding to the word vector in the text sample;

[0086] A training loss calculation module 13, used to calculate the training loss based on the emotion prediction information and the emotion label, and update the preset network model using the training loss to obtain an updated preset network model;

[0087] The training end control module 14 is used to determine the currently updated preset network model as the emotion understanding model when the training end condition is met.

[0088] The device also includes:

[0089] A word segmentation module is used to obtain an initial text and perform word segmentation on the initial text;

[0090] A mask and label adding module, used to perform mask processing on the text after word segmentation using a preset mask and random words, and add sentiment labels to obtain the text sample;

[0091] The preset mask is used according to a first preset probability, and the random word is used according to a second preset probability.

[0092] A preprocessing module is used to perform a preprocessing operation on the initial text before segmenting the initial text; wherein the preprocessing operation includes a deduplication operation and / or a format unification operation.

[0093] The network model prediction module 12 is specifically used to generate corresponding word representations based on the word vectors and the local dependencies; and to capture the global dependencies between words using the word representations through a self-attention mechanism.

[0094] Furthermore, the network model prediction module 12 is specifically used to process the word vector using the feedforward neural network in the preset network model to obtain the output representation of the feedforward neural network; generate a first target representation of the word based on the output representation, the local dependency relationship and the global dependency relationship; process the first target representation using the residual network to obtain the second target representation of the word; and generate sentiment prediction information corresponding to the text sample based on the second target representation.

[0095] The training loss calculation module 13 is specifically used to calculate the training loss based on the sample weight, the focus loss factor, the emotion prediction information and the emotion label.

[0096] It can be seen that the embodiment of the present application uses text samples containing sentiment labels to train the preset network model. The preset network model can map the text samples into word vectors, and the word vectors contain the position information of the words in the text samples, thereby avoiding the problem that the model cannot recognize the vocabulary sequence due to changes in the order of words. In addition, the local dependencies and global dependencies between words are captured through the self-attention mechanism, and the sentiment prediction information corresponding to the text samples is generated based on the local dependencies and the global dependencies. Through the self-attention mechanism, local information and global information can complement each other, which not only improves the efficiency of the model in processing local information, but also can capture the global semantic associations, and can ensure accurate understanding of the emotions of the text.

[0097] See also Figure 4 As shown, an embodiment of the present application discloses an electronic device 20, including a processor 21 and a memory 22; wherein the memory 22 is used to store a computer program; the processor 21 is used to execute the computer program, the emotion understanding model training method disclosed in the above embodiment, and / or the emotion understanding method.

[0098] Regarding the above-mentioned emotion understanding model training method, and / or the specific process of the emotion understanding method, reference may be made to the corresponding contents disclosed in the aforementioned embodiments, which will not be repeated here.

[0099] Furthermore, the memory 22 as a carrier for storing resources may be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the storage method may be temporary storage or permanent storage.

[0100] In addition, the electronic device 20 also includes a power supply 23, a communication interface 24, an input / output interface 25 and a communication bus 26; wherein the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and an external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input / output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0101] Furthermore, an embodiment of the present application also discloses a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the emotion understanding model training method and / or the emotion understanding method disclosed in the aforementioned embodiment.

[0102] Regarding the above-mentioned emotion understanding model training method, and / or the specific process of the emotion understanding method, reference may be made to the corresponding contents disclosed in the aforementioned embodiments, which will not be repeated here.

[0103] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0104] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0105] The above is a detailed introduction to the emotion understanding model training method and device, emotion understanding method, and equipment provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for training a sentiment understanding model, characterized in that: include: Acquire a text sample from a training sample set, wherein the text sample contains a sentiment label; Inputting the text sample into a preset network model so that the preset network model maps the text sample into a word vector, using the word vector to capture local dependencies and global dependencies between words through a self-attention mechanism, and generating sentiment prediction information corresponding to the text sample based on the local dependencies and the global dependencies, wherein the word vector includes position information of the word corresponding to the word vector in the text sample; Calculating a training loss based on the emotion prediction information and the emotion label, and using the training loss to update the preset network model to obtain an updated preset network model; When the training end condition is met, the currently updated preset network model is determined as the emotion understanding model.

2. The emotion understanding model training method according to claim 1, characterized in that: Also includes: Acquire an initial text, and perform word segmentation on the initial text; Using a preset mask and random words to mask the text after word segmentation, and adding sentiment labels to obtain the text sample; The preset mask is used according to a first preset probability, and the random word is used according to a second preset probability.

3. The emotion understanding model training method according to claim 2, characterized in that: Before the word segmentation of the initial text, the method further includes: Performing a preprocessing operation on the initial text; The preprocessing operation includes a deduplication operation and / or a unified format operation.

4. The emotion understanding model training method according to claim 1, characterized in that: The word vector is used to capture the global dependencies between words through the self-attention mechanism, including: Generate a corresponding word representation based on the word vector and the local dependency; The word representations are utilized through a self-attention mechanism to capture the global dependencies between words.

5. The emotion understanding model training method according to claim 1, characterized in that: The generating the sentiment prediction information corresponding to the text sample based on the local dependency relationship and the global dependency relationship includes: Processing the word vector using a feedforward neural network in the preset network model to obtain an output representation of the feedforward neural network; generating a first target representation of a word based on the output representation, the local dependencies, and the global dependencies; Processing the first target representation using a residual network to obtain a second target representation of the word; Generate sentiment prediction information corresponding to the text sample based on the second target representation.

6. The emotion understanding model training method according to any one of claims 1 to 5, characterized in that: The calculating the training loss based on the emotion prediction information and the emotion label includes: The training loss is calculated based on the sample weight, the focal loss factor, the emotion prediction information, and the emotion label.

7. A method for emotion understanding, characterized in that: include: Obtain the target text to be understood; Performing word segmentation on the target text to obtain a model input text; Inputting the model input text into the emotion understanding model to obtain the emotion understanding result output by the emotion understanding model; Wherein, the emotion understanding model is trained by the emotion understanding model training method described in any one of claims 1 to 6.

8. A sentiment understanding model training device, characterized in that: include: A text sample acquisition module, used to acquire text samples from a training sample set, wherein the text samples contain sentiment labels; A network model prediction module, used for inputting the text sample into a preset network model so that the preset network model maps the text sample into a word vector, using the word vector to capture local dependencies and global dependencies between words through a self-attention mechanism, and generating sentiment prediction information corresponding to the text sample based on the local dependencies and the global dependencies, wherein the word vector includes position information of the word corresponding to the word vector in the text sample; A training loss calculation module, used to calculate the training loss based on the emotion prediction information and the emotion label, and update the preset network model using the training loss to obtain an updated preset network model; The training end control module is used to determine the currently updated preset network model as the emotion understanding model when the training end condition is met.

9. An electronic device, characterized in that: comprising a memory and a processor, wherein: The memory is used to store the computer program; The processor is used to execute the computer program to implement the emotion understanding model training method as described in any one of claims 1 to 6, and / or the emotion understanding method as described in claim 7.

10. A computer-readable storage medium, characterized in that: Used to save a computer program, wherein when the computer program is executed by a processor, it implements the emotion understanding model training method as described in any one of claims 1 to 6, and / or the emotion understanding method as described in claim 7.

Images