Emotion detection method, apparatus, terminal device, and readable storage medium

Abstract

The application discloses an emotion detection method and device, a terminal equipment and a readable storage medium, the method comprises the following steps: acquiring audio data, and converting the audio data into initial text data; splitting the initial text data into first text data and second text data, and adding bag labeling to the first text data; inputting the second text data into a preset language correction model to obtain third text data; inputting the third text data into an emotion detection model to detect emotion parameters corresponding to the third text data; based on the fact that the sample information collected by the bank outlets is inaccurate due to noise and noise, and single-sentence-level corpus labeling is difficult, the language model based on BERT is used to intelligently correct the corpus, the influence of noise data on the detection result is reduced, the mapping relationship from a paragraph to a single sentence is constructed through the emotion detection model, and finally the accuracy of the emotion detection of the bank outlets is improved.

Description

Technical Field

[0001] This invention relates to the field of data processing, and more particularly to an emotion detection method, apparatus, terminal device, and readable storage medium. Background Technology

[0002] As bank branch operations and management gradually move towards higher service quality, improving customer experience is one of the important goals, which requires branches to have the ability to accurately analyze customer sentiment.

[0003] Current methods for customer sentiment detection involve training a neural network using strongly supervised machine learning on a large amount of user sample data to obtain a sentiment detection model, which is then used to detect sentiment. The training process typically involves comparing the output of the neural network with the labels corresponding to the input samples, adjusting the weight parameters in the neural network based on the comparison results, until the model converges to obtain a detection model, which is then used to detect the input data and output the detection results.

[0004] The training method for this detection model is strongly supervised labeled training. Since the label is used as the standard for evaluating the model's output, the accuracy of the model's output results depends heavily on the quality of the label annotation of the input data during training. However, the input data processing dimension in the bank branch scenario is at the single sentence level. The label annotation of single sentence data needs to be combined with contextual semantic information. If the labels constructed during preprocessing only correspond to the input single sentence itself and do not combine with contextual semantic information, the labels are difficult to correctly express the actual meaning. Using incorrect labels will lead to a large deviation in the accuracy of the final model detection. Summary of the Invention

[0005] The main objective of this invention is to provide an emotion detection method, apparatus, terminal device, and readable storage medium, aiming to solve the technical problem that the labeling of single-sentence data is difficult during current model training, resulting in inaccurate emotion detection results, and to improve the reliability of the model.

[0006] To achieve the above objectives, the present invention provides an emotion detection method, the emotion detection method comprising the following steps:

[0007] Acquire audio data and convert the audio data into initial text data;

[0008] The initial text data is split into first text data and second text data, and bag labels are added to the first text data;

[0009] The second text data is input into a preset language error correction model to obtain the third text data;

[0010] The third text data is input into the emotion detection model, and the emotion parameters corresponding to the third text data are detected.

[0011] Optionally, before the step of inputting the third text data into the emotion detection model and detecting the emotion corresponding to the third text data based on the mapping relationship between the second text data and the third text data, the method further includes:

[0012] Obtain the training set corresponding to the third text data;

[0013] Based on the training set and the corresponding bag labels, and combining the mapping relationship between the training set and the bag labels, a pre-set neural network model is trained to obtain an emotion detection model.

[0014] Optionally, the step of training a pre-set neural network model based on the training set and the corresponding bag labels, combined with the mapping relationship between the training set and the bag labels, to obtain the emotion detection model includes:

[0015] Obtain the training result output by the preset neural network model after the training set corresponding to the third text data is input into the preset neural network model;

[0016] The training results and the bag labels are input into a preset loss function to obtain the loss value output by the loss function;

[0017] The preset neural network model is adjusted based on the loss value until it converges, thus obtaining the emotion detection model.

[0018] After the step of inputting the training results and the bag labels into a preset loss function to obtain the loss value output by the loss function, the method further includes:

[0019] The accuracy of the emotion detection model in detecting the emotion of the second text data is determined based on the loss value.

[0020] Optionally, the step of inputting the training results and the bag labels into a preset loss function to obtain the loss value output by the loss function includes:

[0021] Obtain all the training results corresponding to all the third text data;

[0022] Input all the training results and bag labels one by one into the preset loss function to obtain a set of loss values;

[0023] Select the maximum loss value from the set of loss values.

[0024] Optionally, before the step of inputting the second text data into a preset language error correction model to obtain the third text data, the method further includes:

[0025] Noise data is added to the second text data to obtain the fourth text data;

[0026] Based on the second and fourth text data, an autoencoder language model is trained to obtain the preset language error correction model.

[0027] Optionally, the step of splitting the initial text data into third text data and second text data, and adding bag labels to the third text data, includes:

[0028] Obtain keywords from the second text data and compare the keywords with a preset keyword library;

[0029] If the comparison results are consistent, then the emotional features corresponding to the keywords are used as the bag labels for the third text data corresponding to the second text data.

[0030] Furthermore, to achieve the above objectives, the present invention also provides an emotion detection device, the emotion detection device comprising:

[0031] The conversion module is used to acquire audio data and convert the audio data into initial text data;

[0032] The splitting module is used to split the initial text data into first text data and second text data, and add bag labels to the first text data;

[0033] The input module is used to input the second text data into a preset language error correction model to obtain the third text data;

[0034] The detection module is used to input the third text data into the emotion detection model and detect the emotion parameters corresponding to the third text data.

[0035] In addition, to achieve the above objectives, the present invention also provides a terminal device, the terminal device including a memory, a processor, and an emotion detection program stored in the memory and executable on the processor, wherein the emotion detection program, when executed by the processor, implements the steps of the emotion detection method as described above.

[0036] In addition, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing an emotion detection program, wherein when the emotion detection is executed by a processor, the steps of the emotion detection method described above are implemented.

[0037] This invention provides an emotion detection method, apparatus, terminal device, and readable storage medium. By acquiring audio data and converting it into initial text data, splitting the initial text data into first text data and second text data, adding bag annotations to the first text data, inputting the second text data into a preset language correction model to obtain third text data, and inputting the third text data into an emotion detection model to detect the emotion parameters corresponding to the third text data, the reliability of the text data is improved by converting audio data into text data and then calibrating the text data into a language model. Then, based on a multi-instance learning-based emotion detection model, a mapping relationship from paragraphs to single sentences is constructed, thereby reducing the difficulty of single-sentence-level prediction annotation and ultimately improving the accuracy of emotion detection. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the functional modules of the terminal device to which the emotion detection device of this application belongs;

[0039] Figure 2 This is a flowchart illustrating an exemplary embodiment of the emotion detection method of this application;

[0040] Figure 3 This is a flowchart illustrating another exemplary embodiment of the emotion detection method of this application;

[0041] Figure 4 This is a flowchart illustrating another exemplary embodiment of the emotion detection method of this application;

[0042] Figure 5 This is a flowchart illustrating another exemplary embodiment of the emotion detection method of this application;

[0043] Figure 6 This is a flowchart illustrating another exemplary embodiment of the emotion detection method of this application;

[0044] Figure 7 This is a flowchart illustrating another exemplary embodiment of the emotion detection method of this application;

[0045] Figure 8 This is the overall flowchart of the emotion detection method in this application.

[0046] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0047] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0048] The main solution of this invention is as follows: acquire audio data and convert it into initial text data; split the initial text data into first text data and second text data; add bag annotations to the first text data; input the second text data into a preset language error correction model to obtain third text data; input the third text data into an emotion detection model to detect the emotion parameters corresponding to the third text data. Based on the current situation where noise and other factors affect the accuracy of sample information collected at bank branches, and the difficulty of annotating single-sentence-level corpora, this invention uses a language model based on BERT (bidirectional encoder representations from transformers, a natural language algorithm) to intelligently correct errors in the corpus, reducing the impact of noise data on the detection results. The emotion detection model constructs a mapping relationship from paragraphs to single sentences, ultimately improving the accuracy of emotion detection at bank branches.

[0049] Specifically, refer to Figure 1 , Figure 1 This is a schematic diagram of the functional modules of the terminal device to which the emotion detection device of this application belongs. The emotion detection device can be independent of the terminal device, capable of acquiring audio data, converting audio data into text data, adding bag annotations to the text data, inputting second text data into a preset language correction model, inputting third text data into an emotion detection model, and obtaining the emotion detection result of the third text data. It can be implemented on the terminal device in hardware or software form. The terminal device can be a smart mobile terminal with emotion detection function, such as a mobile phone or tablet computer, or a fixed terminal device or server with emotion detection function.

[0050] In this embodiment, the terminal device to which the emotion detection device belongs includes at least an output module 110, a processor 120, a memory 130, and a communication module 140.

[0051] The memory 130 stores the operating system and emotion detection program, and the emotion detection device can store such information in the memory 130; the output module 110 can be a display screen, etc. The communication module 140 can include a WIFI module, a mobile communication module, and a Bluetooth module, etc., and communicates with external devices or servers through the communication module 140.

[0052] When the emotion detection program in memory 130 is executed by the processor, it performs the following steps:

[0053] Acquire audio data and convert the audio data into initial text data;

[0054] The initial text data is split into first text data and second text data, and bag labels are added to the first text data;

[0055] The second text data is input into a preset language error correction model to obtain the third text data;

[0056] The third text data is input into the emotion detection model, and the emotion parameters corresponding to the third text data are detected.

[0057] Furthermore, the emotion detection program in memory 130 performs the following steps before being executed by the processor:

[0058] Obtain the training set corresponding to the third text data;

[0059] Based on the training set and the corresponding bag labels, and combining the mapping relationship between the training set and the bag labels, a pre-set neural network model is trained to obtain an emotion detection model.

[0060] Furthermore, the emotion detection program in memory 130 performs the following steps before being executed by the processor:

[0061] Obtain the training result output by the preset neural network model after the training set corresponding to the third text data is input into the preset neural network model;

[0062] The training results and the bag labels are input into a preset loss function to obtain the loss value output by the loss function;

[0063] The preset neural network model is adjusted based on the loss value until it converges, thus obtaining the emotion detection model.

[0064] Furthermore, the emotion detection program in memory 130 performs the following steps before being executed by the processor:

[0065] The accuracy of the emotion detection model in detecting the emotion of the second text data is determined based on the loss value.

[0066] Furthermore, the emotion detection program in memory 130 performs the following steps before being executed by the processor:

[0067] Obtain all the training results corresponding to all the third text data;

[0068] Input all the training results and bag labels one by one into the preset loss function to obtain a set of loss values;

[0069] Select the maximum loss value from the set of loss values.

[0070] Furthermore, the emotion detection program in memory 130 performs the following steps before being executed by the processor:

[0071] Noise data is added to the second text data to obtain the fourth text data;

[0072] Based on the second and fourth text data, an autoencoder language model is trained to obtain the preset language error correction model.

[0073] Furthermore, the emotion detection program in memory 130 performs the following steps before being executed by the processor:

[0074] Obtain keywords from the second text data and compare the keywords with a preset keyword library;

[0075] If the comparison results are consistent, then the emotional features corresponding to the keywords are used as the bag labels for the third text data corresponding to the second text data.

[0076] This invention provides an emotion detection method, apparatus, terminal device, and readable storage medium. By acquiring audio data and converting it into initial text data, splitting the initial text data into first and second text data, adding bag annotations to the first text data, inputting the second text data into a preset language correction model to obtain third text data, and inputting the third text data into an emotion detection model to detect the emotion parameters corresponding to the third text data, the reliability of the text data is improved by converting audio data into text data and then calibrating the text data into a language model. Then, based on a multi-instance learning-based emotion detection model, and based on the mapping relationship from paragraphs to sentences (i.e., paragraph emotion as one of the reference factors for sentence emotion), the difficulty of sentence-level prediction and annotation is reduced, ultimately improving the accuracy of emotion detection.

[0077] Based on, but not limited to, the terminal device architecture described above, this application proposes method embodiments.

[0078] Reference Figure 2 , Figure 2 This is a flowchart illustrating an exemplary embodiment of the emotion detection method of this application. The emotion detection method includes:

[0079] Step S1001: Acquire audio data and convert the audio data into initial text data;

[0080] Specifically, audio data refers to sound files that are collected in real time or saved in the past in emotion detection scenarios. Then, the audio data is converted into initial text data through ARS (Automatic Speech Recognition) technology, that is, the conversion from "sound" to "text". One audio data is a record of a service. For example, when conducting business in a bank, each audio data corresponds to a record of the communication between a customer and a teller.

[0081] As mentioned in the background section, existing models struggle with emotion recognition in paragraph texts. The difficulty lies in the fact that the semantics of individual sentences within a paragraph need to be analyzed in conjunction with the actual time and contextual information. If only individual sentences are analyzed as a whole, the results are often biased. This application uses multi-instance learning for emotion detection. Multi-instance refers to the presence of multiple individual sentences within a paragraph. By matching corresponding emotion features to each individual sentence in the paragraph, the emotion features of the paragraph are determined—this is known as bag labeling. Based on the mapping relationship between individual sentences and the paragraph, the overall emotion of the paragraph can, to some extent, serve as a reference for the emotion of individual sentences. This is because paragraphs and sentences have a subordinate relationship; typically, we use the same emotional tone when narrating a paragraph. Starting from this point, we can improve the progress of individual sentence emotion detection. Then, the bag labels and individual sentence texts are input into a neural network model for training to obtain an emotion detection model. Inputting individual sentence texts into this model outputs the corresponding emotion labels, i.e., emotion features, which are added to the bag labels.

[0082] In one embodiment, when converting audio data using ARS, if the audio data corresponds to more than two people, different people can be identified based on their volume, timbre, dialect background, etc., and then the target person is determined based on their proportion, retaining only the audio data of the target person. For example, when using dialect background as the identification feature, if the current proportion of Mandarin in the audio data is 51% and the proportion of a foreign language is 49%, then the person corresponding to the higher proportion of Mandarin is the target person by default. Alternatively, the settings can be modified according to the actual situation, defaulting to the person corresponding to the lower proportion of dialect as the target person.

[0083] Furthermore, before converting audio data into text data using ARS technology, the audio data is first subjected to noise reduction processing to improve the signal quality of the audio data.

[0084] Step S1002: Split the initial text data into first text data and second text data, and add bag labels to the first text data;

[0085] Specifically, the first text data refers to paragraph text, i.e., segment text. The second text data is the sentence text extracted from the segment text. The segmentation method is as follows: After obtaining the target person's one-way communication record in text form, i.e., the initial text data, through step S1001, the initial text information is sequentially divided into k segments: the text length of the initial text data is c0, where k > c0 / 3, to ensure that the text length of each segment is not too small to have no annotation value. Then, bag labeling is performed on each of the k text segments. Bag labeling refers to the actual emotion labeling of a single sentence. Multiple emotion labels of single sentences constitute a bag label, i.e., a bag label contains multiple emotion labels. Specifically, each single sentence in the segment text is identified, and a bag label is generated based on the identification results.

[0086] In one embodiment, each sentence in the text segment is identified, and bag tags are generated based on the identification results as follows, formula (1):

[0087]

[0088] Where d represents paragraph-level text data, L represents the bag-label value used for subsequent emotion detection model calculations, and x represents sentence-level text data. The idea behind the formula is that when a sentence contains expressions related to "angry," "sad," or "happy," the corresponding emotion feature of that sentence is recorded as "angry," "sad," or "happy," and added to the bag label. It's important to note that a bag label is a set of emotion tags for a paragraph, containing the emotion features of each sentence within the paragraph. A bag label can record multiple emotion features simultaneously. By analyzing and integrating the emotion features in the bag labels, the overall representative emotion of the paragraph can be obtained. Then, by using the paragraph emotion as a reference emotion for sentence labels, the accuracy of document labels can be improved.

[0089] Step S1003: Input the second text data into a preset language error correction model to obtain the third text data;

[0090] Specifically, the second text data can be single-sentence text data converted from audio data after noise reduction. This second text data then undergoes correction processing, specifically by using a pre-built language correction model to correct erroneous or ambiguous words. The pre-built language correction model is the BERT autoencoder language model, employing a 12-layer transformer architecture. It's worth noting that BERT only uses the encoder part of the Transformer; that is, it only needs to acquire feature information to correct erroneous and ambiguous words.

[0091] Exemplarily, when the second text data is collected or converted, errors may occur, resulting in errors or ambiguities in a single sentence of text. The incorrect or ambiguous words are marked with "mask", such as "Trying to compare the West Lake to Xi Shi, light or heavy makeup always suits her well". Based on a deep understanding of each layer in the transformer architecture, it is possible to predict which word was originally in the place marked with "mask". That is, the pre-trained language error correction model of MacBERT will predict the original word according to the context information of the document, extract it from the pre-built corpus, and replace it. In extremely rare cases, when the original word cannot be matched, a random word can be used for replacement from the pre-built corpus.

[0092] Step S1004: Input the third text data into the emotion detection model to detect the emotion parameters corresponding to the third text data.

[0093] Specifically, the third text data refers to the sentence text data processed by the error correction algorithm, that is, high-quality sentence text data. The emotion detection model is a model trained based on BP-MIP (Multi-Instance Problems, multi-instance neural network algorithm). MIP is the fourth learning framework in addition to supervised learning, unsupervised learning, and reinforcement learning. In this application, through BP-MIP, the labeled bag annotations are analyzed, and based on the mapping relationship between the bag annotations and the single sentence data in the third text data corresponding to the bag annotations, the bag annotations of the input single sentence data, that is, the emotion feature parameters, such as {happy, sad, neutral, angry...}, are detected. The main formula of the emotion detection model is as follows:

[0094]

[0095] Among them, Oij represents the model output result, di represents the bag annotation value L, Mj represents the total number of single sentences in the segment text, and j represents any single sentence text in the segment text.

[0096] Through the above solution in this embodiment, specifically by obtaining audio data, converting the audio data into initial text data, splitting the initial text data into first text data and second text data, adding bag annotations to the first text data, inputting the second text data into the pre-trained language error correction model to obtain third text data, and inputting the third text data into the emotion detection model to detect the emotion parameters corresponding to the third text data, a multi-instance learning emotion detection method is provided. By performing noise reduction processing on the audio data, converting it into initial text data after processing, and then optimizing the initial text data through the pre-trained language error correction model, the preprocessing process of the input data is completed. Through the above preprocessing process, the influence of noise, background noise, etc. on the collected original data is greatly reduced, and the accuracy of the final emotion detection is improved.

[0097] Reference Figure 3 , Figure 3 This is a flowchart illustrating another exemplary embodiment of the emotion detection method of this application.

[0098] Before step S1004, where the third text data is input into the emotion detection model, and the emotion corresponding to the third text data is detected based on the mapping relationship between the second text data and the third text data, the method further includes:

[0099] Step A100: Obtain the training set corresponding to the third text data;

[0100] Specifically, the third text data refers to the high-quality text data output by the pre-set language error correction model, which is used as input to the intelligent emotion recognition model. This application focuses on constructing a mapping relationship from paragraphs to individual sentences, that is, each sentence in a paragraph corresponds to an instance of an emotion feature. By learning the emotion features between multiple sentences and their respective paragraphs, the application ultimately achieves the goal of more accurately detecting the emotion features of a single sentence by combining the mapping relationship between the sentence and its paragraph. This is because the emotion representation of a single sentence in a paragraph is usually consistent with the emotion representation of the paragraph.

[0101] Step A200: Based on the training set and the bag labels corresponding to the training set, and combining the mapping relationship between the training set and the bag labels, a pre-set neural network model is trained to obtain an emotion detection model.

[0102] Specifically, an intelligent emotion recognition model based on BP-MIP is built. The idea of ​​the BP-MIP algorithm is to calculate an error by comparing the actual output of the network with the target output, and then feed this error back to the input layer to correct the network's weights and other parameters so that the actual output approaches the target output, and the detection output approaches the bag label.

[0103] A BP-MIP-based intelligent emotion recognition model was constructed. The mapping function f from paragraph text to single-sentence text was defined as the loss function. The second text data was used as input, and the bag-labeled result L was used as the detection value. The intelligent emotion recognition method was trained until it converged. At this point, BP-MIP was a single-sentence level intelligent emotion detection model, and f was defined as shown in formula (2). Detection was performed using single-sentence data from network points, and the result of BP-MIP was the detection result of the intelligent emotion recognition.

[0104] The discrimination module compares the detection values ​​output by the model with the bag labels corresponding to the high-quality text to obtain loss data. Then, based on the loss data, it adjusts the weight parameters in the neural network model to make the output detection values ​​approximate the corresponding bag labels. Formula 2 is the loss function of this application:

[0105]

[0106] Where Oij represents the model output, di represents the bag label value L, Mj represents the total number of sentences in the paragraph, and j represents any sentence in the paragraph.

[0107] This embodiment, through the above scheme, specifically obtains the training set corresponding to the third text data, trains a pre-set neural network model based on the training set and the bag annotations corresponding to the training set, obtains an emotion detection model, and constructs a mapping relationship from paragraph to sentence by defining a loss function from paragraph to sentence, thereby achieving the effect of more accurately detecting the emotion representation of single sentence text through the emotion detection model.

[0108] Reference Figure 4 , Figure 4 This is a flowchart illustrating another exemplary embodiment of the emotion detection method of this application.

[0109] Step A200, the step of training a pre-set neural network model based on the training set and the corresponding bag labels to obtain the emotion detection model includes:

[0110] Step B100: Obtain the training result output by the preset neural network model after the training set corresponding to the third text data is input into the preset neural network model;

[0111] Specifically, the training results of the pre-built neural network based on the third text data are obtained. One sentence text corresponds to one training result, and one segment text can correspond to multiple sentence texts.

[0112] Step B200: Input the training results and the bag labels into a preset loss function to obtain the loss value output by the loss function.

[0113] Specifically, the loss function is as follows:

[0114]

[0115] Where Oij represents the model output, and di represents the bag label value L. The weights in the neural network are updated based on the error data between the model output and the bag labels, as well as the loss value, until the loss value between the training result output by the updated weights and the bag labels is less than a preset threshold. The preset threshold can be adjusted according to actual needs. The smaller the preset threshold, the closer the training result is to the bag labels.

[0116] Step B300: Adjust the preset neural network model based on the loss value until convergence is achieved, thereby obtaining the emotion detection model.

[0117] The accuracy of the emotion detection model in detecting the emotion of the second text data is determined based on the loss value.

[0118] Specifically, the loss value represents the similarity between the training result and the bag label. The smaller the loss value, the more similar the training result is to the bag label, and the higher the accuracy. The neural network is trained until it converges to obtain the emotion detection model.

[0119] This embodiment, through the above scheme, specifically obtains the training result output by the preset neural network model after inputting the training set corresponding to the third text data into the preset neural network model. The training result and the bag annotation are then input into a preset loss function to obtain the loss value output by the loss function. Based on the loss value, the preset neural network model is adjusted until convergence is obtained to obtain the emotion detection model. That is, through the loss function, which is the mapping relationship f from paragraph text to single sentence text, the error between the model detection result and the actual bag annotation result is determined, and the weight parameters of the neural network are adjusted according to the error, ultimately improving the detection accuracy of the emotion detection model.

[0120] Reference Figure 5 , Figure 5 This is a flowchart illustrating another exemplary embodiment of the emotion detection method of this application.

[0121] Step B200, the step of inputting the training results and the bag labels into a preset loss function to obtain the loss value output by the loss function includes:

[0122] Step C100: Obtain all the training results corresponding to all the third text data;

[0123] Specifically, if a third text data contains multiple second text data, and each second text data corresponds to the sentiment detection result of a segment of text, then all sentiment detection results corresponding to that segment of text are obtained.

[0124] Step C200: Input all the training results and bag labels one by one into the preset loss function to obtain a set of loss values;

[0125] Specifically, to prevent overfitting in emotion detection models—that is, to prevent the model's output from being biased towards a particular input data, leading to low model applicability—the common practice is to increase the amount of data and the test sample set. In this application, there are multiple training results for the same third text data corresponding to multiple second text data. Therefore, this application uses the training result that differs most from the bag-of-bag labels from multiple training results to train the neural network model, thereby improving the model training efficiency. By comparing the training results with the same bag labels, a set of loss values ​​is obtained.

[0126] Step C300: Select the maximum loss value from the set of loss values.

[0127] Specifically, refer to the formula:

[0128]

[0129] The max function indicates that, from the training results corresponding to multiple second text data in a third text data set, the training result with the largest loss value compared to the bag label, i.e. the largest error, is selected to train the neural network model, thereby training the model more efficiently.

[0130] In this embodiment, the above-described scheme is implemented by acquiring all the training results corresponding to all the third text data, inputting all the training results and bag annotations one by one into a preset loss function to obtain a set of loss values, and selecting the maximum loss value from the set of loss values, that is, selecting the single sentence text with the greatest emotional difference from multiple single sentence texts corresponding to the paragraph text to train the neural network, thereby improving the reliability of training.

[0131] Reference Figure 6 , Figure 6 This is a flowchart illustrating another exemplary embodiment of the emotion detection method of this application.

[0132] Before step S1003, which involves inputting the second text data into a preset language error correction model to obtain the third text data, the method further includes:

[0133] Step D100: Add noise data to the second text data to obtain the fourth text data;

[0134] Specifically, the fourth text data refers to the initial text data that has undergone word blurring or error replacement processing. To reduce the impact of noise data on the final emotion detection results, i.e., the audio data acquisition is affected by the surrounding environment, and to simulate the actual situation when a single sentence is input into the emotion detection model, it is necessary to add noise to the second text data, and then train the language model based on the noise-added text data.

[0135] Step D200: Split the fourth text data into the fifth text data.

[0136] Specifically, the initial noisy text data is split into sentence-level text data, namely the fifth text data.

[0137] Step D300: Based on the second text data and the noisy text data, train an autoencoder language model to obtain the preset language error correction model.

[0138] Specifically, an autoencoder language model based on Mac-BERT is built. The fourth text data is used as input, and the original second text data is used as labels to train the autoencoder language model until it converges, thus obtaining the language model.

[0139] In this embodiment, the above-described scheme is used to obtain fourth text data by adding noise data to the second text data. The fourth text data is then split into fifth text data. Based on the second text data and the noise data, an autoencoder language model is trained to obtain the preset language error correction model. The trained language model is used to calibrate the initial corpus, thereby reducing the impact of noise data quality and improving the accuracy of emotion detection in bank branches.

[0140] Reference Figure 7 , Figure 7 This is a flowchart illustrating another exemplary embodiment of the emotion detection method of this application.

[0141] Step S1002, the step of splitting the initial text data into third text data and second text data, and adding bag annotations to the third text data, includes:

[0142] Step E100: Obtain keywords from the second text data and compare the keywords with a preset keyword database;

[0143] Specifically, the initial text information is organized in units of "tong" (a type of communication), with each tong corresponding to one customer. Keyword information is extracted from the individual sentences in the text, and the length of a single initial text is c0. The text is then split into segments with at least k segments per tong, where k > c0 / 3, to obtain segment text. The individual sentences in each segment text are then further segmented sequentially to obtain the second text data, i.e., the individual sentence text.

[0144] Step E200: If the comparison result is consistent, then the preset emotion feature is used as the bag label for the third text data corresponding to the second text data.

[0145] Specifically, referring to the description in step S1002, the keyword information in the extracted single sentence text is compared with the words in the corpus. When the comparison result shows that there are related keywords such as "angry", "sad", and "happy" in the single sentence, the corresponding emotional features of the bag annotation are "angry", "sad", and "happy". It should be noted that the bag annotation can record multiple emotional features at the same time.

[0146] The embodiment uses the above scheme to specifically obtain keywords from the second text data, compare the keywords with a preset keyword library, and if the comparison results are consistent, use the preset emotion features as bag labels for the third text data corresponding to the second text data. That is, each sentence is compared and analyzed, and the comprehensive comparison and analysis results are used to generate bag labels for the corresponding paragraph text, thus constructing a mapping relationship from paragraph text to sentence text. Since each sentence is analyzed and compared, the reliability of the final emotion detection is guaranteed.

[0147] Reference Figure 8 , Figure 8 This is the overall flowchart of the emotion detection method in this application.

[0148] First, branch audio data that does not contain sensitive customer information is extracted from the bank's back-end database. ASR (automated speech recognition) technology is used to transcribe the data to obtain initial intelligent detection text data D0, where the number of segments in D0 is n and the length of a single text is c0. The data is then sequentially segmented with at least k segments (k>c0 / 3) to obtain branch intelligent detection segment data D2. The range of the label LF is defined as LF = {"angry", "sad", "happy", "neutral"}. Segment labeling of D2 yields bag labeling results L. The bag labeling rules are shown in formula (1) in step S1002, resulting in the emotional representation of single-sentence texts and segment texts, i.e., bag labeling.

[0149] Then, extract the intelligent network detection segment data D2, i.e., segment text, and split the data according to the single sentence dimension to obtain the intelligent network detection single sentence data D3, i.e., single sentence text. Build an intelligent error correction method m1 based on the Mac-BERT language model, using noisy data as input. Noisy data refers to text data with blurred or incorrectly replaced words. Train the intelligent error correction method until it converges. Transfer m1 to the intelligent emotion detection scenario, using sentence-by-sentence input from D3 to the intelligent error correction method to obtain semantic representations, reducing the impact of noise on the data. Combined with bag-labeled data L, obtain the high-quality intelligent emotion detection dataset D. f That is, the single-sentence text and paragraph annotations based on the intelligent error correction method are used as the training set for the emotion detection model.

[0150] Then, we build an intelligent emotion recognition system based on BP-MIP. Specifically, we define the mapping function f from paragraphs to single sentences as the loss function, and let D... f As input, the bag labeling result L is used as the detection value to train the intelligent emotion recognition method until it converges. At this time, BP-MIP is a single-sentence level intelligent emotion analysis model, and f is defined as shown in formula (2). The detection is performed using network point single-sentence data, and the result of BP-MIP is the detection result of intelligent emotion recognition. The detection result is the emotion label corresponding to the network point single sentence.

[0151] Through the above steps, a mapping relationship from paragraphs to individual sentences is established. This means that the sentiment of individual sentences within a paragraph is used as a reference for the sentiment labeling of that paragraph, reducing the difficulty of relying solely on individual sentences for sentiment labeling and improving the accuracy of the standard. Furthermore, intelligent error correction methods are used to calibrate the corpus, reducing the impact of noise data quality and simulating interference noise in real-world application scenarios, thereby improving the detection accuracy of the sentiment detection model.

[0152] Furthermore, embodiments of this application also propose an emotion detection device, the emotion detection device comprising:

[0153] The conversion module is used to acquire audio data and convert the audio data into initial text data;

[0154] The splitting module is used to split the initial text data into first text data and second text data, and add bag labels to the first text data;

[0155] The input module is used to input the second text data into a preset language error correction model to obtain the third text data;

[0156] The detection module is used to input the third text data into the emotion detection model and detect the emotion parameters corresponding to the third text data.

[0157] Furthermore, this application also proposes a terminal device, which includes a memory, a processor, and an emotion detection program stored in the memory and executable on the processor. When the emotion detection is executed by the processor, it implements the emotion detection steps described above.

[0158] Since this emotion detection program employs all the technical solutions of all the aforementioned embodiments when executed by the processor, it possesses at least all the beneficial effects brought about by all the technical solutions of all the aforementioned embodiments, which will not be elaborated upon here.

[0159] Furthermore, embodiments of this application also propose a computer-readable storage medium storing an emotion detection program, which, when executed by a processor, implements the emotion detection steps described above.

[0160] Since this emotion detection program employs all the technical solutions of all the aforementioned embodiments when executed by the processor, it possesses at least all the beneficial effects brought about by all the technical solutions of all the aforementioned embodiments, which will not be elaborated upon here.

[0161] Compared to existing technologies that typically compare and analyze entire text segments, which is difficult to detect and cannot guarantee accuracy, this application addresses the problem of difficult sentence-level text annotation by implementing comparative analysis at the sentence level. Leveraging the characteristic that the sentiment representation of a paragraph and the sentiment representation of a single sentence in a discourse are usually consistent, a mapping relationship between sentences and paragraphs is constructed, solving the problem of difficult sentence-level text annotation. The language model is trained using noisy data as input until it converges, thus achieving error correction of sentence-level text corpora and reducing the impact of environmental noise on the final detection results. A neural network is trained using sentence-level text as input, and the bag-annotated sentiment is compared with the sentiment of the sentences obtained from the training results to obtain error data. The weight parameters in the neural network are then adjusted based on the error data until convergence, resulting in a sentiment detection model that can detect sentiment simply by inputting a single sentence corpus. This method involves acquiring and associating the sentiment representations of a large number of paragraph texts and sentences within those paragraphs using a model. Ultimately, it constructs a mapping relationship from paragraphs to individual sentences, which greatly improves the reliability of single-sentence sentiment detection. Furthermore, because this method analyzes the text to be detected at the single-sentence level, it also improves the accuracy of sentiment detection.

[0162] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0163] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0164] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, controlled terminal, or network device, etc.) to execute the methods of each embodiment of this application.

[0165] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method of emotion detection, characterized by, The mood detection method comprises the following steps: obtaining audio data and converting the audio data into initial text data; splitting the initial text data into first text data and second text data, and adding bag labels to the first text data, wherein the first text data refers to paragraph text obtained by splitting the initial text data, and the second text data refers to sentence text obtained by splitting the paragraph text; inputting the second text data into a preset language correction model to obtain third text data; inputting the third text data into a mood detection model to detect the mood parameters corresponding to the third text data; wherein, before the step of inputting the third text data into the mood detection model to detect the mood parameters corresponding to the third text data, it further comprises: obtaining a training set corresponding to the third text data; training a preset neural network model based on the training set and the bag labels corresponding to the training set, combining the mapping relationship between the training set and the bag labels, to obtain a mood detection model; wherein, the step of training a preset neural network model based on the training set and the bag labels corresponding to the training set, combining the mapping relationship between the training set and the bag labels, to obtain a mood detection model comprises: obtaining the training results output by the preset neural network model after inputting the training set corresponding to the third text data into the preset neural network model; inputting the training results and the bag labels into a preset loss function to obtain the loss value output by the loss function; adjusting the preset neural network model based on the loss value until convergence to obtain the mood detection model.

2. The emotion detection method of claim 1, wherein, After the step of inputting the training results and the bag labels into a preset loss function to obtain the loss value output by the loss function, it further comprises: determining the accuracy of the mood detection model for the second text data based on the loss value.

3. The emotion detection method of claim 1, wherein, The step of inputting the training results and the bag labels into a preset loss function to obtain the loss value output by the loss function comprises: obtaining all the training results corresponding to all the third text data; inputting all the training results and the bag labels one by one into a preset loss function to obtain a loss value set; selecting the maximum loss value from the loss value set.

4. The emotion detection method of claim 1, wherein, Before the step of inputting the second text data into a preset language correction model to obtain third text data, it further comprises: adding noise data to the second text data to obtain fourth text data; training a self-encoding language model based on the second text data and the fourth text data to obtain the preset language correction model.

5. The emotion detection method of claim 1, wherein, The step of splitting the initial text data into third text data and second text data and adding bag labels to the third text data comprises: obtaining keywords in the second text data and comparing the keywords with a preset keyword library; if the comparison result is consistent, the emotional features corresponding to the keywords are taken as the bag labels of the third text data corresponding to the second text data.

6. An emotion detection apparatus characterized by comprising: The mood detection device comprises: The conversion module is configured to obtain audio data and convert the audio data into initial text data. The splitting module is configured to split the initial text data into first text data and second text data, and add bag labels to the first text data, wherein the first text data refers to paragraph text obtained by splitting the initial text data, and the second text data refers to sentence text obtained by splitting the paragraph text. The input module is configured to input the second text data into a preset language correction model to obtain third text data. The detection module is configured to input the third text data into an emotion detection model to detect an emotion parameter corresponding to the third text data. The emotion detection apparatus further includes a generation module configured to obtain a training set corresponding to the third text data. The generation module is further configured to obtain a training result output by the preset neural network model after the training set corresponding to the third text data is input into the preset neural network model. The generation module is further configured to input the training result and the bag labels into a preset loss function to obtain a loss value output by the loss function. The generation module is further configured to adjust the preset neural network model based on the loss value until the preset neural network model converges, so as to obtain the emotion detection model. The terminal device includes a memory, a processor, and an emotion detection program stored on the memory and executable on the processor, and the emotion detection program, when executed by the processor, implements the steps of the emotion detection method of any one of claims 1-5.

7. A terminal device, characterized by comprising: The computer-readable storage medium stores an emotion detection program, and the emotion detection program, when executed by the processor, implements the steps of the emotion detection method of any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, ​

Images