Intelligent question-answering method, device, equipment, and storage medium based on emotion recognition

Abstract

The present invention relates to artificial intelligence technology and discloses an intelligent question-and-answer method based on emotion recognition, comprising: performing time-series voiceprint processing on user speech to obtain voiceprint features of the user speech, converting the user speech into text to obtain speech text; performing comprehensive emotion detection based on the voiceprint features and the speech text to obtain a first detection result, performing text emotion detection on the speech text to obtain a second detection result; generating an emotion tag based on the first and second detection results; performing semantic recognition on the speech text, using the semantic recognition result to match a question-and-answer library to obtain business speech; using the emotion tag to match a target feedback speech in an emotion feedback speech library to obtain a target feedback speech, and generating a target answer speech based on the business speech and the target feedback speech. The present invention also provides an intelligent question-and-answer device, equipment, and storage medium based on emotion recognition. The present invention can improve the matching and accuracy of intelligent question-and-answer.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an intelligent question-answering method, device, equipment and storage medium based on emotion recognition. Background Art

[0002] Intelligent question-and-answer (Q&A) applications are increasingly widespread. Currently, most intelligent Q&A systems typically generate answers based on user voice, without considering changes in the user's tone of voice or the emotional state of the text. As a result, the generated answers are less relevant and the user experience is poor. Existing techniques for analyzing user responses and generating answers based on sentiment analysis also suffer from issues such as poor relevance and inaccuracy due to uneven quality of voice data and a single-minded approach to emotion detection. Summary of the Invention

[0003] The present invention provides an intelligent question-answering method, apparatus, device and storage medium based on emotion recognition, the main purpose of which is to solve the problems of low matching and accuracy in the intelligent question-answering process.

[0004] To achieve the above objectives, the present invention provides an intelligent question-answering method based on emotion recognition, comprising:

[0005] Acquire user voice, perform time-series voiceprint processing on the user voice to obtain voiceprint features of the user voice, and convert the user voice into text to obtain voice text;

[0006] Performing comprehensive emotion detection on the voiceprint feature and the speech text to obtain a first detection result, and performing text emotion detection on the speech text to obtain a second detection result;

[0007] generating an emotion label according to the first detection result and the second detection result;

[0008] Perform semantic recognition on the voice text, and use the semantic recognition results to match the business words in a preset question-and-answer library;

[0009] The emotional tag is used to match the target feedback speech in a preset emotional feedback speech library, and a target answer speech is generated according to the business speech and the target feedback speech.

[0010] Optionally, performing time-series voiceprint processing on the user voice to obtain a voiceprint feature of the user voice includes:

[0011] Performing intensity detection on the user's voice to obtain voice intensity;

[0012] Deleting a portion of the user's speech whose speech intensity is less than a preset intensity threshold to obtain a plurality of speech segments;

[0013] Performing time sequence detection on each of the plurality of speech segments to obtain a speech moment of each speech segment;

[0014] Voiceprint features are extracted from speech segments at different speech moments to obtain voiceprint features.

[0015] Optionally, extracting voiceprint features from speech segments at different speech moments to obtain voiceprint features includes:

[0016] Performing noise reduction processing on the speech segments at different speech moments to obtain noise-reduced speech segments;

[0017] Performing multi-dimensional convolution processing on the noise reduction speech segment to obtain multi-dimensional features;

[0018] Performing mean pooling processing on the multi-dimensional features to obtain pooled features;

[0019] Performing full-connection processing on the pooled features using a first fully-connected layer to obtain fully-connected speech segment features;

[0020] The fully connected speech segment features are fully connected using the second fully connected layer to obtain the voiceprint features of the input speech segments at the different speech moments.

[0021] Optionally, performing comprehensive emotion detection based on the voiceprint feature and the speech text to obtain a first detection result includes:

[0022] Performing part-of-speech analysis on the speech text to obtain a weighted proportion;

[0023] Obtaining the voiceprint strength of the voiceprint feature, and calculating the average volume based on the voiceprint strength and the corresponding speech moment;

[0024] Counting the number of words in the speech text, and calculating the speech speed according to the number of words and the corresponding speech time;

[0025] An emotion value is calculated using the proportion weight, the average volume, and the speech speed, and the emotion value is used as the first detection result.

[0026] Optionally, performing text emotion detection on the speech text to obtain a second detection result includes:

[0027] Extracting keywords associated with emotions from the speech text;

[0028] Replacing the keywords with emotion words based on a preset emotion dictionary to obtain replacement text;

[0029] The replacement text is input into a pre-trained emotion recognition model to obtain a second detection result.

[0030] Optionally, the method of using the semantic recognition result to match the business speech in a preset question-and-answer library includes:

[0031] Obtaining the text semantics of each existing question in the question-answer database;

[0032] Performing similarity detection based on the semantic recognition result and the text semantics of existing questions in the question-answer database;

[0033] The text semantics with the greatest similarity in the similarity detection results are selected as the target semantics, and the corresponding business words are extracted from the question-answer database according to the target semantics.

[0034] Optionally, generating an emotion label according to the first detection result and the second detection result includes:

[0035] Performing a weighted operation on the first detection result and the second detection result to obtain a target emotion score;

[0036] The target emotion score is matched using a preset emotion label table to obtain an emotion label.

[0037] In order to solve the above problems, the present invention further provides an intelligent question-answering device based on emotion recognition, the device comprising:

[0038] A user voice processing module is used to obtain user voice, perform time-series voiceprint processing on the user voice to obtain the voiceprint features of the user voice, and perform text conversion on the user voice to obtain voice text;

[0039] an emotion detection result generating module, configured to perform comprehensive emotion detection on the voiceprint feature and the speech text to obtain a first detection result, and perform text emotion detection on the speech text to obtain a second detection result;

[0040] An emotion label generation module, configured to generate an emotion label based on the first detection result and the second detection result;

[0041] A business speech acquisition module is used to perform semantic recognition on the speech text and use the semantic recognition results to match the business speech in a preset question and answer library;

[0042] The target answer speech generation module is used to use the emotional tag to match the target feedback speech in the preset emotional feedback speech library, and generate the target answer speech according to the business speech and the target feedback speech.

[0043] In order to solve the above problem, the present invention further provides an electronic device, comprising:

[0044] at least one processor; and,

[0045] a memory communicatively connected to the at least one processor; wherein,

[0046] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the above-mentioned intelligent question-answering method based on emotion recognition.

[0047] In order to solve the above problems, the present invention also provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is executed by a processor in an electronic device to implement the above-mentioned intelligent question-answering method based on emotion recognition.

[0048] The embodiment of the present invention extracts time-series voiceprint features from user voice, which helps reduce the amount of useless information contained in the user voice, reduces the computing resource usage when analyzing the language, and improves the analysis efficiency and accuracy. By generating emotion tags from both voiceprint emotion detection and speech text emotion detection, the multi-angle analysis of emotion is achieved, and the accuracy of user voice emotion detection is improved. By generating answer words based on both emotion tags and speech text, the matching and accuracy of intelligent question and answer are improved. Therefore, the intelligent question and answer method, device, equipment and storage medium based on emotion recognition proposed by the present invention can solve the problem of low matching and accuracy in the process of intelligent question and answer. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A flowchart of an intelligent question-answering method based on emotion recognition provided by one embodiment of the present invention;

[0050] Figure 2 A schematic diagram of a process for obtaining voiceprint features of a user's speech according to an embodiment of the present invention;

[0051] Figure 3 A schematic diagram of a process for obtaining voiceprint features according to an embodiment of the present invention;

[0052] Figure 4 A schematic diagram of a process for performing comprehensive emotion detection based on voiceprint features and speech text according to an embodiment of the present invention;

[0053] Figure 5 A schematic diagram of a process for performing text emotion detection based on speech text according to an embodiment of the present invention;

[0054] Figure 6 A schematic diagram of a process for generating emotion tags according to an embodiment of the present invention;

[0055] Figure 7 A schematic diagram of a process for generating business speech provided by an embodiment of the present invention;

[0056] Figure 8 A functional module diagram of an intelligent question-answering device based on emotion recognition provided by one embodiment of the present invention;

[0057] Figure 9 A schematic diagram of the structure of an electronic device for implementing the intelligent question-answering method based on emotion recognition provided by one embodiment of the present invention.

[0058] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0059] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0060] The embodiment of the present application provides an intelligent question-answering method based on emotion recognition. The execution subject of the intelligent question-answering method based on emotion recognition includes but is not limited to at least one of the electronic devices such as a server, a terminal, etc. that can be configured to execute the method provided by the embodiment of the present application. In other words, the intelligent question-answering method based on emotion recognition can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0061] Reference Figure 1 FIG. 1 is a flow chart of an intelligent question-answering method based on emotion recognition provided by an embodiment of the present invention. In this embodiment, the intelligent question-answering method based on emotion recognition includes the following steps S1-S5:

[0062] S1. Acquire user voice, perform time-series voiceprint processing on the user voice to obtain voiceprint features of the user voice, and perform text conversion on the user voice to obtain voice text.

[0063] In an embodiment of the present invention, the user's voice can be acquired through a sound collection device in a service terminal of the user or customer service. For example, the user's voice can be acquired through a recording application pre-installed in the service terminal, or a computer program with a sound collection function, or a microphone capable of audio collection. By using the service terminal to collect the user's voice, the real-time performance of the acquired user voice can be improved.

[0064] In another embodiment of the present invention, a Python statement with a data capture function can be used to capture pre-stored user voice from a blockchain node for storing user voice. By utilizing the high throughput of blockchain for data, the efficiency of obtaining the user voice can be improved.

[0065] For details, see Figure 2 As shown, performing time-series voiceprint processing on the user's voice to obtain the voiceprint features of the user's voice includes the following steps S21-S24:

[0066] S21, performing intensity detection on the user's voice to obtain voice intensity;

[0067] S22, deleting the portion of the user voice whose voice intensity is less than a preset intensity threshold, to obtain multiple voice segments;

[0068] S23, performing time sequence detection on each of the plurality of speech segments to obtain a speech moment of each speech segment;

[0069] S24. Extract voiceprint features from speech segments at different speech moments to obtain voiceprint features.

[0070] In embodiments of the present invention, a device with a sound detection function, such as a decibel meter, such as a PocketRTA decibel meter or the SIA SmaartLive decibel test tool, can be used to detect the intensity of the user's voice to obtain the voice intensity. Alternatively, a computer program with a sound intensity detection function, such as one written in a computer language such as Python or Java, can be used to detect the intensity of the user's voice to obtain the voice intensity.

[0071] In this embodiment of the present invention, after obtaining the speech intensity, multiple speech segments are obtained by deleting the portion of the user speech with a sound intensity less than a preset intensity threshold, while retaining the portion of the user speech with a sound intensity greater than or equal to the preset intensity threshold. For example, if there is a 30-second user speech, where the speech intensity of seconds 0-10 and seconds 20-25 is less than the preset intensity threshold, the portions 0-10 and 20-25 of the user speech are deleted, resulting in two speech segments: 10-20 seconds and 25-30 seconds.

[0072] In one practical application scenario, when a user is conversing with a customer service robot, for example, there are often pauses and punctuation between different characters or within the same character. For example, when a user asks a question, there is a period of silence before the customer service robot responds. Another example is when a user is explaining something, there may be pauses in the middle of a long sentence. This embodiment of the present invention detects the intensity of the user's speech and removes the non-human voice portions of the speech, which helps reduce computing resource usage during subsequent language analysis and improves analysis efficiency.

[0073] In an embodiment of the present invention, when performing time sequence detection on the multiple speech segments to obtain the speech moment of each speech segment, the following time sequence detection algorithm may be used:

[0074]

[0075] Among them, T i is the speech moment of the i-th speech segment, t1 is the starting moment of the i-th speech segment, t2 is the ending moment of the i-th speech segment, α is a preset constant coefficient, and α is usually 2.

[0076] For example, there is a speech segment A from 10:10:10 to 10:20:30, then the starting time t1 of speech segment A is 10:10:10, and the ending time of speech segment A is 10:20:30. When α is 2, the above timing detection algorithm can be used to obtain the speech time of speech segment A as 10:15:15.

[0077] In the embodiment of the present invention, please refer to Figure 3 As shown, the voiceprint feature extraction is performed on the speech segments at different speech moments to obtain the voiceprint feature, including the following steps S31-S35:

[0078] S31, performing noise reduction processing on the speech segments at different speech moments to obtain noise-reduced speech segments;

[0079] S32, performing multi-dimensional convolution processing on the noise reduction speech segment to obtain multi-dimensional features;

[0080] S33, performing mean pooling processing on the multi-dimensional features to obtain pooled features;

[0081] S34, using the first fully connected layer to perform fully connected processing on the pooled features to obtain fully connected speech segment features;

[0082] S35. Use the second fully connected layer to perform fully connected processing on the fully connected speech segment features to obtain the voiceprint features of the input speech segments at the different speech moments.

[0083] In an embodiment of the present invention, in order to remove noise from the speech segment, a preset noise reduction filter is used to perform noise filtering on the input speech segment to obtain a noise-reduced speech segment, wherein the filter includes but is not limited to a Butterworth filter, a Chebyshev filter, and a Bessel filter.

[0084] In detail, since the speech segment may contain a large amount of noise (for example, background noise, current noise, etc.), directly analyzing the speech segment will take up a lot of computing resources, resulting in low analysis efficiency. The embodiment of the present invention performs noise reduction processing on the speech segments at different speech moments, which can reduce the noise in the speech segment and then highlight the human voice part, which is conducive to improving the efficiency and accuracy of subsequent speech analysis.

[0085] In an embodiment of the present invention, a convolutional neural network with multiple convolutional layers is used to perform multi-dimensional convolution processing on the noise reduction speech segment, that is, a convolutional neural network with convolution kernels of different sizes is used to convolve the noise reduction speech segment. Features of different dimensions of the noise reduction speech segment can be obtained based on convolution kernels of different sizes, thereby realizing multi-dimensional extraction of features of the noise reduction speech segment, thereby improving the accuracy of subsequent speech analysis.

[0086] The embodiment of the present invention performs mean pooling processing on multi-dimensional features, utilizing the characteristic of mean pooling that can retain more features to achieve maximum retention of multi-dimensional features, which is beneficial to improving the accuracy of subsequent speech analysis.

[0087] The embodiment of the present invention can use a convolutional neural network containing two fully connected layers to perform two fully connected processes on the pooled features, which can increase the network complexity and thus improve the accuracy of the obtained voiceprint features.

[0088] In detail, in an embodiment of the present invention, an acoustic model can be used to convert the user's speech into text to obtain speech text. The acoustic model models the pronunciation of each word to establish a database containing multiple words and the standard pronunciation corresponding to each word. The user's voice at each moment in the user's speech is collected to obtain the user's voice at each moment, and then the voice is probabilistically matched with the words in the pre-built database containing multiple words and the standard pronunciation corresponding to each word, thereby realizing speech recognition of the user's speech and obtaining speech text. The acoustic model includes but is not limited to HMM (Hidden Markov Model) and GMM (Gaussian Mixture Model).

[0089] In the embodiment of the present invention, when the user's voice is recognized through the acoustic model, since there is no need to perform feature extraction processing such as convolution and pooling on the language, it can be beneficial to improve the efficiency of voice recognition and acquisition of voice text.

[0090] S2. Perform comprehensive emotion detection on the voiceprint feature and the speech text to obtain a first detection result, and perform text emotion detection on the speech text to obtain a second detection result.

[0091] In an embodiment of the present invention, an intelligent model with emotion recognition capabilities can be used to perform emotion detection on the voiceprint features to obtain an emotion detection result. The intelligent model includes, but is not limited to, a PLSA (Probabilistic Latent Semantic Analysis) model and an LDA (Latent Dirichlet Allocation) model.

[0092] In the embodiment of the present invention, please refer to Figure 4 As shown, performing comprehensive emotion detection based on the voiceprint feature and the speech text to obtain a first detection result includes the following steps S41-S44:

[0093] S41, performing part-of-speech analysis on the speech text to obtain a weighted proportion;

[0094] S42. Obtain the voiceprint strength of the voiceprint feature, and calculate the average volume based on the voiceprint strength and the corresponding speech time;

[0095] S43, counting the number of words in the speech text, and calculating the speech speed according to the number of words in the text and the corresponding speech time;

[0096] S44. Calculate an emotion value using the proportion weight, the average volume, and the speech speed, and use the emotion value as a first detection result.

[0097] Specifically, the calculating the average volume according to the voiceprint strength of the voiceprint feature and the corresponding speech moment includes:

[0098] The average volume is calculated using the following averaging algorithm:

[0099]

[0100] Among them, Voi i is the average volume of the speech segment corresponding to the i-th voiceprint feature, D i is the speech intensity of the speech segment corresponding to the i-th voiceprint feature, s1 is the starting time of the speech segment corresponding to the i-th voiceprint feature, and s2 is the ending time of the speech segment corresponding to the i-th voiceprint feature.

[0101] Furthermore, the calculating of the speech speed according to the number of words in the text and the corresponding speech moments includes:

[0102] The speech speed is calculated using the following rate algorithm:

[0103]

[0104] Among them, V n is the speech speed of the speech segment corresponding to the nth voiceprint feature, N is the number of text words in the speech segment corresponding to the nth voiceprint feature, s1 is the starting time of the speech segment corresponding to the ith voiceprint feature, and s2 is the ending time of the speech segment corresponding to the ith voiceprint feature.

[0105] In the embodiment of the present invention, the calculating of the emotion value using the proportion weight, the average volume, and the speech speed includes:

[0106] The emotion value is calculated using the following weighting algorithm based on the weight, average volume, and speech speed:

[0107] J=Δ*(θ*W i +β*Voi i +γ*V n )

[0108] Wherein, J is the emotion value, W i is the weight of the speech segment corresponding to the i-th voiceprint feature, Voi i is the average volume of the speech segment corresponding to the i-th voiceprint feature, V n is the speech speed of the speech segment corresponding to the nth voiceprint feature, θ, β, γ are preset constant coefficients, and Δ is a preset model parameter.

[0109] In the embodiment of the present invention, please refer to Figure 5 As shown, performing text emotion detection on the speech text to obtain a second detection result includes the following steps S51-S53:

[0110] S51, extracting keywords associated with emotions from the voice text;

[0111] S52, replacing the keywords with emotion words based on a preset emotion dictionary to obtain replacement text;

[0112] S53: Input the replacement text into a pre-trained emotion recognition model to obtain a second detection result.

[0113] In detail, in the embodiment of the present invention, the keywords associated with emotions refer to the words that can reflect emotions from a certain perspective; in the embodiment of the present invention, an emotion-related keyword dictionary library can be established, in which words that can reflect emotions are included, such as "haha", "hum", "uncomfortable" and so on; when extracting keywords from speech text, the speech text can be divided into segments, and the obtained segments are compared with the words included in the keyword dictionary library in turn, so as to determine the keywords associated with emotions in the text to be identified.

[0114] In an embodiment of the present invention, when the word frequency of each word segment can be determined by the word frequency inverse document frequency statistical method, the greater the word frequency, the more likely this word segment is a keyword in the text to be identified; in an embodiment of the present invention, a preset threshold can be set to screen and filter keywords.

[0115] In an embodiment of the present invention, the emotion dictionary library includes emotion words, such as "happy", "sad", "angry", "doubtful" and so on. Based on the emotion words in the emotion dictionary library, each keyword in the voice text can be replaced with the corresponding associated emotion words, and the text obtained after the replacement is completed is the replacement text.

[0116] In an embodiment of the present invention, the emotion recognition model may be a neural network model that is pre-trained using training texts, and the emotion recognition model includes but is not limited to a random forest model.

[0117] S3. Generate an emotion label according to the first detection result and the second detection result.

[0118] In the embodiment of the present invention, please refer to Figure 6 As shown, generating an emotion tag according to the first detection result and the second detection result includes the following steps S61-S62:

[0119] S61: Perform a weight operation on the first detection result and the second detection result to obtain a target emotion score;

[0120] S62: Use a preset emotion label table to match the target emotion score to obtain an emotion label.

[0121] In detail, the importance corresponding to the obtained emotion detection results may be different, so the target emotion score may be determined through weight calculation.

[0122] In an embodiment of the present invention, the correspondence between emotion scores and emotion labels may be stored in an emotion label table. After determining the target emotion score, the emotion label table may be used to match the target emotion score to obtain the corresponding emotion label.

[0123] For example, if the emotion score is between 0-40, the corresponding emotion label may be unhappy; if the emotion score is between 40-70, the corresponding emotion label may be normal; and if the emotion score is between 70-100, the corresponding emotion label may be happy.

[0124] S4. Perform semantic recognition on the voice text, and use the semantic recognition results to match the business words in a preset question and answer library.

[0125] In an embodiment of the present invention, a natural language processing (NLP) model may be used to perform semantic recognition on the speech text, and the semantic recognition result is expressed in the form of a text vector.

[0126] In the embodiment of the present invention, please refer to Figure 7 As shown, the method of using the semantic recognition results to match the business words in the preset question-answer library includes the following steps S71-S73:

[0127] S71. Obtain the text semantics of each existing question in the question-answer database;

[0128] S72: performing similarity detection based on the semantic recognition result and the text semantics of existing questions in the question-answer database;

[0129] S73. Select the text semantics with the greatest similarity in the similarity detection results as the target semantics, and extract the corresponding business words from the question and answer database according to the target semantics.

[0130] In an embodiment of the present invention, a similarity result is obtained by calculating a distance value between the semantic recognition result and a text vector corresponding to the text semantics of an existing question in the question-answer database.

[0131] Specifically, the distance between the semantic recognition result and the text vector corresponding to the text semantics of the existing question in the question-answer database can be calculated using the following formula:

[0132]

[0133] Wherein, D is the distance value, R is the text semantics of the i-th question in the question-answer library, T is the semantic recognition result of the speech text, and θ is a preset coefficient.

[0134] S5. Use the emotion tag to match the target feedback speech in the preset emotion feedback speech library, and generate the target answer speech according to the business speech and the target feedback speech.

[0135] In an embodiment of the present invention, the emotional feedback script library may store adjustment sentences (i.e., feedback scripts) corresponding to different emotional tags. The adjustment sentences may be used to adjust the conversation atmosphere, alleviate conversation emotions, and improve the user experience.

[0136] In an embodiment of the present invention, the business speech and target feedback speech can be adjusted in the form of disassembly, combination, etc. through a preset dialogue template, and then the target answer speech can be generated to answer the questions raised in the user's voice and adjust the user's emotions.

[0137] The embodiment of the present invention extracts time-series voiceprint features from user voice, which helps reduce the amount of useless information contained in the user voice, reduces the computing resource usage when analyzing the language, and improves the analysis efficiency and accuracy. By generating emotion tags from both voiceprint emotion detection and speech text emotion detection, the multi-angle analysis of emotion is achieved, and the accuracy of user voice emotion detection is improved. By generating answer words based on both emotion tags and speech text, the matching and accuracy of intelligent question and answer are improved. Therefore, the intelligent question and answer method based on emotion recognition proposed by the present invention can solve the problem of low matching and accuracy in the intelligent question and answer process.

[0138] like Figure 8 , which is a functional module diagram of an intelligent question-answering device based on emotion recognition provided by an embodiment of the present invention.

[0139] The emotion recognition-based intelligent question-answering device 100 of the present invention can be installed in an electronic device. Depending on the functionality implemented, the emotion recognition-based intelligent question-answering device 100 can include a user voice processing module 101, an emotion detection result generation module 102, an emotion tag generation module 103, a business speech acquisition module 104, and a target answer speech generation module 105. The modules of the present invention, also referred to as units, refer to a series of computer program segments that can be executed by an electronic device processor and can perform fixed functions, and are stored in the memory of the electronic device.

[0140] In this embodiment, the functions of each module / unit are as follows:

[0141] The user voice processing module 101 is used to obtain user voice, perform time-series voiceprint processing on the user voice to obtain the voiceprint features of the user voice, and perform text conversion on the user voice to obtain voice text;

[0142] The emotion detection result generating module 102 is configured to perform comprehensive emotion detection based on the voiceprint features and the speech text to obtain a first detection result, and perform text emotion detection on the speech text to obtain a second detection result;

[0143] The emotion label generation module 103 is used to generate an emotion label according to the first detection result and the second detection result;

[0144] The business speech acquisition module 104 is used to perform semantic recognition on the speech text and use the semantic recognition results to match the business speech in a preset question and answer library;

[0145] The target answer speech generation module 105 is used to use the emotion tag to match the target feedback speech in the preset emotion feedback speech library, and generate the target answer speech according to the business speech and the target feedback speech.

[0146] In detail, each module in the intelligent question-answering device 100 based on emotion recognition in the embodiment of the present invention adopts the same Figures 1 to 7 The same technical means are used as the intelligent question-answering method based on emotion recognition described in , and can produce the same technical effects, so I will not go into details here.

[0147] like Figure 9 , which is a schematic diagram of the structure of an electronic device for implementing an intelligent question-answering method based on emotion recognition provided by an embodiment of the present invention.

[0148] The electronic device 1 may include a processor 10, a memory 11, a communication bus 12, and a communication interface 13. It may also include a computer program stored in the memory 11 and executable on the processor 10, such as an intelligent question-answering program based on emotion recognition.

[0149] In some embodiments, the processor 10 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor 10 is the control core (Control Unit) of the electronic device, connecting the various components of the entire electronic device using various interfaces and lines, and executing the programs or modules stored in the memory 11 (for example, executing an intelligent question-answering program based on emotion recognition, etc.), as well as calling the data stored in the memory 11, to execute various functions of the electronic device and process data.

[0150] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 may also be an external storage device of an electronic device, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Furthermore, the memory 11 may also include both an internal storage unit and an external storage device of the electronic device. The memory 11 can not only be used to store application software and various types of data installed in the electronic device, such as the code of an intelligent question-answering program based on emotion recognition, but can also be used to temporarily store data that has been output or is to be output.

[0151] The communication bus 12 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0152] The communication interface 13 is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), optionally, the user interface may also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.

[0153] Figure 9 Only the electronic device with components is shown, and it can be understood by those skilled in the art that Figure 9The structure shown does not constitute a limitation on the electronic device 1 , and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0154] For example, although not shown, the electronic device may further include a power source (such as a battery) for powering various components. Preferably, the power source may be logically connected to the at least one processor 10 via a power management device, thereby implementing functions such as charge management, discharge management, and power consumption management through the power management device. The power source may further include any components such as one or more DC or AC power sources, a recharging device, a power failure detection circuit, a power converter or inverter, a power status indicator, etc. The electronic device may further include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0155] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0156] The intelligent question-answering program based on emotion recognition stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When running in the processor 10, it can achieve the following:

[0157] Acquire user voice, perform time-series voiceprint processing on the user voice to obtain voiceprint features of the user voice, and convert the user voice into text to obtain voice text;

[0158] Performing comprehensive emotion detection on the voiceprint feature and the speech text to obtain a first detection result, and performing text emotion detection on the speech text to obtain a second detection result;

[0159] generating an emotion label according to the first detection result and the second detection result;

[0160] Perform semantic recognition on the voice text, and use the semantic recognition results to match the business words in a preset question-and-answer library;

[0161] The emotional tag is used to match the target feedback speech in a preset emotional feedback speech library, and a target answer speech is generated according to the business speech and the target feedback speech.

[0162] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to the description of the relevant steps in the corresponding embodiment of the accompanying drawings, which will not be repeated here.

[0163] Furthermore, if the modules / units integrated into the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0164] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a processor of an electronic device, the computer program can implement:

[0165] Acquire user voice, perform time-series voiceprint processing on the user voice to obtain voiceprint features of the user voice, and convert the user voice into text to obtain voice text;

[0166] Performing comprehensive emotion detection on the voiceprint feature and the speech text to obtain a first detection result, and performing text emotion detection on the speech text to obtain a second detection result;

[0167] generating an emotion label according to the first detection result and the second detection result;

[0168] Perform semantic recognition on the voice text, and use the semantic recognition results to match the business words in a preset question-and-answer library;

[0169] The emotional tag is used to match the target feedback speech in a preset emotional feedback speech library, and a target answer speech is generated according to the business speech and the target feedback speech.

[0170] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the module division is merely a logical function division, and other division methods may be used in actual implementation.

[0171] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0172] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.

[0173] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0174] Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0175] Blockchain, as used in this article, refers to a novel application model for computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Blockchain is essentially a decentralized database, a series of data blocks generated using cryptographic methods. Each block contains information about a batch of online transactions, used to verify the validity of this information (to prevent counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, the platform product service layer, and the application service layer.

[0176] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results.

[0177] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. Terms such as "first" and "second" are used to indicate names and do not imply any particular order.

[0178] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. An intelligent question-answering method based on emotion recognition, characterized in that: The method comprises: Acquire user voice, perform time-series voiceprint processing on the user voice to obtain voiceprint features of the user voice, and convert the user voice into text to obtain voice text; Performing comprehensive emotion detection based on the average volume of the voiceprint feature, the weight of the part of speech ratio of the voice text, and the voice speed to obtain a first detection result; Extracting keywords associated with emotions from the speech text, replacing the keywords with emotion words based on a preset emotion dictionary to obtain replacement text, and inputting the replacement text into a pre-trained emotion recognition model to obtain a second detection result; generating an emotion label according to the first detection result and the second detection result; Perform semantic recognition on the voice text, and use the semantic recognition results to match the business words in a preset question-and-answer library; The emotional tag is used to match the target feedback speech in a preset emotional feedback speech library, and a target answer speech is generated according to the business speech and the target feedback speech.

2. The intelligent question-answering method based on emotion recognition according to claim 1, characterized in that: The performing time-series voiceprint processing on the user's voice to obtain the voiceprint features of the user's voice includes: Performing intensity detection on the user's voice to obtain voice intensity; Deleting a portion of the user's speech whose speech intensity is less than a preset intensity threshold to obtain a plurality of speech segments; Performing time sequence detection on each of the plurality of speech segments to obtain a speech moment of each speech segment; Voiceprint features are extracted from speech segments at different speech moments to obtain voiceprint features.

3. The intelligent question-answering method based on emotion recognition according to claim 2, characterized in that: The voiceprint feature extraction is performed on the speech segments at different speech moments to obtain the voiceprint features, including: Performing noise reduction processing on the speech segments at different speech moments to obtain noise-reduced speech segments; Performing multi-dimensional convolution processing on the noise reduction speech segment to obtain multi-dimensional features; Performing mean pooling processing on the multi-dimensional features to obtain pooled features; Performing full-connection processing on the pooled features using a first fully-connected layer to obtain fully-connected speech segment features; The fully connected speech segment features are fully connected using the second fully connected layer to obtain the voiceprint features of the input speech segments at the different speech moments.

4. The intelligent question-answering method based on emotion recognition according to claim 1, wherein: The comprehensive emotion detection is performed based on the average volume of the voiceprint feature, the weight of the part of speech ratio of the voice text, and the voice speed to obtain a first detection result, including: Performing part-of-speech analysis on the speech text to obtain a weighted proportion; Obtaining the voiceprint strength of the voiceprint feature, and calculating the average volume based on the voiceprint strength and the corresponding speech moment; Counting the number of words in the speech text, and calculating the speech speed according to the number of words and the corresponding speech time; An emotion value is calculated using the proportion weight, the average volume, and the speech speed, and the emotion value is used as the first detection result.

5. The intelligent question-answering method based on emotion recognition according to claim 1, wherein: The method of using the semantic recognition results to match the business words in the preset question-answer library includes: Obtaining the text semantics of each existing question in the question-answer database; Performing similarity detection based on the semantic recognition result and the text semantics of existing questions in the question-answer database; The text semantics with the greatest similarity in the similarity detection results are selected as the target semantics, and the corresponding business words are extracted from the question-answer database according to the target semantics.

6. The intelligent question-answering method based on emotion recognition according to any one of claims 1 to 5, characterized in that: Generating an emotion label according to the first detection result and the second detection result includes: Performing a weighted operation on the first detection result and the second detection result to obtain a target emotion score; The target emotion score is matched using a preset emotion label table to obtain an emotion label.

7. An intelligent question-answering device based on emotion recognition, characterized in that: The device comprises: A user voice processing module is used to obtain user voice, perform time-series voiceprint processing on the user voice to obtain the voiceprint features of the user voice, and perform text conversion on the user voice to obtain voice text; The emotion detection result generation module is configured to perform comprehensive emotion detection based on the average volume of the voiceprint feature, the weight of the part of speech ratio of the voice text, and the speech speed to obtain a first detection result; extract keywords associated with emotions in the voice text, replace the keywords with emotion words based on a preset emotion dictionary library to obtain replacement text, and input the replacement text into a pre-trained emotion recognition model to obtain a second detection result; An emotion label generation module, configured to generate an emotion label based on the first detection result and the second detection result; A business speech acquisition module is used to perform semantic recognition on the speech text and use the semantic recognition results to match the business speech in a preset question and answer library; The target answer speech generation module is used to use the emotional tag to match the target feedback speech in the preset emotional feedback speech library, and generate the target answer speech according to the business speech and the target feedback speech.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the intelligent question-answering method based on emotion recognition as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the intelligent question-answering method based on emotion recognition as described in any one of claims 1 to 6 is implemented.

Images