Voice interaction method and device, electronic equipment and computer readable storage medium

Abstract

The embodiment of the invention discloses a voice interaction method and device, electronic equipment and a computer readable storage medium, and the method comprises the steps: responding to the received current voice information, and determining a response text for the current voice information; identifying acoustic feature information corresponding to the current voice information, and identifying emotion type information corresponding to the current voice information; fusing the voice parameters corresponding to the emotion type information with the acoustic feature information to obtain fused feature information; and synthesizing target voice information based on the fused feature information and the response text, and broadcasting target voice corresponding to the target voice information. The emotion and acoustic characteristics of the user are considered in the generated target voice, so that the user experience can be improved when interaction with the user is carried out based on the target voice.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of speech processing, and specifically to a speech interaction method, device, electronic device, and computer-readable storage medium. Background Art

[0002] With the development of intelligent voice interaction, the convenience of life has been greatly improved. Feedback on user input is provided through voice response, and interaction with users is achieved through voice.

[0003] However, most of the current intelligent voice responses are still mechanical responses, lacking the ability to recognize the user's emotional state and provide corresponding emotional feedback, leaving the user experience to be improved. Summary of the invention

[0004] The embodiments of the present application provide a voice interaction method, device, electronic device and computer-readable storage medium, which can consider user emotions to perform voice interaction processing and improve user experience.

[0005] In a first aspect, an embodiment of the present application provides a voice interaction method, the method comprising:

[0006] In response to receiving the current voice message, determining a response text for the current voice message;

[0007] Identify acoustic feature information corresponding to the current voice information, and identify emotion type information corresponding to the current voice information;

[0008] Fusing the speech parameter corresponding to the emotion type information with the acoustic feature information to obtain fused feature information;

[0009] The target voice information is synthesized based on the fused feature information and the response text, and the target voice corresponding to the target voice information is broadcasted.

[0010] In a second aspect, an embodiment of the present application further provides a voice interaction device, the device comprising:

[0011] A response module, configured to determine a response text for the current voice information in response to receiving the current voice information;

[0012] A recognition module, used to recognize acoustic feature information corresponding to the current voice information, and recognize emotion type information corresponding to the current voice information;

[0013] A fusion module, used for fusing the speech parameters corresponding to the emotion type information with the acoustic feature information to obtain fused feature information;

[0014] A synthesis module is used to synthesize target voice information based on the fusion feature information and the response text, and broadcast the target voice corresponding to the target voice information.

[0015] In a third aspect, an embodiment of the present application further provides an electronic device, which includes an air conditioner, the air conditioner including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps in the above-mentioned voice interaction method when executed by the processor.

[0016] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above-mentioned voice interaction method are implemented.

[0017] In a fifth aspect, the embodiments of the present application further provide a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in various optional implementations described in the embodiments of the present application.

[0018] In response to receiving a current voice message, an embodiment of the present application determines a response text for the current voice message, identifies acoustic feature information corresponding to the current voice message, and identifies emotion type information corresponding to the current voice message, fuses voice parameters corresponding to the emotion type information with the acoustic feature information to obtain fused feature information, synthesizes target voice information based on the fused feature information and the response text, and broadcasts a target voice corresponding to the target voice message.

[0019] Among them, by combining the user's emotion type to generate a target voice based on the response text, the user's emotion is taken into account in the generated target voice, and then the user experience can be improved when interacting with the user based on the target voice. Among them, by combining the acoustic features of the current voice information input by the user to generate the target voice, the matching degree of the target voice and the user's current voice information is improved, and the adaptability of the target voice to the user is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1is a scene diagram of the voice interaction method provided in an embodiment of the present application;

[0022] Figure 2 It is a flowchart of the voice interaction method provided in the embodiment of the present application;

[0023] Figure 3 is a structural diagram of a voice interaction device provided in an embodiment of the present application;

[0024] Figure 4 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in this application to clearly and completely describe the technical solutions in this application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0026] The embodiments of the present application provide a voice interaction method, device, electronic device and computer-readable storage medium. Specifically, the embodiments of the present application provide a voice interaction device suitable for electronic devices, which is used to process the voice output by the device during the voice interaction process to obtain a target voice that matches the user's emotional type and the user's acoustic characteristics. The electronic device includes a terminal device or a server, and the terminal device includes but is not limited to air conditioners, televisions, refrigerators, washing machines, stereos and other devices with voice interaction capabilities. For example, a voice module is built into or integrated in the air conditioner to process and feedback the voice input by the user to interact with the user through voice. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN, Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms, etc. The server can be directly or indirectly connected via wired or wireless communication.

[0027] See also Figure 1 , Figure 1 : is a schematic diagram of a scenario in which a terminal device according to an embodiment of the present application executes the voice interaction method, wherein the specific execution process of the terminal device executing the voice interaction method is as follows:

[0028] In response to receiving the current voice information, the terminal device 10 determines a response text for the current voice information, identifies the acoustic feature information corresponding to the current voice information, and identifies the emotion type information corresponding to the current voice information, fuses the voice parameters corresponding to the emotion type information with the acoustic feature information to obtain fused feature information, synthesizes the target voice information based on the fused feature information and the response text, and broadcasts the target voice corresponding to the target voice information.

[0029] In this embodiment of the present application, the current voice information is the recognized current voice information of the user, for example, a wake-up word, question text or operation command input by the user through voice.

[0030] It is understandable that the terminal device of the embodiment of the present application generates a target voice based on the response text by combining the user's emotion type, so that the generated target voice takes the user's emotion into consideration, thereby improving the user experience when interacting with the user based on the target voice. Among them, the target voice is generated by combining the acoustic features of the current voice information input by the user, improving the matching degree of the target voice with the user's current voice information, and improving the adaptability of the target voice to the user.

[0031] It should be noted that the order of description of the following embodiments is not intended to limit the priority order of the embodiments.

[0032] See also Figure 2 , Figure 2 A flowchart of a voice interaction method provided by an embodiment of the present application. Although a logical sequence is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown in the drawings.

[0033] 101. In response to receiving a current voice message, determine a response text for the current voice message.

[0034] It should be noted that the response text is a text for the voice content of the current voice message, and is used to reply to the voice content of the current voice message. For example, the response text is an output text simulating a customer service conversation.

[0035] It is understandable that the response text can be read from the text library, for example, based on the voice content corresponding to the current voice information, the response text corresponding to the voice content is matched from the text library. The text library pre-stores the response text corresponding to each question text, and when matching the response text, the response text is selected based on the matching between the voice content and the question text.

[0036] In some feasible solutions, the response text can also be output through a large model (LLM). For example, the voice content corresponding to the current voice information is input into the large model, and the response text for the voice content is output through the large model.

[0037] Among them, since voice interaction is usually achieved through multiple conversations, the response text can also be determined in combination with the context of the conversation.

[0038] 102. Identify acoustic feature information corresponding to the current voice information, and identify emotion type information corresponding to the current voice information.

[0039] It should be noted that the acoustic feature information includes scoring parameter information for the acoustic features, where the acoustic features include pitch, volume, speaking speed, timbre, etc. The scoring parameter information refers to the score or value of the current voice information in each acoustic feature, for example, the volume is 80 decibels, the speaking speed is 300 words per minute, etc.

[0040] The emotion type information is the user emotion type reflected by the user voice, such as the user emotion type including depression, anxiety, fear, excitement or peace.

[0041] 103. Fuse the speech parameters corresponding to the emotion type information with the acoustic feature information to obtain fused feature information.

[0042] It should be noted that each emotion type corresponds to different voice parameters, and the voice parameters are also scores or values ​​of different emotion types for each acoustic feature.

[0043] Among them, fusion feature information is obtained through fusion, and the fusion feature information is used to comprehensively represent the user's acoustic features and emotion types.

[0044] 104. Synthesize target voice information based on the fused feature information and the response text, and broadcast a target voice corresponding to the target voice information.

[0045] Among them, in general scenarios, speech can be synthesized directly based on the response text. However, since the speech generated only based on the response text lacks consideration of user emotions, the output speech emotion is monotonous and it is difficult to achieve emotional resonance with the user, and the user experience needs to be improved.

[0046] In the embodiment of the present application, by generating the target voice information corresponding to the response text based on the fused feature information, the target voice corresponding to the target voice information comprehensively considers the user's emotion type and the user's acoustic characteristics, thereby improving the matching or adaptability of the target voice with the user's emotions and acoustic characteristics, and improving the user experience.

[0047] Optionally, in some feasible solutions, the emotion type information corresponding to the current voice information can be identified and classified by an emotion classification model, that is, optionally, in some embodiments of the present application, the step of “identifying the emotion type information corresponding to the current voice information” includes:

[0048] Outputting the emotion type information corresponding to the current voice information through the emotion classification model;

[0049] The emotion classification model is obtained after training the original classification model, and the basic network of the original classification includes at least one of a convolutional neural network or a long short-term memory network.

[0050] It should be noted that the emotion classification model is a machine learning model, which is used to classify and identify the emotion type information corresponding to the current voice information to improve the recognition efficiency.

[0051] Among them, convolutional neural networks (CNN) or long short-term memory networks (LSTM) have significant effects in the classification field. Therefore, choosing convolutional neural networks (CNN) or long short-term memory networks (LSTM) as the basic network training to obtain an emotion classification model will help improve the accuracy of emotion classification.

[0052] Optionally, in some scenarios, in order to reduce the complexity of the model structure, the current voice information may be first subjected to feature extraction, and the extracted features may be used as input to the emotion classification model, so that the model directly processes the extracted features without having to perform tasks such as feature extraction, thereby reducing the complexity of the model. That is, optionally, in some embodiments of the present application, the step of "outputting the emotion type information corresponding to the current voice information through the emotion classification model" includes:

[0053] Performing voice feature extraction on the current voice information to obtain first voice feature information;

[0054] The first speech feature information is input into the emotion classification model so as to output the emotion type information corresponding to the current speech information through the emotion classification model.

[0055] It should be noted that the first voice feature information reflects the features of the current voice information, for example, the first voice feature information includes pitch, volume, speaking speed, timbre, etc. In the embodiment of the present application, the voice feature extraction can be implemented by voice processing technology, for example, by using the Mel Frequency Cepstral Coefficient (MFCC) algorithm to extract the features of the current voice information to obtain the first voice feature information.

[0056] After the first voice feature information is input into the emotion classification model, the emotion classification model outputs the confidence of the first voice feature information for each emotion type, and selects the emotion type corresponding to the highest confidence according to the confidence level as the emotion type information corresponding to the current voice information.

[0057] It is understandable that the emotion classification model can be obtained by training sample data. For example, the emotion classification model can be obtained by training the original classification model with a general or public emotional speech data set. That is, optionally, in some embodiments of the present application, before the step of "inputting the first speech feature information into the emotion classification model to output the emotion type information corresponding to the current speech information through the emotion classification model", the method further includes:

[0058] Acquire general sample information, wherein the general sample information includes general sample voice information and a sample emotion label corresponding to the general sample voice information;

[0059] The original classification model is trained using the general sample voice information and the sample emotion labels corresponding to the general sample voice information to obtain an emotion classification model.

[0060] The sample emotion label is also the sample emotion classification result, such as the emotion labels such as depression and excitement corresponding to the general sample voice information.

[0061] Among them, the general sample information is the samples in the public emotional speech data set. The original classification model is trained through the general samples, so that the emotion classification model obtained after training has general emotion classification capabilities and can be applied to general emotion classification tasks.

[0062] Optionally, you can also collect and annotate relevant speech data based on specific application scenarios, and use this data to fine-tune the model. The goal of fine-tuning is to enable the model to better understand and recognize emotional expressions in specific application scenarios.

[0063] Optionally, on the basis of using public or general sample data to train the original classification model, in order to be more suitable for the emotion classification of the target user, personalized voice information corresponding to different emotion types of the target user can also be collected, and the emotion classification model obtained by training based on the general sample is retrained or optimized based on the personalized voice information of the different emotion types to improve the accuracy of the model in classifying the target user's emotions. That is, optionally, in some embodiments of the present application, after the step of "training the original classification model to obtain the emotion classification model through the general sample voice information and the sample emotion label corresponding to the general sample voice information", the method also includes:

[0064] Obtain personalized voice information corresponding to different emotional types of target users;

[0065] Extracting features from the personalized voice information using a feature extraction model to obtain second voice feature information corresponding to the personalized voice information;

[0066] The emotion classification model is optimized by using the second speech feature information of each emotion type of the target user to obtain an optimized emotion classification model;

[0067] The step of inputting the first speech feature information into the emotion classification model to output the emotion type information corresponding to the current speech information through the emotion classification model includes:

[0068] The first speech feature information is input into the optimized emotion classification model, so as to output the emotion type information corresponding to the current speech information through the optimized emotion classification model.

[0069] It can be understood that by optimizing the model in combination with the personalized voice information of the target user in different emotion types, the accuracy of the optimized emotion classification model in analyzing the emotion type task of the target user is improved.

[0070] Optionally, in order to improve the accuracy of extracting the second voice feature information, the personalized voice information may be preprocessed first, and the preprocessed personalized voice information may be input into a feature extraction model to extract the second voice feature information.

[0071] Among them, preprocessing includes noise reduction, normalization, etc. Noise reduction includes identifying noise in personalized voice information and filtering the noise from personalized voice information. Normalization includes amplitude normalization or spectrum normalization, wherein amplitude normalization includes normalizing the amplitude of personalized voice information to a fixed range, such as the interval [-1,1]. The specific calculation steps are: calculating the maximum value (the maximum absolute value of personalized voice information), dividing the value of each sampling point by the maximum value, obtaining the normalized signal, and completing amplitude normalization. For spectrum normalization, it refers to normalizing the spectrum of personalized voice information to eliminate the influence of recording equipment and environment. The calculation steps include: processing personalized voice information through short-time Fourier transform (STFT) to obtain a spectrum graph, normalizing each frequency channel in the spectrum graph to make its mean 0 and standard deviation 1, and inverse STFT processing (converting the normalized spectrum graph back to a time series signal through STFT).

[0072] Optionally, in order to improve the accuracy of extracting the second speech feature information, the long-distance dependency between the features is considered, that is, optionally, in some embodiments of the present application, the feature extraction model is obtained by the following operations, specifically:

[0073] The sample audio information is extracted through a convolutional neural network to generate a preliminary feature representation;

[0074] Context encoding the preliminary feature representations through a temporal feature model to capture long-distance dependencies between the preliminary feature representations;

[0075] Randomly masking some segments in the sample audio information to determine the masked audio segments;

[0076] Based on the long-distance dependency, predict the masked audio segment to obtain a predicted audio segment;

[0077] A basic model is trained based on the masked audio segment and the predicted audio segment to obtain a feature extraction model.

[0078] For example, a convolutional neural network is used to extract features of personalized voice information to obtain a preliminary feature representation, and an attention mechanism model (such as a Transformer model) or a temporal feature model (such as a long short-term memory network LSTM) is used to contextually encode the preliminary feature representation to obtain a long-distance dependency relationship, and then self-supervised learning is used to predict the masked part of the audio signal, and the model is trained based on the predicted audio signal and the masked signal. Among them, the basic network is a network suitable for audio feature extraction, and the basic network includes but is not limited to a convolutional neural network, a recurrent neural network, an autoencoder, or a network based on an attention mechanism.

[0079] Among them, the self-supervised learning in this scheme is specifically: by randomly masking the audio data of a certain period of time in the input audio signal, the model needs to predict the masked part through context information, deepen the context understanding correctly, and correct the wrong one, so as to learn the rich features and structural information of the audio signal.

[0080] Optionally, since both the speech parameters and the acoustic feature information include values ​​or scores of acoustic features, the fusion of the speech parameters and the acoustic feature information can be achieved based on weights, that is, optionally, in some embodiments of the present application, the step of “fusing the speech parameters corresponding to the emotion type information with the acoustic feature information to obtain fused feature information” includes:

[0081] Determining speech parameters corresponding to the emotion type information;

[0082] Determining a first value for each acoustic feature from the speech parameter, and determining a second value for each acoustic feature from the acoustic feature information;

[0083] For each acoustic feature, a fusion value of the first value and the second value of each acoustic feature is calculated according to a preset weight;

[0084] The fusion feature information is generated according to the fusion value corresponding to each acoustic feature.

[0085] It can be understood that the weight-based fusion scheme ensures the accuracy and effectiveness of the fused feature information.

[0086] It is understandable that based on the output target speech, a feedback mechanism can also be configured to receive user feedback, retrieve user satisfaction, and feed it back to the emotion classification model and feature extraction model, and adjust the preset weights to optimize the output target speech.

[0087] In summary, the embodiment of the present application generates a target voice based on the response text by combining the user's emotion type, so that the generated target voice takes the user's emotion into consideration, thereby improving the user experience when interacting with the user based on the target voice. Among them, the target voice is generated by combining the acoustic features of the current voice information input by the user, improving the matching degree between the target voice and the user's current voice information, and improving the adaptability of the target voice to the user.

[0088] The model is optimized by combining the user's personalized voice information of different emotion types to improve the accuracy of the model's emotion classification for target users.

[0089] In order to better implement the voice interaction method of the present application, the present application also provides a voice interaction device based on the above voice interaction method. The meanings of the nouns are the same as those in the above voice interaction method, and the specific implementation details can refer to the description in the method embodiment.

[0090] See also Figure 3 , Figure 3 : is a schematic diagram of the structure of a voice interaction device provided in an embodiment of the present application, wherein the voice interaction device may be specifically as follows:

[0091] A response module 201 is used to determine a response text for the current voice information in response to receiving the current voice information;

[0092] The recognition module 202 is used to recognize the acoustic feature information corresponding to the current voice information, and recognize the emotion type information corresponding to the current voice information;

[0093] A fusion module 203 is used to fuse the speech parameters corresponding to the emotion type information with the acoustic feature information to obtain fused feature information;

[0094] The synthesis module 204 is used to synthesize the target voice information based on the fusion feature information and the response text, and broadcast the target voice corresponding to the target voice information.

[0095] Optionally, in some embodiments of the present application, the identifying the emotion type information corresponding to the current voice information includes:

[0096] Outputting the emotion type information corresponding to the current voice information through the emotion classification model;

[0097] The emotion classification model is obtained after training the original classification model.

[0098] Optionally, in some embodiments of the present application, outputting the emotion type information corresponding to the current voice information through the emotion classification model includes:

[0099] Performing voice feature extraction on the current voice information to obtain first voice feature information;

[0100] The first speech feature information is input into the emotion classification model so as to output the emotion type information corresponding to the current speech information through the emotion classification model.

[0101] Optionally, in some embodiments of the present application, before inputting the first voice feature information into the emotion classification model to output the emotion type information corresponding to the current voice information through the emotion classification model, the method further includes:

[0102] Acquire general sample information, wherein the general sample information includes general sample voice information and a sample emotion label corresponding to the general sample voice information;

[0103] The original classification model is trained using the general sample voice information and the sample emotion labels corresponding to the general sample voice information to obtain an emotion classification model.

[0104] Optionally, in some embodiments of the present application, after the original classification model is trained to obtain the emotion classification model using the general sample voice information and the sample emotion labels corresponding to the general sample voice information, the method further includes:

[0105] Obtain personalized voice information corresponding to different emotional types of target users;

[0106] Extracting features from the personalized voice information using a feature extraction model to obtain second voice feature information corresponding to the personalized voice information;

[0107] The emotion classification model is optimized by using the second speech feature information of each emotion type of the target user to obtain an optimized emotion classification model;

[0108] The step of inputting the first speech feature information into the emotion classification model to output the emotion type information corresponding to the current speech information through the emotion classification model includes:

[0109] The first speech feature information is input into the optimized emotion classification model, so as to output the emotion type information corresponding to the current speech information through the optimized emotion classification model.

[0110] Optionally, in some embodiments of the present application,

[0111] The feature extraction model is obtained through the following operations, specifically:

[0112] The sample audio information is extracted through a convolutional neural network to generate a preliminary feature representation;

[0113] Context encoding the preliminary feature representations through a temporal feature model to capture long-distance dependencies between the preliminary feature representations;

[0114] Randomly masking some segments in the sample audio information to determine the masked audio segments;

[0115] Based on the long-distance dependency, predict the masked audio segment to obtain a predicted audio segment;

[0116] A basic model is trained based on the masked audio segment and the predicted audio segment to obtain a feature extraction model.

[0117] Optionally, in some embodiments of the present application, the fusing the speech parameter corresponding to the emotion type information with the acoustic feature information to obtain the fused feature information includes:

[0118] Determining speech parameters corresponding to the emotion type information;

[0119] Determining a first value for each acoustic feature from the speech parameter, and determining a second value for each acoustic feature from the acoustic feature information;

[0120] For each acoustic feature, a fusion value of the first value and the second value of each acoustic feature is calculated according to a preset weight;

[0121] The fusion feature information is generated according to the fusion value corresponding to each acoustic feature.

[0122] In the embodiment of the present application, the response module 201 first determines the response text for the current voice information in response to receiving the current voice information. Then, the recognition module 202 recognizes the acoustic feature information corresponding to the current voice information, and recognizes the emotion type information corresponding to the current voice information. Subsequently, the fusion module 203 fuses the voice parameters corresponding to the emotion type information with the acoustic feature information to obtain fused feature information. Then, the synthesis module 204 synthesizes the target voice information based on the fused feature information and the response text, and broadcasts the target voice corresponding to the target voice information.

[0123] The embodiment of the present application generates a target voice based on the response text by combining the user's emotion type, so that the generated target voice takes the user's emotion into consideration, thereby improving the user experience when interacting with the user based on the target voice. The target voice is generated by combining the acoustic features of the current voice information input by the user, thereby improving the matching degree between the target voice and the user's current voice information, and improving the adaptability of the target voice to the user.

[0124] In addition, the present application also provides an electronic device, such as Figure 4 As shown, it shows a schematic diagram of the structure of the electronic device involved in this application, specifically:

[0125] The electronic device may include components such as a processor 301 with one or more processing cores, a memory 302 with one or more computer-readable storage media, a power supply 303, and an input unit 304. Those skilled in the art will appreciate that Figure 4 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0126] The processor 301 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 302, and calling data stored in the memory 302, it executes various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 301 may include one or more processing cores; preferably, the processor 301 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 301.

[0127] The memory 302 can be used to store software programs and modules. The processor 301 executes various functional applications and data processing by running the software programs and modules stored in the memory 302. The memory 302 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 302 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 302 may also include a memory controller to provide the processor 301 with access to the memory 302.

[0128] The electronic device also includes a power supply 303 for supplying power to various components. Preferably, the power supply 303 can be logically connected to the processor 301 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 303 can also include any components such as one or more DC or AC power supplies, recharging systems, power supply device debugging circuits, power converters or inverters, and power status indicators.

[0129] The electronic device may further include an input unit 304, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0130] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 301 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 302 according to the following instructions, and the processor 301 will run the application programs stored in the memory 302, thereby implementing the steps in any one of the voice interaction methods provided in the embodiments of the present application.

[0131] In response to receiving a current voice message, an embodiment of the present application determines a response text for the current voice message, identifies acoustic feature information corresponding to the current voice message, and identifies emotion type information corresponding to the current voice message, fuses voice parameters corresponding to the emotion type information with the acoustic feature information to obtain fused feature information, synthesizes target voice information based on the fused feature information and the response text, and broadcasts a target voice corresponding to the target voice message.

[0132] Among them, by combining the user's emotion type to generate a target voice based on the response text, the user's emotion is taken into account in the generated target voice, and then the user experience can be improved when interacting with the user based on the target voice. Among them, by combining the acoustic features of the current voice information input by the user to generate the target voice, the matching degree of the target voice and the user's current voice information is improved, and the adaptability of the target voice to the user is improved.

[0133] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0134] It can be understood that the electronic device of the embodiment of the present application is suitable for an air conditioner, and the above method is executed by the air conditioner to perform voice interaction with the target user to obtain the above effect.

[0135] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0136] To this end, the present application provides a computer-readable storage medium, on which a computer program is stored. The computer program can be loaded by a processor to execute the steps in any voice interaction method provided in the present application.

[0137] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0138] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0139] Since the instructions stored in the computer-readable storage medium can execute the steps in any voice interaction method provided in the present application, the beneficial effects that can be achieved by any voice interaction method provided in the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0140] The above is a detailed introduction to a voice interaction method, device, electronic device and computer-readable storage medium provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A voice interaction method, characterized in that: The method comprises: In response to receiving the current voice message, determining a response text for the current voice message; Identify acoustic feature information corresponding to the current voice information, and identify emotion type information corresponding to the current voice information; Fusing the speech parameter corresponding to the emotion type information with the acoustic feature information to obtain fused feature information; The target voice information is synthesized based on the fused feature information and the response text, and the target voice corresponding to the target voice information is broadcasted.

2. The voice interaction method according to claim 1, characterized in that: The identifying the emotion type information corresponding to the current voice information includes: Outputting the emotion type information corresponding to the current voice information through the emotion classification model; The emotion classification model is obtained after training the original classification model.

3. The voice interaction method according to claim 2, characterized in that: Outputting the emotion type information corresponding to the current voice information through the emotion classification model includes: Performing voice feature extraction on the current voice information to obtain first voice feature information; The first speech feature information is input into the emotion classification model so as to output the emotion type information corresponding to the current speech information through the emotion classification model.

4. The voice interaction method according to claim 1, characterized in that: Before inputting the first voice feature information into the emotion classification model to output the emotion type information corresponding to the current voice information through the emotion classification model, the method further includes: Acquire general sample information, wherein the general sample information includes general sample voice information and a sample emotion label corresponding to the general sample voice information; The original classification model is trained using the general sample voice information and the sample emotion labels corresponding to the general sample voice information to obtain an emotion classification model.

5. The voice interaction method according to claim 4, characterized in that: After the original classification model is trained to obtain the emotion classification model by using the general sample voice information and the sample emotion labels corresponding to the general sample voice information, the method further includes: Obtain personalized voice information corresponding to different emotional types of target users; Extracting features from the personalized voice information using a feature extraction model to obtain second voice feature information corresponding to the personalized voice information; The emotion classification model is optimized by using the second speech feature information of each emotion type of the target user to obtain an optimized emotion classification model; The step of inputting the first speech feature information into the emotion classification model to output the emotion type information corresponding to the current speech information through the emotion classification model includes: The first speech feature information is input into the optimized emotion classification model, so as to output the emotion type information corresponding to the current speech information through the optimized emotion classification model.

6. The voice interaction method according to claim 5, characterized in that: The feature extraction model is obtained by the following operations, specifically: The sample audio information is extracted through a convolutional neural network to generate a preliminary feature representation; Context encoding the preliminary feature representations through a temporal feature model to capture long-distance dependencies between the preliminary feature representations; Randomly masking some segments in the sample audio information to determine the masked audio segments; Based on the long-distance dependency, predict the masked audio segment to obtain a predicted audio segment; A basic model is trained based on the masked audio segment and the predicted audio segment to obtain a feature extraction model.

7. The voice interaction method according to claim 1, characterized in that: The step of fusing the speech parameter corresponding to the emotion type information with the acoustic feature information to obtain fused feature information includes: Determining speech parameters corresponding to the emotion type information; Determining a first value for each acoustic feature from the speech parameter, and determining a second value for each acoustic feature from the acoustic feature information; For each acoustic feature, a fusion value of the first value and the second value of each acoustic feature is calculated according to a preset weight; The fusion feature information is generated according to the fusion value corresponding to each acoustic feature.

8. A voice interaction device, characterized in that: The device comprises: A response module, configured to determine a response text for the current voice information in response to receiving the current voice information; A recognition module, used to recognize acoustic feature information corresponding to the current voice information, and recognize emotion type information corresponding to the current voice information; A fusion module, used for fusing the speech parameters corresponding to the emotion type information with the acoustic feature information to obtain fused feature information; A synthesis module is used to synthesize target voice information based on the fusion feature information and the response text, and broadcast the target voice corresponding to the target voice information.

9. An electronic device, characterized in that: The invention comprises an air conditioner, wherein the air conditioner comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the voice interaction method as claimed in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps in the voice interaction method according to any one of claims 1 to 7 are implemented.

Images