Voice interaction method and device, household appliance and computer readable storage medium

Abstract

The invention relates to the technical field of smart home, and discloses a method for voice interaction, and the method comprises the steps: determining user corpus text information and user behavior state information based on input user voice interaction information; constructing a voice interaction prompt rule based on the user behavior state information; determining a target response text according to the user corpus text information and a voice interaction prompt rule; and performing voice synthesis processing on the target response text to obtain a target response audio, and performing voice interaction. By acquiring the behavior state information of the user during voice interaction and dynamically customizing the voice interaction prompt rule, the voice interaction prompt rule better fitting the user personalization and the current situation can be constructed, so that more natural and personalized voice interaction experience is provided, and the intelligent level of voice interaction is improved. The invention further discloses a device for voice interaction, a household appliance and a computer readable storage medium.

Description

Technical Field

[0001] The present application relates to the field of smart home technology, for example, to a method and device for voice interaction, home appliance, and computer-readable storage medium. Background Art

[0002] At present, with the rapid development of smart homes, smart voice air conditioner products have gradually become an important member of family life. These products usually integrate voice interaction functions, allowing users to control the temperature, wind speed, mode and other parameters of the air conditioner through voice commands.

[0003] Although existing intelligent voice air conditioning systems provide users with convenient operation methods through voice interaction, they have obvious deficiencies in personalized interaction. These systems usually use standardized, preset reply content, but fail to provide customized interactive experience according to the specific situation and needs of users. In the prior art, more reply options are generally added to the preset reply corpus to increase the naturalness and fluency of the interaction.

[0004] In the process of implementing the embodiments of the present disclosure, it is found that there are at least the following problems in the related art:

[0005] The reply method of the preset corpus is relatively fixed in content and highly formatted. This leads to a relatively simple and mechanical user experience, and also weakens the user's satisfaction with the intelligent voice interaction system.

[0006] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present application, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0007] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical components or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0008] The embodiments of the present disclosure provide a method and apparatus for voice interaction, a home appliance, and a computer-readable storage medium to improve the diversity of device response content during voice interaction between a user and the home appliance.

[0009] In some embodiments, the method for voice interaction includes: determining user corpus text information and user behavior status information based on input user voice interaction information; constructing voice interaction prompt rules based on the user behavior status information; determining target response text according to the user corpus text information and voice interaction prompt rules; performing speech synthesis processing on the target response text to obtain target response audio and perform voice interaction.

[0010] Optionally, constructing a voice interaction prompt rule based on user behavior status information includes: determining at least one weight keyword based on the user behavior status information; obtaining a preset prompt word rule; and constructing a voice interaction prompt rule according to the weight keyword and the prompt word rule.

[0011] Optionally, the user behavior status information includes voiceprint recognition information, and / or emotion recognition information, and / or speaking rate recognition information; then determining at least one weight keyword based on the user behavior status information includes: determining the user's age group, and / or user identity, and / or user gender as weight keywords based on the voiceprint recognition information; determining the user's emotional state as a weight keyword based on the emotion recognition information; and determining the speaking rate of the user's conversation as a weight keyword based on the speaking rate recognition information.

[0012] Optionally, the voice interaction prompt rules are constructed according to weight keywords and prompt prompt rules, including: determining one or more of the perspective rules, sentence rules, emotional rules, and functional rules of voice interaction according to the weight keywords; constructing the voice interaction prompt rules based on the determined voice interaction rules and prompt prompt rules; the prompt prompt rules include: a first reply message, used to represent the execution result feedback text corresponding to the user's operation intention; or, the first reply message, used to represent the execution result feedback text corresponding to the user's operation intention; and, a second reply message, used to represent the guiding extended interaction text.

[0013] Optionally, determining the target response text based on the user corpus text information and the voice interaction prompt rules includes: obtaining a customized pre-trained language model for the home appliance field; inputting the user corpus text information and the voice interaction prompt rules into the customized pre-trained language model for the home appliance field; outputting the target response text; the target response text is a personalized response text generated based on the user voice interaction information.

[0014] Optionally, the speech synthesis processing of the target response text to obtain the target response audio includes: speech synthesis of the target text; sound effect processing of the synthesized speech according to the response perspective of the target text; and tail sound processing and speech speed processing of the speech after the sound effect processing to obtain the target response audio.

[0015] Optionally, the synthesized speech is processed for sound effects according to the reply perspective of the target text, including: determining the frequency points to be adjusted in the synthesized speech according to the reply perspective; and adjusting the frequencies of the frequency points to be adjusted to optimize the sound quality and expressiveness of the sound.

[0016] In some embodiments, the apparatus for voice interaction includes a processor and a memory storing program instructions, and the processor is configured to execute the above-mentioned method for voice interaction when running the program instructions.

[0017] In some embodiments, the household appliance includes: a device body; and the above-mentioned device for voice interaction, which is installed on the device body.

[0018] In some embodiments, the computer-readable storage medium stores program instructions, and when the program instructions are executed, the computer is used to execute the above-mentioned method for voice interaction.

[0019] The method and device for voice interaction, home appliance, and computer-readable storage medium provided by the embodiments of the present disclosure can achieve the following technical effects:

[0020] By obtaining the user's behavioral status information during voice interaction and dynamically customizing the voice interaction prompt rules, we can build voice interaction prompt rules that are more in line with the user's personality and current situation, thereby providing a more natural and personalized voice interaction experience; then, based on the corpus text information of the voice interaction and the voice interaction prompt rules, we can generate the target response text for the user's characteristics, achieve personalized replies, and more accurately determine the response text, reduce misunderstandings and irrelevant answers, and improve the accuracy and efficiency of voice interaction; then, by using voice synthesis processing technology, the target response audio of the interaction can better simulate the expression of natural language, and the voice interaction is more fluent and natural. In this way, we can adapt to the behavioral patterns and voice characteristics of different users during voice interaction and improve the intelligence level of voice interaction.

[0021] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] One or more embodiments are exemplarily described by corresponding drawings, which do not limit the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements, and the drawings do not constitute a scale limitation, and wherein:

[0023] Figure 1 is an environmental schematic diagram of an implementation environment of an embodiment of the present disclosure;

[0024] Figure 2 is a schematic diagram of a voice interaction method provided by an embodiment of the present disclosure;

[0025] Figure 3 is a flow chart of another method for voice interaction provided by an embodiment of the present disclosure;

[0026] Figure 4 is a flow chart of another method for voice interaction provided by an embodiment of the present disclosure;

[0027] Figure 5 is a schematic diagram of a device for voice interaction provided by an embodiment of the present disclosure;

[0028] Figure 6 It is a schematic diagram of another device for voice interaction provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] In order to be able to understand the features and technical contents of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0030] The terms "first", "second", etc. in the specification and claims of the embodiments of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments of the embodiments of the present disclosure described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.

[0031] Unless otherwise stated, the term "plurality" means two or more.

[0032] In the embodiment of the present disclosure, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B indicates: A or B.

[0033] The term "and / or" is a description of the association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B.

[0034] The term "correspondence" may refer to an association relationship or a binding relationship. The correspondence between A and B means that there is an association relationship or a binding relationship between A and B.

[0035] In the embodiments of the present disclosure, smart home appliances refer to home appliances that are formed by introducing microprocessors, sensor technology, and network communication technology into home appliances. They have the characteristics of intelligent control, intelligent perception, and intelligent application. The operation process of smart home appliances often relies on the application and processing of modern technologies such as the Internet of Things, the Internet, and electronic chips. For example, smart home appliances can be connected to electronic devices to enable users to remotely control and manage the smart home appliances.

[0036] In the disclosed embodiments, the terminal device refers to an electronic device with a wireless connection function. The terminal device can communicate with the above-mentioned smart home appliances by connecting to the Internet, or can directly communicate with the above-mentioned smart home appliances through Bluetooth, Wi-Fi, etc. In some embodiments, the terminal device is, for example, a mobile device, a computer, or a vehicle-mounted device built into a hover car, or any combination thereof. Mobile devices may include, for example, mobile phones, smart home devices, wearable devices, smart mobile devices, virtual reality devices, etc., or any combination thereof, wherein wearable devices include, for example: smart watches, smart bracelets, pedometers, etc.

[0037] Figure 1 Schematic diagram of the implementation environment of the embodiment of the present disclosure. Figure 1 As shown, the implementation environment may include a home appliance 100 , a router 200 and a server 300 .

[0038] The home appliance 100 can establish a connection with the Internet using a Wi-Fi (Wireless Fidelity) module. The home appliance 100 includes smart home appliances, such as smart speakers, smart refrigerators, smart TVs, etc. These devices usually have built-in or external microphones for capturing user voice inputs, and also have built-in or external speakers for outputting synthesized voice responses. It can also include mobile devices, such as smart phones, tablet computers, etc. These devices implement voice interaction functions through built-in microphones and speakers. It can also be a computer device with a microphone and a speaker.

[0039] The router 200 is used to connect all the devices in the home and transmit the voice data of these devices to the Internet or a local server.

[0040] The server 300 may be a server or a server cluster composed of several servers, or may be a cloud computing service center, which is not limited in the embodiments of the present disclosure. It is used to process user voice interaction information, including complex computing tasks such as voice recognition, natural language processing, and voice synthesis. Cloud servers usually have powerful processing capabilities and a large amount of storage space.

[0041] It should be understood that Figure 1The number of home appliances, routers and servers in the figure is only for illustration. Any number of home appliances, routers and servers may be provided according to actual needs. For example, one mobile terminal may correspond to multiple home appliances.

[0042] In this way, when users use home appliances such as smart speakers at home, they wake up the device through voice and issue commands or ask questions. After the smart device captures the user's voice, it transmits the voice data to the server through the home router. After the server receives the voice data, it first performs voice recognition, converts the voice into text information, and analyzes the user's behavior status information. Based on the user's behavior status information, the server constructs voice interaction prompt rules, and determines the target response text in combination with the user's corpus text information. After determining the response text, the server uses speech synthesis technology to convert the text into speech and generates the target response audio. Finally, the server transmits the target response audio back to the user's smart device through the network, and plays it to the user through the device's speaker, completing the voice interaction process.

[0043] Figure 2 is a schematic diagram of a voice interaction method provided by an embodiment of the present disclosure. Figure 1 in the environment shown.

[0044] Combination Figure 2 As shown, the method includes:

[0045] Step S201: The server determines user corpus text information and user behavior status information based on input user voice interaction information.

[0046] Step S202: The server constructs voice interaction prompt rules based on the user behavior status information.

[0047] Step S203: The server determines the target response text according to the user corpus text information and the voice interaction prompt rules.

[0048] Step S204: the server performs speech synthesis processing on the target response text, obtains the target response audio, and performs voice interaction.

[0049] In this way, by obtaining the behavioral state information of the user during voice interaction and dynamically customizing the voice interaction prompt rules, it is possible to construct voice interaction prompt rules that are more in line with the user's personalization and current situation, thereby providing a more natural and personalized voice interaction experience; then, based on the corpus text information of the voice interaction and the voice interaction prompt rules, a target response text tailored to the user's characteristics is generated to achieve personalized replies, while more accurately determining the response text, reducing misunderstandings and irrelevant answers, and improving the accuracy and efficiency of voice interaction; then, using speech synthesis processing technology, the interactive target response audio can better simulate the expression of natural language, making the voice interaction more fluent and natural. It can adapt to the behavioral patterns and voice characteristics of different users during voice interaction, and improve the intelligence level of voice interaction.

[0050] Among them, user voice interaction information refers to the voice data transmitted from the user to the server through the smart device; user corpus text information refers to the text data converted from the user voice interaction, which is used to represent the specific content or operation intention of the user input through voice; user behavior state information refers to the background information and behavior patterns related to user interaction, which helps to better understand the user's intentions and needs. For example, the user voice interaction information (i.e., the user voice command) is "help me turn on the lights in the living room", and the corresponding corpus text information is "turn on the lights in the living room". The user's behavior state information includes the user's age corresponding to the voice interaction information, the user's emotions when issuing the command, the speed at which the user issues the command, and other information.

[0051] To obtain user corpus text information, the server can use automatic speech recognition technology to convert the acquired user voice interaction information into text information, and in the speech recognition process, use pre-trained acoustic models and language models to parse the phoneme sequence of the speech and map it to the corresponding text. Through natural language understanding technology, the text content is analyzed to extract key information such as commands, queries, intentions, etc., and entities (such as indoor space, target equipment, etc.), relationships and user intentions in the text are identified to understand the user's corpus text information. Through accurate speech recognition and natural language understanding, the server can more accurately capture the user's intentions and needs, thereby providing more appropriate responses.

[0052] Optionally, constructing a voice interaction prompt rule based on the user behavior status information includes: determining at least one weight keyword based on the user behavior status information; obtaining a preset prompt word rule; and constructing a voice interaction prompt rule based on the weight keyword and the prompt word rule.

[0053] Weighted keywords refer to information words related to user behavior, and weights are assigned based on the strength of relevance to user behavior. Pre-set prompt rules refer to pre-defined prompt rule libraries, which are pre-designed based on different interaction scenarios and user needs. Here, weighted keywords and matched prompt rules are used to construct specific voice interaction prompt rules applicable to the current user. According to the importance of weighted keywords, the parameters in the prompt rules, such as perspective, sentence structure, speech speed, question-and-answer logic, etc., are adjusted to optimize the interaction experience.

[0054] Optionally, the user behavior status information includes voiceprint recognition information, and / or emotion recognition information, and / or speech rate recognition information; then based on the user behavior status information, at least one weighted keyword is determined, including:

[0055] Determine the user's age group, and / or user identity, and / or user gender as weighted keywords based on the voiceprint recognition information;

[0056] Based on the emotion recognition information, the user's emotional state is determined as a weighted keyword;

[0057] According to the speech rate recognition information, the speech rate of the user's conversation is determined as a weighted keyword.

[0058] Here, based on the voiceprint features, the user's age group, identity, gender and other attributes are determined through the pre-trained voiceprint recognition model, and these attributes are used as weight keywords for customized interaction. The weight order is from large to small: user age group, user identity, and user gender. Among them, when the user age group is used as the weight keyword, the weight keyword can include old, middle-aged, and young; when the user identity is used as the weight keyword, the weight keyword can include the user's name or nickname, such as Mr. X; when the user's gender is used as the weight keyword, the weight keyword can include male and female.

[0059] By using the emotion recognition model, the user's emotions can be classified into emotional states such as happiness, anger, sadness, etc. through the emotional features in the user's voice, such as pitch changes, energy level, voice tremor, etc. These states are used as weighted keywords, and the weighted keywords can include happy, calm, angry, and sad.

[0060] By measuring the speech speed of the user's voice, the user's speech speed is divided into fast, normal, slow and other levels according to the speech speed detection result, which is used as a weight keyword. In this embodiment, the user's speech speed is defined as 3 levels: fast speech speed (less than or equal to 150 mm / word), medium speech speed (less than 300 mm / word, greater than 150 mm / word), slow speech speed (greater than or equal to 300 mm / word).

[0061] Optionally, according to the weighted keywords and prompt rules, a voice interaction prompt rule is constructed, including:

[0062] Determine one or more of the perspective rules, sentence rules, emotion rules, and function rules of voice interaction according to the weighted keywords;

[0063] Based on the determined voice interaction rules and prompt rules, construct voice interaction prompt rules;

[0064] Prompt rules include:

[0065] The first reply information is used to indicate the execution result feedback text corresponding to the user's operation intention; or

[0066] The first reply information is used to represent the execution result feedback text corresponding to the user's operation intention; and the second reply information is used to represent the guiding extended interaction text.

[0067] Perspective rules refer to the response context perspective of voice interaction determined based on weighted keywords. For example, a more respectful tone is used for older users, and a more friendly tone is used for child users.

[0068] Sentence rules refer to selecting appropriate sentence structures, such as interrogative sentences, declarative sentences, or imperative sentences, based on weighted keywords, or specifying the sentence length of the reply text, such as short and medium sentences, long sentences, etc., to adapt to different interaction scenarios and user needs.

[0069] Emotional rules refer to the language characteristics of the reply text, such as humor, friendliness, patience, seriousness, etc., which are used to adjust the tone and emotion of the voice reply.

[0070] Functional rules refer to the reply content that contains specific skills or functions, such as "check the weather", "play music", "reminders", etc., which are used to guide further voice interaction functions. Or, recommend functions or services that users may be interested in.

[0071] The voice interaction prompt rules are formed by combining the determined voice interaction rules with the preset prompt rules to form a complete set of voice interaction prompt rules. In this way, the first reply information and the second reply information are customized in combination with the prompt rules to achieve feedback and guidance that matches the user's operation intention.

[0072] Here, the first reply information is a mandatory part, which is used to reply to the user's feedback text of the execution result of the explicit intention; the second reply information is an optional part, which is used to output additional guided extended interactive content to the user. In the case where the reply information includes the first reply information and the second reply information, the reply processing method is called a two-stage reply technique.

[0073] For example, for a middle-aged female user, when the voice interaction information is "Set the air conditioner temperature to 26 degrees", after being processed by the cloud server, the voice air conditioner executes the target response audio "The temperature is now set to 26 degrees. I am here to give you a home with suitable temperature."

[0074] In this way, in the two-segment reply mode, based on the user status feature information obtained in real time for different family users, different prompt rules can be customized to guide how to generate personalized texts for different paragraphs.

[0075] For example, the prompt prompt rules include: prompt ("Do A processing on the reply text, use B perspective as the reply context, the front-end reply needs to use C, and can reflect E that controls D, and the back-end reply needs to include replies with skills such as F"). Among them, the letters A to F refer to the rules of voice interaction determined by weighted keywords. For example, A is a two-part reply script, B is a perspective rule: male elderly; C is a sentence rule: short and medium sentences; D is an emotional rule: accuracy; E is a sentence rule: common language; F is a function rule: common function guidance.

[0076] Then, based on the determined voice interaction rules and prompt rules, the voice interaction prompt rules are constructed as follows: the reply text is processed into a two-part reply script, with the perspective of an elderly male as the reply context. The front-end reply needs to use short and medium sentences and common phrases that can reflect control accuracy. The back-end reply needs to include common functions to guide replies of such skills.

[0077] Optionally, when it is identified that the user's age group is young, the server determines a voice interaction rule according to the user's age group, including:

[0078] The sentence structure of the first reply message is very short, result-oriented (mainly refers to the feedback on the success or failure of a function execution), common language, and modal particles; the sentence structure of the second reply message is short sentences and interrogative sentences; the functional rule is to answer the new skill domain text; the perspective rule is the child's voice. For example, the voice interaction message sent by the child is "turn on the air conditioner", and the target response text of the air conditioner execution is: "Turn it on! Do you want to listen to the nursery rhymes I can play?"

[0079] Optionally, when it is identified that the user's age group is medium, the server determines a voice interaction rule according to the user's age group, including:

[0080] The sentence structure rules of the first reply information are medium-long sentences, accuracy (mainly referring to the specific action description of a function), and professional vocabulary; the sentence structure rules of the second reply information are long sentences; the functional rules are function guidance, common skill guidance, reminder guidance, and the perspective rule is adult voice. For example, the voice interaction information sent by the middle-aged user is: "Turn on the air conditioner", and the target response text of the air conditioner is: "The bedroom air conditioner has been turned on, and the comfortable temperature of 23 degrees has been set for you."

[0081] Optionally, when it is identified that the user is old, the server determines a voice interaction rule according to the user's age group, including:

[0082] The sentence structure rules of the first reply information are short and medium sentences, accuracy, and common words; the sentence structure rules of the second reply information are long sentences; the function rules are common function guidance, and the perspective rules are adult voices. For example, the voice interaction information sent by the elderly user is: "Turn on the air conditioner", and the air conditioner execution target response text is: "The air conditioner is turned on, and you can also say auto wind to me to adjust the wind speed."

[0083] Optionally, when the user identity is identified, the server determines a voice interaction rule according to the user identity, including:

[0084] The sentence pattern of the first reply message is to give the user's name or nickname after the answer. For example, the voice interaction message sent by Mr. X is: "Air conditioning heating", and the air conditioning execution target response text is: "It has been adjusted to the heating mode for you, Mr. X. Proper ventilation is required for long-term operation."

[0085] Optionally, when the gender of the user is identified, the server determines a voice interaction rule according to the gender of the user, including:

[0086] When the gender information is identified as male, the function rules of the second reply information are new skill recommendation, equipment maintenance knowledge introduction, and information recommendation;

[0087] When the gender information is identified as female, the functional rules of the second reply information are recommendation of usage environment information, introduction of equipment cleaning knowledge, and recommendation of clothing information.

[0088] Optionally, when it is recognized that the user's emotional state is happy, the server determines a voice interaction rule according to the user's emotional state, including:

[0089] The sentence structure of the first reply message is a mid-length sentence with extensibility (mainly referring to the description of other states of the device after a function is executed) and a text answer using professional terms. The sentence structure of the second reply message is a long sentence with functional rules such as news recommendation and product recommendation. For example, when the user's voice is recognized to send a voice interaction message of "turn on the air conditioner", the emotional state is happy, and the air conditioner execution target response text is: "The air conditioner in the living room is turned on, and the current temperature setting is 24 degrees. Do you want me to report the result of last night's Premier League match to you?"

[0090] Optionally, when it is recognized that the user's emotional state is calm, the server determines a voice interaction rule according to the user's emotional state, including:

[0091] The sentence structure of the first reply information is short sentences, and the text answers with accuracy; the sentence structure of the second reply information is medium-short sentences, and the function rule is product additional function recommendation. For example, when the user's voice is recognized to send the voice interaction information of "turn on the air conditioner", the emotional state is calm, and the air conditioner execution target response text is: "The air conditioner in the living room is turned on, and you can also say to me 'turn on the fresh air' to make the room air fresh."

[0092] Optionally, when it is recognized that the user's emotional state is angry, the server determines a voice interaction rule according to the user's emotional state, including:

[0093] The sentence structure of the first reply message is very short, and the result text answer is given, and no second reply message prompt is given. For example, when the user's voice is recognized to send the voice interaction information "turn on the air conditioner" and the emotional state is angry, the air conditioner execution target response text is: "turned on".

[0094] The descriptive words in the above rule definitions are some key weighted words for the application scenarios of the embodiments of the present application. In the above embodiments, only prompt rules are made for the above key words based on the currently known input information. However, based on this design idea, it is entirely possible to add more prompt rules for other key words according to the actual input of more user feature information, so as to enrich the diversity of automatically generated text, adapt to the behavioral patterns and voice characteristics of different users during voice interaction, and improve the intelligence level of voice interaction.

[0095] Figure 3 is a flowchart of another method for voice interaction provided by an embodiment of the present disclosure, which is applied to Figure 1 in the environment shown.

[0096] Combination Figure 3 As shown, the method includes:

[0097] Step S301: The server determines user corpus text information and user behavior status information based on input user voice interaction information.

[0098] Step S302: The server constructs voice interaction prompt rules based on the user behavior status information.

[0099] Step S303, the server obtains a customized pre-trained language model in the field of home appliances;

[0100] Step S304: the server inputs the user corpus text information and voice interaction prompt rules into a customized pre-trained language model for the home appliance field.

[0101] Step S305, the server obtains the output target response text; the target response text is a personalized response text generated based on the user voice interaction information.

[0102] Step S306: The server performs speech synthesis processing on the target response text to obtain the target response audio and perform voice interaction.

[0103] Here, the customized pre-trained language model for home appliances refers to an optimized model obtained by special training using the open source GPT-2 basic model based on the user-home appliance interaction data accumulated under the current home appliance NLP model. This model can generate answers to questions raised by users about home appliance function control and operation and maintenance.

[0104] Optionally, obtaining a customized pre-trained language model for the home appliance field includes:

[0105] Select the pre-trained GPT-2 model as the base model;

[0106] Use text data from the home appliance domain to adjust the base model to fit the specific home appliance domain;

[0107] According to the characteristics of the home appliance field, design the network structure or loss function of the adjusted basic model;

[0108] The training parameters are adjusted during the training process, and the evaluated model is used as a customized pre-trained language model for the home appliance field after evaluation.

[0109] In this way, the output target response text can not only answer users' questions about air-conditioning function control, such as temperature adjustment, wind speed setting, and mode switching; it can also provide professional suggestions for the operation and maintenance of home appliances, such as troubleshooting, energy-saving suggestions, etc. At the same time, it has emotional color and external knowledge information to achieve the purpose of personalized response.

[0110] Figure 4 is a flowchart of another method for voice interaction provided by an embodiment of the present disclosure, which is applied to Figure 1 in the environment shown.

[0111] Combination Figure 4 As shown, the method includes:

[0112] Step S401: The server determines user corpus text information and user behavior status information based on input user voice interaction information.

[0113] Step S402: The server constructs voice interaction prompt rules based on the user behavior status information.

[0114] Step S403: The server determines the target response text according to the user corpus text information and the voice interaction prompt rules.

[0115] Step S404: the server performs speech synthesis on the target text.

[0116] Step S405: the server performs sound effect processing on the synthesized speech according to the response perspective of the target text.

[0117] Step S406: the server performs tail sound processing and speech speed processing on the voice after the sound effect processing to obtain the target response audio.

[0118] Here, the reply perspective refers to the reply context perspective of the voice interaction. In this embodiment, male and female are used as different reply perspectives. In other embodiments of this solution, reply perspectives differentiated by age group, reply perspectives differentiated by emotion, etc. may also be included.

[0119] Specifically, according to the response perspective of the target text, the synthesized speech is processed with sound effects, including:

[0120] Determine the frequency points to be adjusted in the synthesized speech according to the answer perspective;

[0121] Adjust the frequency of the frequency point to be adjusted to optimize the sound quality and expressiveness of the sound.

[0122] Here, the voice audio signal is frequency-specifically adjusted, including enhancement and attenuation of the frequency point to be adjusted, to improve the sound quality and expressiveness of the sound. In this embodiment, the frequency point to be adjusted is a frequency that is relatively sensitive to male and female voices.

[0123] For example, according to the answering perspective of the target text, the synthesized speech is processed with sound effects, including:

[0124] When the answering perspective is male voice, determining the first gain frequency point and the second attenuation frequency point;

[0125] When the answering perspective is a female voice, a first filtering frequency point, a second gain frequency point and a third attenuation frequency point are determined;

[0126] The signals below the first filtering frequency are filtered out; the signal strengths of the first gain frequency and the second gain frequency are enhanced; and the signal strengths of the second attenuation frequency and the third attenuation frequency are weakened.

[0127] Specifically, when the reply perspective is male voice, the first gain frequency is 60Hz, which increases 2dB to 60Hz; the second attenuation frequency includes 100Hz, 250Hz, 500Hz, 900Hz and 2000Hz, with 3dB attenuation at 100Hz, 9dB attenuation at 250Hz, 2dB attenuation at 500Hz, 12dB attenuation at 900Hz and 2dB attenuation at 2000Hz.

[0128] When the reply perspective is female voice, the first filtering frequency point is 160Hz, and the frequencies below 160Hz are filtered out; the second gain frequency points include 320Hz and 345Hz, with the 320Hz frequency point increased by 5dB and the 345Hz frequency point increased by 2dB; the third attenuation frequency points include 500Hz, 900Hz and 2000Hz, with the 500Hz frequency point attenuated by 2dB, the 900Hz frequency point attenuated by 12dB, and the 2000Hz frequency point attenuated by 3dB.

[0129] Optionally, the voice after sound effect processing is subjected to tail sound processing and speech speed processing, including: delaying the end of the voice audio and adding a tail sound effect with geometric attenuation of a set duration; processing the speech speed of the voice audio to make it correspond to the speech speed of the user's voice interaction information.

[0130] In this way, the server can generate highly natural and personalized target response audio, which not only enhances the user's interactive experience, but also improves the efficiency and accuracy of information transmission. By performing specific sound effects processing on the tail of the voice, such as geometric attenuation, the end of the voice is made more natural, avoiding the sudden truncation common in synthetic speech, and improving the overall fluency and listening comfort of the voice. The server adjusts the playback speed of the voice according to the response perspective of the target text, which is divided into three gears: fast, medium, and slow, so that the speed of the synthetic voice matches the speed of the real speaker, or is customized according to the user's preferences, thereby improving the comprehensibility and communication effect of the voice information.

[0131] Combination Figure 5As shown, the embodiment of the present disclosure provides a device for voice interaction, including a user feature information input module 501, a voice interaction prompt rule processing module 502, a target response text generation module 503, and a voice synthesis processing module 504. The user feature information input module 501 is configured to determine user corpus text information and user behavior status information based on the input user voice interaction information; the voice interaction prompt rule processing module 502 is configured to construct a voice interaction prompt rule based on the user behavior status information; the target response text generation module 503 is configured to determine the target response text according to the user corpus text information and the voice interaction prompt rule; the voice synthesis processing module 504 is configured to perform voice synthesis processing on the target response text, obtain the target response audio, and perform voice interaction.

[0132] Combination Figure 6 As shown, an embodiment of the present disclosure provides a device for voice interaction, including a processor (processor) 600 and a memory (memory) 601. Optionally, the device 60 may also include a communication interface (Communication Interface) 602 and a bus 603. Among them, the processor 600, the communication interface 602, and the memory 601 can communicate with each other through the bus 603. The communication interface 602 can be used for information transmission. The processor 600 can call the logic instructions in the memory 601 to execute the method for voice interaction of the above embodiment.

[0133] In addition, the logic instructions in the memory 601 described above can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0134] The memory 601 is a computer-readable storage medium that can be used to store software programs and computer executable programs, such as program instructions / modules corresponding to the method in the embodiment of the present disclosure. The processor 600 executes the functional application and data processing by running the program instructions / modules stored in the memory 601, that is, implementing the method for voice interaction in the above embodiment.

[0135] The memory 601 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 601 may include a high-speed random access memory and may also include a non-volatile memory.

[0136] The disclosed embodiments also provide a household appliance, comprising: a device body, and the above-mentioned device for voice interaction. The device for voice interaction is installed on the device body. The installation relationship described here is not limited to placement inside the product body, but also includes installation connections with other components of the household appliance, including but not limited to physical connections, electrical connections, or signal transmission connections. It can be understood by those skilled in the art that the device for voice interaction can be adapted to a feasible device body, thereby realizing other feasible embodiments.

[0137] An embodiment of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to execute the above-mentioned method for voice interaction.

[0138] The technical solution of the embodiment of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the embodiment of the present disclosure. The aforementioned storage medium may be a non-transient storage medium, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media that can store program codes.

[0139] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent only possible changes. Unless explicitly required, separate components and functions are optional, and the order of operation may vary. The parts and features of some embodiments may be included in or replace the parts and features of other embodiments. Moreover, the words used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and the claims, unless the context clearly indicates, the singular forms of "a", "an" and "the" are intended to include plural forms as well. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of listings containing one or more associated ones. In addition, when used in the present application, the term "comprise" and its variants "comprises" and / or comprising refer to the presence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the presence of other identical elements in the process, method or device comprising the elements. In this article, each embodiment may focus on the differences from other embodiments, and the same and similar parts between the various embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can refer to the description of the method part.

[0140] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. The technicians may use different methods for each specific application to implement the described functions, but such implementations should not be considered to exceed the scope of the embodiments of the present disclosure. The technicians may clearly understand that, for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above may refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0141] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units can be only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to implement this embodiment. In addition, each functional unit in the embodiment of the present disclosure may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit.

[0142] The flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to the embodiment of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and the block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in a different order from the order disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A method for voice interaction, characterized in that: include: Based on the input user voice interaction information, determine the user corpus text information and user behavior status information; Construct voice interaction prompt rules based on user behavior status information; Determine the target response text based on the user corpus text information and voice interaction prompt rules; Perform speech synthesis processing on the target response text to obtain the target response audio and perform voice interaction.

2. The method according to claim 1, characterized in that The method of constructing a voice interaction prompt rule based on user behavior status information includes: Based on the user behavior status information, determining at least one weighted keyword; Get the preset prompt rules; Construct voice interaction prompt rules based on weighted keywords and prompt rules.

3. The method according to claim 2, characterized in that The user behavior status information includes voiceprint recognition information, and / or emotion recognition information, and / or speech rate recognition information; then determining at least one weighted keyword based on the user behavior status information includes: Determine the user's age group, and / or user identity, and / or user gender as weighted keywords based on the voiceprint recognition information; Based on the emotion recognition information, the user's emotional state is determined as a weighted keyword; According to the speech rate recognition information, the speech rate of the user's conversation is determined as a weighted keyword.

4. The method according to claim 2, characterized in that: The voice interaction prompt rule is constructed according to the weighted keywords and prompt prompt rules, including: Determine one or more of the perspective rules, sentence rules, emotion rules, and function rules of voice interaction according to the weighted keywords; Based on the determined voice interaction rules and prompt rules, construct voice interaction prompt rules; The prompt rule includes: first reply information, which is used to indicate the execution result feedback text corresponding to the user's operation intention; or The first reply information is used to represent the execution result feedback text corresponding to the user's operation intention; and the second reply information is used to represent the guiding extended interaction text.

5. The method according to claim 1, characterized in that Determining the target response text according to the user corpus text information and the voice interaction prompt rule includes: Obtain customized pre-trained language models for the home appliance field; Inputting user corpus text information and voice interaction prompt rules into the home appliance field customized pre-trained language model; Output a target response text; the target response text is a personalized response text generated based on the user voice interaction information.

6. The method according to any one of claims 1 to 5, characterized in that: The step of performing speech synthesis processing on the target response text to obtain the target response audio includes: Perform speech synthesis on the target text; Performing sound effects processing on the synthesized speech according to the answer perspective of the target text; The voice after sound effect processing is processed with tail sound and speech speed to obtain the target response audio.

7. The method according to claim 6, characterized in that The step of performing sound effect processing on the synthesized speech according to the answer perspective of the target text includes: Determine the frequency points to be adjusted in the synthesized speech according to the answer perspective; Adjust the frequency of the frequency point to be adjusted to optimize the sound quality and expressiveness of the sound.

8. A device for voice interaction, comprising a processor and a memory storing program instructions, characterized in that: The processor is configured to execute the method for voice interaction as described in any one of claims 1 to 7 when running the program instructions.

9. A household appliance, characterized in that: include: Equipment body; The device for voice interaction as described in claim 8 is installed on the device body.

10. A computer-readable storage medium storing program instructions, characterized in that: When the program instructions are executed, the computer is used to execute the method for voice interaction as described in any one of claims 1 to 7.

Images