Human-computer interaction method and device
By collecting and analyzing the voice data of the interactive object, combining the language style of the virtual assistant, providing voice interaction content that matches emotions and styles, it solves the problem of poor human-computer interaction experience in the existing technology and improves the naturalness and immersion of the interaction .
Patent Information
- Application Number
- CN202311586322.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-24
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2043-11-24
AI Technical Summary
When existing virtual assistants provide voice interaction, they cannot effectively match the current emotional state of the interactive object and the language style of the virtual assistant, resulting in a poor human-computer interaction experience.
By collecting voice data of the interactive object, determining its current emotional state, and determining the interactive text and tone based on the emotional state and the language style of the virtual assistant, providing matching voice interaction content, including collecting voice data, converting it into text and audio features. Combining the algorithm model, we judge the emotional state and adjust the interactive content according to the attribute parameters of the virtual assistant.
The human-computer interaction experience is improved, so that the voice interaction content of the virtual assistant matches the current emotional state of the interaction object and the language style of the virtual assistant, enhancing the naturalness and immersion of the interaction.
Smart Images

Figure CN117577088B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of human-computer interaction, and in particular to a human-computer interaction method and device. Background Art
[0002] A virtual assistant is an intelligent system based on artificial intelligence technology that can conduct language conversations and interactions with humans. However, when providing voice interaction, existing virtual assistants usually convert the conversation content of the interactive object into text data first, and then provide corresponding voice interaction to the interactive object based on the text data. This voice interaction can only provide simple replies to the conversation content of the interactive object, resulting in a poor human-computer interaction experience. Summary of the invention
[0003] In view of this, an embodiment of the present invention provides a human-computer interaction method and device, so that a virtual assistant provides the interactive object with voice interaction content that matches the current emotional state of the interactive object and the language style set by the current character of the virtual assistant, thereby improving the human-computer interaction experience.
[0004] In a first aspect, an embodiment of the present invention provides a human-computer interaction method applicable to a virtual assistant, the method comprising:
[0005] Collecting voice data of the interaction object;
[0006] Determining the current emotional state of the interactive object according to the voice data;
[0007] Determining the interactive text and interactive tone based at least on the current emotional state and the language style set by the current character of the virtual assistant;
[0008] Provide voice interaction to the interaction object according to the interaction text and the interaction tone.
[0009] Further, determining the current emotional state of the interactive object according to the voice data includes:
[0010] Converting the speech data into text data and extracting text features from the text data;
[0011] Extracting audio features from the speech data;
[0012] The current emotional state is determined according to the audio feature and / or the text feature.
[0013] Further, determining the interactive text and interactive tone based at least on the current emotional state and the language style set by the current character of the virtual assistant includes:
[0014] The interactive text and the interactive tone are determined according to the current emotional state, the voice style of the current character setting of the virtual assistant, and the current attribute parameters of the virtual assistant.
[0015] Further, providing voice interaction to the interactive object according to the interactive text and the interactive tone includes:
[0016] Converting the interactive text into interactive speech according to the interactive tone;
[0017] The interactive voice is output.
[0018] Furthermore, the method further comprises:
[0019] In response to detecting an interaction operation with the interactive object or detecting a virtual item donation operation of the interactive object, current attribute parameters of the virtual assistant are updated.
[0020] Furthermore, when providing voice interaction to the interactive object according to the interactive text and the interactive tone, the method further includes:
[0021] Control the virtual assistant image in the human-computer interaction interface to make body movements and expressions that match the currently provided voice interaction content.
[0022] Furthermore, before collecting the voice data of the interaction object, the method further includes:
[0023] In response to detecting that any preset interaction condition is satisfied, voice interaction content matching the currently satisfied preset interaction condition is actively provided to the interaction object.
[0024] In a second aspect, an embodiment of the present invention provides a human-computer interaction device, the device comprising:
[0025] A collection unit, used to collect voice data of an interaction object;
[0026] an emotion determination unit, configured to determine a current emotional state of the interaction object according to the voice data;
[0027] An interaction determination unit, configured to determine an interaction text and an interaction tone based at least on the current emotional state and a language style set by the current character of the virtual assistant;
[0028] A voice providing unit is used to provide voice interaction to the interactive object according to the interactive text and the interactive tone.
[0029] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method as described in any one of the first aspects is implemented.
[0030] In a fourth aspect, an embodiment of the present invention provides an electronic device, the device comprising:
[0031] a memory for storing one or more computer program instructions;
[0032] A processor, wherein the one or more computer program instructions are executed by the processor to implement the method as described in any one of the first aspects.
[0033] The human-computer interaction method of the embodiment of the present invention collects the voice data of the interactive object, determines the current emotional state of the interactive object based on the voice data, and then determines the interactive text and interactive tone based at least on the current emotional state and the language style of the current character setting of the virtual assistant, and then provides voice interaction to the interactive object based on the interactive text and interactive tone. Thus, by determining the interactive text and interactive tone based at least on the current emotional state of the interactive object and the language style of the current character setting of the virtual assistant, the embodiment of the present invention can enable the virtual assistant to provide the interactive object with voice interaction content that matches the current emotional state of the interactive object and the language style of the current character setting of the virtual assistant, thereby improving the human-computer interaction experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:
[0035] Figure 1 is a flow chart of a human-computer interaction method according to an embodiment of the present invention;
[0036] Figure 2 is a flow chart of a method for determining an emotional state according to an embodiment of the present invention;
[0037] Figure 3 A schematic diagram of a human-computer interaction process according to an embodiment of the present invention;
[0038] Figure 4 is a schematic diagram of a human-computer interaction device according to an embodiment of the present invention;
[0039] Figure 5 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0040] The present application is described below based on embodiments, but the present application is not limited to these embodiments. In the detailed description of the present application below, some specific details are described in detail. It is possible for those skilled in the art to fully understand the present application without the description of these details. In order to avoid confusing the essence of the present application, known methods, processes, flows, components and circuits are not described in detail.
[0041] In addition, persons of ordinary skill in the art will appreciate that the drawings provided herein are for illustration purposes and are not necessarily drawn to scale.
[0042] Unless the context clearly requires otherwise, the words "include", "comprising" and similar words throughout the application should be interpreted as including rather than exclusive or exhaustive; that is, the meaning is "including but not limited to".
[0043] In the description of this application, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, the meaning of "plurality" is two or more.
[0044] The solutions described in this specification and in the examples, if they involve the processing of personal information, will be processed on the premise of having a legal basis (such as obtaining the consent of the subject of personal information, or being necessary for the performance of a contract, etc.), and will only be processed within the scope of regulations or agreements. If a user refuses to process personal information other than the necessary information for basic functions, it will not affect the user's use of basic functions.
[0045] Figure 1 FIG. 1 is a flow chart of a human-computer interaction method according to an embodiment of the present invention. Figure 1 As shown, the human-computer interaction method may specifically include the following steps:
[0046] It should be understood that in this embodiment, the executor of the human-computer interaction method may be a virtual assistant. Specifically, the virtual assistant may implement the human-computer interaction method in this embodiment by calling and executing the corresponding program code. Among them, the virtual assistant may be mounted in a corresponding terminal device such as a smart phone, a tablet computer, a smart wearable device, a laptop computer, a desktop computer or a smart speaker. Alternatively, in some embodiments, the virtual assistant may also be mounted in a corresponding data processing terminal of a machine device such as a vehicle, a robot or an aircraft, and this application does not limit this.
[0047] It should be understood that in this embodiment, the voice interaction between the virtual assistant and the interactive object can be actively initiated by the interactive object. Specifically, the interactive object can actively initiate a dialogue with the virtual assistant. After the interactive object initiates the dialogue, the virtual assistant can give the interactive object a corresponding reply based on the content of the dialogue initiated by the interactive object.
[0048] Alternatively, the voice interaction between the virtual assistant and the interactive object may also be initiated by the virtual assistant. Specifically, when the virtual assistant detects that any preset interaction condition is met, it may actively provide the interactive object with voice interaction content that matches the currently met preset interaction condition.
[0049] In an optional implementation, the preset interaction condition may be detecting that the interactive object makes a corresponding operation. When this preset interaction condition is met, the voice interaction content provided by the virtual assistant may be content related to the current operation made by the interactive object (for example, when the interactive object is detected to invoke an application or when the interactive object is detected to perform a touch interaction with the virtual assistant image displayed in the human-computer interaction interface, the virtual assistant may respond to the application invocation operation currently performed by the interactive object or the touch interaction currently performed by the interactive object).
[0050] In another optional implementation, the preset interaction condition may also be detecting that the current time is within a corresponding time range. When this preset interaction condition is met, the voice interaction content provided by the virtual assistant may be a reminder to the interactive object that matches the current time range (for example, when it is detected that the current time is within the interactive object's usual rest time, the virtual assistant may remind the interactive object to take a rest). The time range may be specifically set by the interactive object itself or may be determined by the virtual assistant based on the interactive object's historical behavior, and this application does not limit this.
[0051] In another optional implementation, the preset interaction condition may also be detecting that the current attribute parameters of the virtual assistant meet the corresponding threshold. When this preset interaction condition is met, the voice interaction content provided by the virtual assistant may be content that matches the current attribute parameters (for example, when detecting that the virtual assistant's boredom level or mood value meets the corresponding threshold, the virtual assistant may provide the interactive object with voice interaction content that is used to indicate that the virtual assistant is currently bored or very happy).
[0052] It should be understood that the preset interaction conditions given above and the interaction content provided by the virtual assistant when each preset interaction condition is met are only for illustration. In actual application, they can be pre-set by the program developer or the interaction object.
[0053] Furthermore, in order not to affect the normal work and learning of the interactive object, in this embodiment, before providing active voice interaction to the interactive object, the virtual assistant can also detect whether the interactive object is currently busy. If it is confirmed that the interactive object is busy, the virtual assistant can temporarily stop active interaction to avoid disturbing the interactive object.
[0054] Optionally, the virtual assistant may determine the status by detecting the number of operations performed by the interactive object on the terminal device or the machine device within the current preset time period. For example, the virtual assistant may confirm that the interactive object is currently in a busy state when it detects that the number of operations performed by the interactive object on the terminal device or the machine device within the current preset time period is greater than a preset number threshold. Correspondingly, the virtual assistant may also confirm that the interactive object is not currently in a busy state when it detects that the number of operations performed by the interactive object on the terminal device or the machine device within the current preset time period is less than or equal to a preset number threshold.
[0055] S100: Collect voice data of the interaction object.
[0056] Specifically, during the process of voice interaction with the interactive object, the virtual assistant can call the voice collection device (such as a microphone) of the terminal device or machine device to collect the voice data of the interactive object in real time. The interactive object can specifically refer to any user who is currently performing voice interaction with the virtual assistant.
[0057] S200: Determine the current emotional state of the interaction object according to the voice data.
[0058] Specifically, language is an external manifestation of human emotional state. After collecting the voice data of the interactive object, the virtual assistant can determine the current emotional state of the interactive object based on the collected voice data. The current emotional state may refer to the emotional state of the interactive object at the current moment.
[0059] Optionally, the text features and audio features of human speech in different emotional states may be different (for example, when a human is angry, some words used to express anger will appear in the speech, and the tone, volume and speaking speed of the speech will also increase or become faster; for example, when a human is sad, some words used to express sadness, sadness or depression will appear in the speech, and the tone, volume and speaking speed of the speech will also decrease or slow down). Therefore, in step S200, the virtual assistant can determine the audio features and text features of the interactive object's conversational speech based on the voice data, and use the text features and / or the audio features as a basis for judging the current emotional state of the interactive object.
[0060] Figure 2 FIG. 1 is a flow chart of a method for determining an emotional state according to an embodiment of the present invention. Figure 2 As shown, the emotional state determination method may specifically include the following steps:
[0061] It should be understood that the current emotional state determination method can be specifically used to implement the above-mentioned step S200, that is, by executing the current emotional state determination method, the virtual assistant can determine the current emotional state of the interactive object.
[0062] S210: Convert the speech data into text data and extract text features from the text data.
[0063] Specifically, the virtual assistant may convert the voice data into text data and extract text features from the text data, wherein the text features may specifically be keywords in the text data used to represent the current emotional state of the interactive object.
[0064] Optionally, in step S210, the virtual assistant may perform speech recognition on the collected voice data, thereby converting the voice data into text data. Specifically, the voice data collected by the virtual assistant through the voice collection device is usually in the form of electrical signals, which represent the amplitude and frequency of the sound. In order to obtain the required information from the electrical signal, the virtual assistant can first digitize the signal, and then perform preprocessing operations such as denoising and enhancement on the digitized signal. Afterwards, the virtual assistant can input the preprocessed signal into the acoustic model to convert the signal into phonemes (that is, the smallest recognizable sound unit) through the acoustic model. After obtaining the phonemes converted from the signal, the virtual assistant can input the phonemes into the language model to match these phonemes through the language model to obtain the corresponding text data.
[0065] The acoustic model may be a trained Hidden Markov Model (HMM) or a neural network model, and the acoustic model may be used to model phonemes, thereby predicting the phonemes at each time point. The language model may be a trained N-gram model (N-Gram model) or a neural network model, and the language model may be used to predict the most likely word sequence based on phonemes, thereby determining the corresponding text data.
[0066] S220: Extract audio features from the speech data.
[0067] Specifically, the virtual assistant can also extract audio features from the collected voice data, wherein the audio features may include relevant features such as pitch, volume, and speech speed.
[0068] Optionally, in this embodiment, step S220 and step S210 may be performed simultaneously. Specifically, in the process of converting voice data into text data, the virtual assistant may simultaneously extract required audio features.
[0069] S230: Determine the current emotional state according to the audio feature and / or the text feature.
[0070] Specifically, after obtaining the audio features and the text features, the virtual assistant can determine the current emotional state based on the audio features and / or the text features.
[0071] Optionally, in step S230, when determining the current emotional state according to the audio features or the text features, the virtual assistant may input the extracted audio features or text features into the corresponding algorithm model respectively, so as to judge the probability that the current emotional state of the interactive object belongs to each preset emotional state through the algorithm model, and then determine the current emotional state of the interactive object according to the judgment result. Wherein, the preset emotional state may be all the preset emotional states that humans can have (for example, it may include fear, anger, happiness, sadness, disgust, surprise, contempt, pride, guilt, shame, jealousy and gratitude, etc.), and the algorithm model may be a neural network model or a logistic regression model trained by using the audio features or the text features labeled with the corresponding preset emotional states.
[0072] It should be understood that when the current emotional state is determined based on the audio features and the text features at the same time, the virtual assistant can also perform a weighted sum of the probabilities obtained using the audio features and the probabilities obtained using the text features after determining the probability that the current emotional state of the interactive object belongs to each preset emotional state through the algorithm model, and then determine the current emotional state of the interactive object based on the weighted sum result. Wherein, when performing the weighted sum, the weight corresponding to each probability result can be preset by the program developer. For example: assuming that the probability of the current emotional state of the interactive object being angry is 10% using the audio features, the probability of being happy is 80%, and the probability of being surprised is 10%, and the probability of the current emotional state of the interactive object being angry is 20% using the text features, the probability of being happy is 60%, and the probability of being surprised is 20%, and the weights of the two probability results are set to 1:1, the virtual assistant can perform a weighted sum of the probability results based on the weights, thereby obtaining the probability that the current emotional state of the interactive object is angry is 15%, the probability of being happy is 70%, and the probability of being surprised is 15%, and then it can be determined that the current emotional state of the interactive object is happy. It should be understood that the weight corresponding to each probability result can also be determined according to the uncertainty of each probability result, and the uncertainty can be used to characterize the accuracy of the judgment of each probability result.
[0073] Alternatively, in an optional implementation, when the current emotional state is determined based on audio features and text features at the same time, the virtual assistant can also input the audio features and the text features into a corresponding algorithm model to directly determine the current emotional state of the interactive object through the algorithm model. Compared with the above implementation, the algorithm model at this time can be a neural network model or a logistic regression model (logistic regression) trained by audio features and text features marked with corresponding preset emotional states. The algorithm model can combine audio features and text features to jointly determine the current emotional state of the interactive object.
[0074] It should be understood that in an optional implementation, the virtual assistant can also determine the current emotional state of the interactive object by interacting with the background server. Specifically, the virtual assistant can upload the collected voice data to the background server through the corresponding interface. After receiving the voice data, the background server will process and analyze the emotion of the voice data, determine the emotional state corresponding to the voice data, and then feed back the determined emotional state to the virtual assistant. Therefore, this embodiment can also enable the virtual assistant to determine the current emotional state of the interactive object.
[0075] It should be understood that the above-mentioned emotional state determination method is only for illustration. In actual application, the emotional state determination method adopted by the virtual assistant can be set by the program developer according to actual conditions.
[0076] S300: Determine the interaction text and interaction tone based at least on the current emotional state and the language style set by the current character of the virtual assistant.
[0077] Specifically, after determining the current emotional state of the interactive object, the virtual assistant can determine the interactive text and interactive tone based at least on the current emotional state and the language style set by its own current character.
[0078] Among them, the current character setting can be a character setting pre-selected by the interactive object for the virtual assistant. Specifically, in order to bring a better interactive experience to the interactive object, this embodiment will pre-configure a variety of character settings for the virtual assistant, each of which can have corresponding vocabulary, grammar, voice and rhetoric, thereby forming a unique language style. In step S300, the virtual assistant can determine the interactive text to be provided to the interactive object and the interactive tone used when providing the interactive text based on the language style of its current character setting and in combination with the current emotional state.
[0079] Optionally, in order to bring a better interactive experience to the interactive object, in this embodiment, the program developer can also set corresponding attribute parameters for the virtual assistant (such as level value, intimacy, mood value, boredom level and / or vitality value, etc.). The attribute parameters can affect the voice interaction content provided by the virtual assistant to the interactive object during the voice interaction between the virtual assistant and the interactive object (for example, intimacy can affect the closeness between the virtual assistant and the interactive object in the voice interaction content provided by the virtual assistant, and the vitality value can affect the mental state in the voice interaction content provided by the virtual assistant. It should be understood that each attribute parameter and the content affected by each attribute parameter can be set by the program developer according to actual needs, and this application does not limit this). In step S300, the virtual assistant can determine the interactive text and the interactive tone according to the current emotional state, the voice style set by its current character, and its current attribute parameters. Furthermore, in this embodiment, the attribute parameters of the virtual assistant can also be updated and adjusted accordingly by the virtual assistant after detecting the corresponding operation made by the interactive object. Specifically, the virtual assistant may update the current attribute parameter upon detecting an interaction operation with the interactive object (that is, voice interaction or touch interaction) or upon detecting a virtual item gift operation of the interactive object. Furthermore, updating the current attribute parameter may refer to increasing or decreasing the specific value of the attribute parameter. The virtual item may be virtual food, virtual gifts or other virtual props, etc., which is not limited in this application.
[0080] Optionally, when determining the interactive text and interactive tone, the virtual assistant can use a trained transformer model to implement the steps. The transformer model is a neural network model based on a self-attention mechanism, and the transformer model can be used to generate a corresponding interactive text with a tone change to reply to the interactive object's voice data based on the language style input by the virtual assistant, the current emotional state of the interactive object, and / or attribute parameters.
[0081] S400: providing voice interaction to the interaction object according to the interaction text and interaction tone.
[0082] Specifically, after determining the interaction text and the interaction tone, the virtual assistant can determine the interaction voice according to the interaction text and the interaction tone, and provide voice interaction to the interaction object according to the interaction voice.
[0083] Optionally, in step S400, the virtual assistant may convert the interactive text into interactive speech according to the interactive tone, and after obtaining the interactive speech, output the interactive speech through a speech output device of a terminal device or a machine device, thereby providing corresponding voice interaction to the interactive object.
[0084] Optionally, for a terminal device or machine device with a human-computer interaction interface, the virtual assistant can also display a corresponding virtual assistant image in the human-computer interaction interface. Furthermore, in order to enhance the immersion of the interactive object, while providing voice interaction to the interactive object, the virtual assistant can control the virtual assistant image in the human-computer interaction interface to make body movements and expressions that match the currently provided interactive content.
[0085] It should be understood that in this embodiment, the interactive object also performs touch interaction with the virtual assistant by touching the corresponding body parts of the virtual assistant image. Specifically, after detecting the touch interaction performed by the interactive object, the virtual assistant can make body movements and expressions corresponding to the touch interaction (for example, when the virtual assistant detects that the interactive object makes a touch operation on the head of the virtual assistant image, the virtual assistant makes a body movement of touching its own head and a happy expression). At the same time, the virtual assistant can also provide the interactive object with voice interaction content corresponding to the touch interaction (for example, the virtual assistant can provide the interactive object with voice interaction content used to indicate that it is currently very happy).
[0086] Figure 3 FIG. 1 is a schematic diagram of the human-computer interaction process of an embodiment of the present invention. Figure 3 As shown, the virtual assistant can collect voice data 31 of the interactive object during voice interaction with the interactive object. After collecting the voice data 31, on the one hand, the virtual assistant can extract the audio features 32 in the voice data 31. On the other hand, the virtual assistant can also convert the voice data into text data 33, and extract the text features 34 in the text data 33. After obtaining the audio features 32 and the text features 34, the virtual assistant can use the audio features 32 and the text features 34 to determine the current emotional state 35 of the interactive object. After determining the current emotional state 35, the virtual assistant can determine the interactive text and interactive tone 38 based on the language style 36 set by its current character, and in combination with the current emotional state 35 and its current attribute parameters 37. Then, the virtual assistant can provide voice interaction to the interactive object based on the interactive text and interactive tone 38.
[0087] The human-computer interaction method of the embodiment of the present invention collects the voice data of the interactive object, determines the current emotional state of the interactive object based on the voice data, and then determines the interactive text and interactive tone based at least on the current emotional state and the language style of the current character setting of the virtual assistant, and then provides voice interaction to the interactive object based on the interactive text and interactive tone. Thus, by determining the interactive text and interactive tone based at least on the current emotional state of the interactive object and the language style of the current character setting of the virtual assistant, the embodiment of the present invention can enable the virtual assistant to provide the interactive object with voice interaction content that matches the current emotional state of the interactive object and the language style of the current character setting of the virtual assistant, thereby improving the human-computer interaction experience.
[0088] Figure 4 Schematic diagram of a human-computer interaction device according to an embodiment of the present invention. Figure 4 As shown, the human-computer interaction device of the embodiment of the present invention includes a collection unit 41, an emotion determination unit 2, an interaction determination unit 43 and a voice providing unit 44.
[0089] Specifically, the collection unit 41 is used to collect voice data of the interaction object;
[0090] The emotion determination unit 42 is used to determine the current emotional state of the interaction object according to the voice data;
[0091] The interaction determination unit 43 is used to determine the interaction text and interaction tone based on at least the current emotional state and the language style set by the current character of the virtual assistant;
[0092] The voice providing unit 44 is used to provide voice interaction to the interaction object according to the interaction text and interaction tone.
[0093] The human-computer interaction device of the embodiment of the present invention collects the voice data of the interactive object, determines the current emotional state of the interactive object based on the voice data, and then determines the interactive text and interactive tone based at least on the current emotional state and the language style set by the current character of the virtual assistant, and then provides voice interaction to the interactive object based on the interactive text and interactive tone. Thus, by determining the interactive text and interactive tone based at least on the current emotional state of the interactive object and the language style set by the current character of the virtual assistant, the embodiment of the present invention can enable the virtual assistant to provide the interactive object with voice interaction content that matches the current emotional state of the interactive object and the language style set by the current character of the virtual assistant, thereby improving the human-computer interaction experience.
[0094] Figure 5 is a schematic diagram of an electronic device according to an embodiment of the present invention. The electronic device may be a terminal device equipped with a virtual assistant according to the above embodiment, such as Figure 5As shown, the electronic device includes a general computer hardware structure, which at least includes a processor 51 and a memory 52. The processor 51 and the memory 52 are connected via a bus 53. The memory 52 is suitable for storing instructions or programs executable by the processor 51. The processor 51 can be an independent microprocessor or a collection of one or more microprocessors. Thus, the processor 51 executes the instructions stored in the memory 52, thereby executing the method flow of the embodiment of the present invention as described above to realize the processing of data and the control of other devices. The bus 53 connects the above-mentioned multiple components together, and at the same time connects the above-mentioned components to the display controller 54 and the display device and the input / output (I / O) device 55. The input / output (I / O) device 55 can be a mouse, a keyboard, a modem, a network interface, a touch input device, a somatosensory input device, a printer, and other devices known in the art. Typically, the input / output device 55 is connected to the system via an input / output (I / O) controller 56.
[0095] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices (equipment) or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may adopt a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0096] The present application is described with reference to flowcharts of methods, apparatuses (devices) and computer program products according to embodiments of the present application. It should be understood that each process in the flowchart can be implemented by computer program instructions.
[0097] These computer program instructions may be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device that implements the process Figure 1 A function specified in a process or multiple processes.
[0098] These computer program instructions may also be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce instructions for implementing the process Figure 1 A device that specifies functions in a process or multiple processes.
[0099] Another embodiment of the present invention relates to a non-volatile storage medium for storing a computer-readable program, wherein the computer-readable program is used for a computer to execute part or all of the above method embodiments.
[0100] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by specifying relevant hardware through a program, and the program is stored in a storage medium, including several instructions for a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0101] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims
1. A human-computer interaction method, applicable to a virtual assistant, characterized in that: The method comprises: Collecting voice data of the interaction object; Determining the current emotional state of the interactive object according to the voice data; Determining the interactive text and interactive tone based at least on the current emotional state and the language style of the current character setting of the virtual assistant, wherein the language style represents the vocabulary, grammar, phonetics, and rhetoric corresponding to the current character setting; Providing voice interaction to the interactive object according to the interactive text and the interactive tone; Wherein, determining the interactive text and interactive tone based at least on the current emotional state and the language style set by the current character of the virtual assistant includes: The interaction text and the interaction tone are determined according to the current emotional state, the voice style of the current character setting of the virtual assistant and the current attribute parameters of the virtual assistant. The current attribute parameters include intimacy and / or vitality value. The intimacy is used to influence the closeness relationship between the virtual assistant and the interaction object in the voice interaction content provided by the virtual assistant. The vitality value is used to influence the mental state in the voice interaction content provided by the virtual assistant.
2. The method according to claim 1, characterized in that Determining the current emotional state of the interactive object according to the voice data includes: Converting the speech data into text data and extracting text features from the text data; Extracting audio features from the speech data; The current emotional state is determined according to the audio feature and / or the text feature.
3. The method according to claim 1, characterized in that Providing voice interaction to the interactive object according to the interactive text and the interactive tone includes: Converting the interactive text into interactive speech according to the interactive tone; The interactive voice is output.
4. The method according to claim 1, characterized in that: The method further comprises: In response to detecting an interaction operation with the interactive object or detecting a virtual item donation operation of the interactive object, current attribute parameters of the virtual assistant are updated.
5. The method according to claim 1, characterized in that When providing voice interaction to the interactive object according to the interactive text and the interactive tone, the method further includes: Control the virtual assistant image in the human-computer interaction interface to make body movements and expressions that match the currently provided voice interaction content.
6. The method according to claim 1, characterized in that Before collecting the voice data of the interaction object, the method further includes: In response to detecting that any preset interaction condition is satisfied, voice interaction content matching the currently satisfied preset interaction condition is actively provided to the interaction object.
7. A human-computer interaction device, characterized in that: The device comprises: A collection unit, used to collect voice data of an interaction object; an emotion determination unit, configured to determine a current emotional state of the interaction object according to the voice data; An interaction determination unit, configured to determine an interaction text and an interaction tone based at least on the current emotional state and a language style of a current character setting of the virtual assistant, wherein the language style represents vocabulary, grammar, phonetics, and rhetoric corresponding to the current character setting; A voice providing unit, configured to provide voice interaction to the interaction object according to the interaction text and interaction tone; Wherein, determining the interactive text and interactive tone based at least on the current emotional state and the language style set by the current character of the virtual assistant includes: The interaction text and the interaction tone are determined according to the current emotional state, the voice style of the current character setting of the virtual assistant and the current attribute parameters of the virtual assistant. The current attribute parameters include intimacy and / or vitality value. The intimacy is used to influence the closeness relationship between the virtual assistant and the interaction object in the voice interaction content provided by the virtual assistant. The vitality value is used to influence the mental state in the voice interaction content provided by the virtual assistant.
8. A computer-readable storage medium storing computer program instructions, characterized in that: The computer program instructions implement the method according to any one of claims 1 to 6 when executed by a processor.
9. An electronic device, characterized in that: The device comprises: a memory for storing one or more computer program instructions; A processor, wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of claims 1 to 6.
