Voice conversation method and device, terminal equipment and storage medium

Abstract

The embodiment of the invention discloses a voice conversation method and device, terminal equipment and a storage medium, and the method comprises the steps: obtaining user state information which comprises first state information corresponding to a first user of first terminal equipment and / or second state information corresponding to a second user, the second user and the first user are in a relative and friend relationship; through a dialogue model, according to the first state information and / or the second state information, dialogue content is generated; and obtaining voiceprint features of a target user, generating dialogue voice according to the voiceprint features and the dialogue content, and playing the dialogue voice. By implementing the embodiment, the accompanying effect provided for the user by the AI voice chat can be enhanced, and the man-machine interaction of the terminal equipment is improved.

Description

Technical Field

[0001] The present application relates to the field of human-computer interaction technology, and in particular to a voice dialogue method, apparatus, terminal equipment and storage medium. Background Art

[0002] Currently, AI (Artificial Intelligence) chat technology is constantly developing. Users can provide companionship to users, especially to the elderly, by talking to AI chat programs. However, current AI chat programs still require users to actively wake up. For example, users must actively open the software and enter content to wake up, or users must wake up through voice before the AI ​​chat program will provide feedback to the user. The companionship effect that can be provided is poor. Summary of the invention

[0003] The embodiments of the present application disclose a voice dialogue method, apparatus, terminal device and storage medium, which can enhance the companionship effect provided by AI voice chat to users and improve the human-computer interactivity of the terminal device.

[0004] The present application embodiment discloses a voice conversation method, which is applied to a first terminal device, and the method includes:

[0005] Acquire user status information, where the user status information includes first status information corresponding to a first user of the first terminal device and / or second status information corresponding to a second user, where the second user is a relative or friend of the first user;

[0006] generating dialogue content according to the first state information and / or the second state information through a dialogue model;

[0007] Acquire the voiceprint features of the target user, generate a conversation voice according to the voiceprint features and the conversation content, and play the conversation voice; wherein the target user includes the second user, or other friends or relatives of the first user.

[0008] In one embodiment, generating the dialogue content according to the first state information and / or the second state information by using the dialogue model includes:

[0009] If it is detected that the first state information satisfies a first preset condition, generating a dialogue content through a dialogue model according to the first state information, or through the dialogue model according to the first state information and the second state information;

[0010] The first state information includes first scene information of the first user, and the first preset condition includes: the first scene information matches any first preset scene information among a plurality of first preset scene information; and / or,

[0011] The first state information includes emotion information of the first user, and the first preset condition includes: the emotion information of the first user matches any preset emotion information among a plurality of preset emotion information.

[0012] In one embodiment, generating the dialogue content according to the first state information and / or the second state information by using the dialogue model includes:

[0013] If the second state information is received, generating a dialogue content through the dialogue model according to the second state information, or through the dialogue model according to the first state information and the second state information; wherein the second state information is sent when the second terminal device detects that the second state information meets a second preset condition;

[0014] The second state information includes second scene information of the second user, and the second preset condition of the second user includes: the second scene information matches any second preset scene information among a plurality of second preset scene information.

[0015] In one embodiment, generating the dialogue content according to the first state information and / or the second state information through the dialogue model includes:

[0016] Determine a target user from a plurality of friends and relatives of the first user according to the first status information and / or the second status information;

[0017] using the first state information and / or the second state information, and the user information of the target user as input information of a dialogue model;

[0018] The input information is input into a dialogue model, and a dialogue feature of the target user is acquired through the dialogue model according to the user information, and based on the dialogue feature, a dialogue content corresponding to the target user is generated according to the first state information and / or the second state information.

[0019] In one embodiment, after using the first state information and / or the second state information and the user information of the target user as input information of the dialogue model, the method further includes:

[0020] If the input information is the same as the historical input information, obtaining historical information corresponding to the historical input information, wherein the historical information includes the historical input information, historical conversation content corresponding to the historical input information, and historical reply content of the first user;

[0021] The inputting the input information into a dialogue model, acquiring a dialogue feature of the target user according to the user information through the dialogue model, and generating dialogue content corresponding to the target user based on the dialogue feature and the first state information and / or the second state information, includes:

[0022] The input information and the historical information are input into the dialogue model, and the dialogue features of the target user are obtained according to the user information through the dialogue model, and based on the dialogue features, according to the first state information and / or the second state information, and the historical information, the dialogue content corresponding to the target user is generated.

[0023] In one embodiment, after playing the conversation voice, the method further includes:

[0024] detecting a reply voice of the first user to the conversation voice;

[0025] Performing content recognition on the reply voice to determine the reply content corresponding to the reply voice;

[0026] The reply content, the conversation content, and the input information are correspondingly saved as historical information.

[0027] In one embodiment, after performing content recognition on the reply voice and determining the reply content corresponding to the reply voice, the method further includes:

[0028] The reply content and the conversation content are sent to the terminal device of the target user.

[0029] The embodiment of the present application discloses a voice dialogue device, which is applied to a first terminal device, and the device includes:

[0030] an information acquisition module, configured to acquire user status information, wherein the user status information includes first status information corresponding to a first user of the first terminal device and / or second status information corresponding to a second user, wherein the second user is a relative or friend of the first user;

[0031] a content generation module, configured to generate dialogue content according to the first state information and / or the second state information through a dialogue model;

[0032] A voice playback module is used to obtain the voiceprint features of the target user, generate conversation voice according to the voiceprint features and the conversation content, and play the conversation voice; wherein the target user includes the second user, or other relatives and friends of the first user.

[0033] In one embodiment, the content generation module is also used to generate dialogue content through a dialogue model based on the first state information, or through the dialogue model based on the first state information and the second state information, if it is detected that the first state information satisfies a first preset condition; the first state information includes first scene information of the first user, and the first preset condition includes: the first scene information matches any first preset scene information of multiple first preset scene information; and / or, the first state information includes emotional information of the first user, and the first preset condition includes: the emotional information of the first user matches any preset emotional information of multiple preset emotional information.

[0034] In one embodiment, the content generation module is also used to generate dialogue content through the dialogue model according to the second state information, or through the dialogue model according to the first state information and the second state information, if the second state information is received; wherein the second state information is sent when the second terminal device detects that the second state information meets the second preset condition; the second state information includes the second scene information of the second user, and the second preset condition of the second user includes: the second scene information matches any second preset scene information among multiple second preset scene information.

[0035] In one embodiment, the content generation module is also used to determine a target user from a plurality of friends and relatives of the first user based on the first state information and / or the second state information; use the first state information and / or the second state information, and the user information of the target user as input information of a dialogue model; input the input information into the dialogue model, and obtain dialogue features of the target user according to the user information through the dialogue model, and based on the dialogue features, generate dialogue content corresponding to the target user according to the first state information and / or the second state information.

[0036] In one embodiment, the content generation module is also used to obtain historical information corresponding to the historical input information if the input information is the same as the historical input information, and the historical information includes the historical input information, the historical conversation content corresponding to the historical input information, and the historical reply content of the first user; the input information is input into the conversation model, and the conversation features of the target user are obtained through the conversation model according to the user information, and based on the conversation features, the conversation content corresponding to the target user is generated according to the first state information and / or the second state information, including: the input information and the historical information are input into the conversation model, and the conversation features of the target user are obtained through the conversation model according to the user information, and based on the conversation features, the conversation content corresponding to the target user is generated according to the first state information and / or the second state information, and the historical information.

[0037] In one embodiment, the voice dialogue device also includes an information storage module, which is used to detect the reply voice of the first user to the dialogue voice; perform content recognition on the reply voice to determine the reply content corresponding to the reply voice; and save the reply content, the dialogue content, and the input information as historical information.

[0038] In one embodiment, the voice dialogue device further comprises an information sending module, which is used to send the reply content and the dialogue content to the terminal device of the target user.

[0039] The present application discloses a terminal device, including:

[0040] A memory storing executable program code;

[0041] a processor coupled to the memory;

[0042] The processor calls the executable program code stored in the memory to execute the method described in any one of the above embodiments.

[0043] An embodiment of the present application discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the processor executes the method described in any one of the above embodiments.

[0044] Through the voice dialogue method, device, terminal device and storage medium disclosed in the embodiment of the present application, the first terminal device can obtain user status information, which can include the first status information corresponding to the first user of the first terminal device, and / or the second status information corresponding to the second user who is a relative or friend of the first user. The first terminal device can generate dialogue content according to the first status information and / or the second status information through the dialogue model. The first terminal device then obtains the voiceprint feature of the target user, generates dialogue voice according to the voiceprint feature and the dialogue content, and plays the dialogue voice, wherein the target user can include the second user, or other relative or friend users who are relatives or friends of the first user. In implementing this embodiment, the first terminal device can generate dialogue content according to the user status information through the dialogue model, and generate dialogue voice according to the voiceprint information of the target user and the dialogue content. The personalized dialogue content and dialogue voice can provide the first user with a more intimate and comfortable voice dialogue experience. Moreover, the first terminal device can actively initiate a voice dialogue according to the user status information, thereby enhancing the companionship effect provided by the AI ​​voice chat for the user and improving the human-computer interactivity of the terminal device. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 It is a schematic diagram of an application scenario of a voice dialogue method disclosed in an embodiment of the present application;

[0047] Figure 2 It is a flowchart of a voice dialogue method disclosed in an embodiment of the present application;

[0048] Figure 3 It is a flowchart of another voice dialogue method disclosed in an embodiment of the present application;

[0049] Figure 4 It is a flowchart of another voice dialogue method disclosed in an embodiment of the present application;

[0050] Figure 5 It is a flowchart of another voice dialogue method disclosed in an embodiment of the present application;

[0051] Figure 6 It is a modular schematic diagram of a voice dialogue device disclosed in an embodiment of the present application;

[0052] Figure 7It is a structural block diagram of a terminal device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0054] It should be noted that the terms "including" and "having" in the embodiments of the present application and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0055] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first terminal device may be referred to as a second terminal device, and similarly, a second terminal device may be referred to as a first terminal device. Both the first terminal device and the second terminal device are terminal devices, but they are not the same terminal device.

[0056] The embodiments of the present application disclose a voice dialogue method, apparatus, terminal device and storage medium, which can enhance the companionship effect provided by AI voice chat to users and improve the human-computer interactivity of the terminal device.

[0057] The following is a detailed description with reference to the accompanying drawings.

[0058] like Figure 1 As shown, Figure 1 1 is a schematic diagram of an application scenario of a voice dialogue method disclosed in an embodiment of the present application, and the application scenario may include a first terminal device 110, a first user 120, a second terminal device 130, and a second user 140, wherein the first terminal device 110 and the second terminal device 130 may include but are not limited to mobile phones, tablet computers, wearable devices, laptop computers, PCs (Personal Computers), etc., without limitation. The first terminal device 110 may detect the first state information of the first user 120, and the second terminal device 120 may detect the second state information of the second user 140, wherein the first user 120 and the second user 140 may be relatives or friends, for example, Figure 1The first user 120 in the example may be the grandmother of the second user 140. It is understandable that although Figure 1 Only one second user 140 is shown as a relative or friend of the first user 120, but the users who are relatives or friends of the first user 120 may include one or more, such as the son, daughter, friend, etc. of the first user 120 may also be the second user. Figure 1 The above is only an example, and each friend or relative user may correspond to a second terminal device 120, that is, the second terminal device 120 may include one or more.

[0059] The first terminal device 110 may include a dialogue model and a voice output device. The dialogue model is an AI chat program. The first terminal device 110 may generate dialogue content through the dialogue model, and then play the dialogue voice through the voice output device, so as to have a voice dialogue with the first user 120 and provide companionship for the first user 120. Optionally, there may be a communication connection between the first terminal device 110 and the second terminal device 130. The first terminal device 110 and the second terminal device 130 may also be connected to the same server. The second terminal device 130 may send the second state information corresponding to the second user 140 to the first terminal device 110. The second terminal device 130 may also send the second state information corresponding to the second user 140 to the server, and the server may send the second state information to the first terminal device 110.

[0060] In one embodiment, the first terminal device 110 may obtain user status information, which may include first status information corresponding to the first user 120 and / or second status information corresponding to the second user 140. The first terminal device 110 may generate a conversation content according to the first status information and / or the second status information through a conversation model. The first terminal device 110 then obtains the voiceprint feature of the target user, generates a conversation voice according to the voiceprint feature and the conversation content, and plays the conversation voice, wherein the target user includes the second user 140, or other friends and relatives of the first user 110. As an example, when the target user is the second user 140, the first terminal device 110 may obtain the voiceprint feature of the second user 140, generate a conversation voice according to the voiceprint feature of the second user 140 and the conversation content, and play the conversation voice. Figure 1 For example, the first terminal device 110 can use the voiceprint feature of the second user 140 to play the conversation voice of "Grandma, I'm out of school, what are you doing?"

[0061] like Figure 2 As shown, Figure 2: is a flow chart of a voice dialogue method disclosed in an embodiment of the present application. The voice dialogue method can be applied to the first terminal device in the above embodiment. The voice dialogue method may include the following steps:

[0062] Step 210, obtaining user status information.

[0063] It is understandable that, in order to provide companionship to the first user, the first terminal device may detect the first status information of the first user, and the second terminal device of the second user may also detect the second status information of the second user, the second user may refer to a relative or friend user of the first user, and the second terminal device may send the second status information to the first terminal device. The first terminal device may obtain user status information, which may include the first status information corresponding to the first user detected by the first terminal device, and / or the second status information corresponding to the second user sent by the second terminal device and received by the first terminal device.

[0064] Among them, the state information may include the scene information of the user's current location, the user's current emotional information, etc., without limitation. The scene information may be used to characterize the specific activity that the user is currently performing at a specific location, such as the user playing football on a football field, the user running in a park, the user sleeping at home, etc. Emotional information may include happiness, sadness, etc., without limitation. It is understandable that the content included in the state information may be specifically determined according to the permissions opened by the first terminal device or the second terminal device. Generally speaking, the more permissions opened by the terminal device, the more content included in the state information, so that the content of the voice conversation is closer to the real scene.

[0065] Optionally, the first terminal device may determine the first state information corresponding to the first user through the first user data collected by the sensor of the first terminal device, and the second terminal device may determine the second state information corresponding to the second user through the second user data collected by the sensor of the second terminal device. It can be understood that, as a user who needs to be accompanied, the data volume of the first user data may be greater than the data volume of the second user data. Optionally, the second user data may include necessary data and non-essential data, and the necessary data refers to the data that must be used when determining the second state information, and the non-essential data refers to the data that may not be used when determining the second state information. The necessary data and the non-essential data may be pre-set. The second terminal device may be provided with a collection switch for non-essential data. When the collection switch is in the on state, the second terminal device may collect non-essential data and necessary data. When the collection switch is in the off state, the second terminal device only collects necessary data and does not collect non-essential data. Through this method, the accuracy of the conversation content generated according to the user state information can be guaranteed while protecting the privacy of the second user.

[0066] Step 220: Generate dialogue content according to the first state information and / or the second state information through the dialogue model.

[0067] The first terminal device inputs the acquired first state information and / or second state information into the dialogue model, and the dialogue model can generate dialogue content according to the first state information and / or second state information. The dialogue model can be a neural network model, including but not limited to NLP (Natural Language Processing) models such as convolutional neural networks, recurrent neural networks, and Transformers. The embodiment of the present application does not limit the type of the dialogue model.

[0068] The dialogue model may have the ability to generate corresponding content based on the input user status information. The dialogue model may be trained based on a text data set. Furthermore, the dialogue model may store dialogue features and voiceprint features corresponding to multiple friends and relatives of the first user. The dialogue features are used to characterize the language expression habits of the friends and relatives, and the voiceprint features refer to the voice features of the friends and relatives.

[0069] Optionally, the dialogue model can generate dialogue content based on the first state information. For example, the first user is the grandmother of the second user, and the scene information included in the second state information represents the scene of the first user walking in the park. The dialogue model can generate the dialogue content of "Grandma, remember to drink water after walking" based on the first state information. Optionally, the dialogue model can generate dialogue content based on the second state information. For example, the first user is the grandmother of the second user, and the scene information included in the second state information represents the scene of the second user going home from school. The dialogue model can generate the dialogue content of "Grandma, I'm out of school, what are you doing?" based on the second state information. Optionally, the dialogue model can generate dialogue content based on the first state information and the second state information. For example, the first user is the grandmother of the second user, the emotional information included in the first state information represents that the first user is depressed, and the scene information included in the second state information represents that the second user is currently playing football on the football field. The dialogue model can generate the dialogue content of "Grandma, I'm playing football, do you want to come and see me?" based on the first state information and the second state information, so that the first user can get rid of the depressed mood. It should be noted that the above dialogue content is only an example and does not mean that the dialogue model will definitely generate the dialogue content.

[0070] Optionally, the first terminal device may also add real information such as the current time, current air quality, and current weather to the first state information to input into the dialogue model. The real information may refer to information that the first terminal device can obtain from the network, and the dialogue content generated by the dialogue model may also be associated with the real information. By implementing this embodiment, the accuracy of the dialogue content can be further improved.

[0071] Step 230, obtain the voiceprint features of the target user, generate conversation voice according to the voiceprint features and the conversation content, and play the conversation voice.

[0072] The target user refers to the user corresponding to the conversation voice output by the first terminal device, that is, the user simulated by the conversation between the first terminal device and the first user. The target user may include the second user, or other friends and relatives of the first user.

[0073] In some embodiments, if the first terminal device obtains the second state information corresponding to the second user, the target user may be the second user; if the first terminal device does not obtain the second state information corresponding to the second user, but only obtains the first state information corresponding to the first user, the target user may be any relative or friend user determined from multiple relative or friend users. For example, if the first terminal device obtains the first state information corresponding to the first user, the emotional information included in the first state information indicates that the first user is depressed, and the multiple relative or friend users may include the first user's son, daughter, friend, partner, etc., then any relative or friend user may be determined from the multiple relative or friend users as the target user to comfort the first user.

[0074] In other embodiments, if the first terminal device obtains the second state information corresponding to the second user, other relatives and friends other than the second user may be selected as the target user. For example, if the first user is the grandmother of the second user, and the scene information included in the second state information represents the scene of the second user returning home from school, then the target user may also be the father of the second user, that is, the son of the first user. The dialogue model may generate a dialogue content of "Mom, Xiaobao is out of school, I will go to see you later!" based on the second state information. This dialogue content is also only an example.

[0075] The target user's voiceprint features may be stored in the cloud or in the first terminal device, and the first terminal device may obtain the target user's voiceprint features, generate a conversation voice according to the voiceprint features and the conversation content through a TTS (Text-to-Speech) model, and then play the conversation voice. The TTS model may include a recurrent neural network model or a converter model.

[0076] As an optional implementation, in addition to playing the conversation voice, the above steps 210 to 230 can also be executed by the server, and the conversation model can also be deployed on the server. Specifically, the server can obtain the user state information, and generate the conversation content according to the first state information and / or the second state information through the conversation model, and then obtain the voiceprint features of the target user, and generate the conversation voice according to the voiceprint features and the conversation content, and finally send the conversation voice to the first terminal device, and the first terminal device plays the conversation voice. Implementing this implementation can reduce the amount of calculation of the first terminal device and speed up the generation of the conversation voice.

[0077] In an embodiment of the present application, the first terminal device can generate conversation content according to user status information through a conversation model, and generate conversation voice according to the voiceprint information of the target user and the conversation content. The personalized conversation content and conversation voice can provide the first user with a more intimate and comfortable voice conversation experience. Moreover, the first terminal device can actively initiate a voice conversation based on the user status information, thereby enhancing the companionship effect provided by AI voice chat to the user and improving the human-computer interactivity of the terminal device.

[0078] like Figure 3 As shown, Figure 3 : is a flowchart of another voice dialogue method disclosed in an embodiment of the present application. The voice dialogue method can be applied to the first terminal device in the above embodiment. The voice dialogue method may include the following steps:

[0079] Step 310: Obtain user status information.

[0080] Step 320: If it is detected that the first state information satisfies the first preset condition, a dialogue content is generated through a dialogue model according to the first state information, or through a dialogue model according to the first state information and the second state information.

[0081] The first terminal device can obtain the first status information corresponding to the first user at a preset time interval, and detect whether the first status information meets the first preset condition. If the first terminal device detects that the first status information meets the first preset condition, the first terminal device can input the first status information or the first status information and the second status information into the dialogue model, and the dialogue model can generate dialogue content based on the first status information, or the first status information and the second status information.

[0082] Optionally, the first state information may include the first scene information of the first user, and the first preset condition may include that the first scene information matches any of the first preset scene information in the plurality of first preset scene information. The first scene information is determined based on the data collected by the position sensor and the motion sensor of the first terminal device. The first terminal device may collect the position data through the position sensor of the first terminal device, and collect the motion data through the motion sensor, and determine the first scene information based on the data collected by the position sensor and the motion sensor. For example, the position data collected by the position sensor indicates that the first user is in the park, and the motion data collected by the motion sensor indicates that the first user is moving slowly. Then, the first terminal device may determine that the first scene information is that the first user is walking in the park. The plurality of first preset scene information may be preset manually, and may be preset based on the living habits of the first user. For example, the first user has the habit of walking in the park. In order to enable the first terminal device to communicate with the first user when the first user walks in the park, the scene of the first user walking in the park may be saved as a first preset scene information.

[0083] Optionally, the first state information may include emotional information of the first user, and the first preset condition may include that the emotional information of the first user matches any preset emotional information of multiple preset emotional information. The first terminal device may collect data through multiple biosensors, and determine the emotional information of the first user based on the data collected by the multiple biosensors. The present application does not limit the determination of the emotional information. Multiple first preset emotional information may include but are not limited to sadness, happiness, sorrow, etc., and there is no limitation on this.

[0084] Step 330: If the second state information is received, generate the dialogue content through the dialogue model according to the second state information, or through the dialogue model according to the first state information and the second state information.

[0085] Similar to the method of detecting the first status information by the first terminal device, the second terminal device may also obtain the second status information corresponding to the second user at preset time intervals, and detect whether the second status information satisfies the second preset condition. If the second terminal device detects that the second status information satisfies the second preset condition, the second terminal device may send the second status information that satisfies the second preset condition to the first terminal device. When the first terminal device receives the second status information sent by the second terminal device, it is determined that the second status information corresponding to the second user satisfies the second preset requirement. The first terminal device may input the second status information or the first status information and the second status information into the dialogue model. The dialogue model may generate dialogue content based on the second status information, or the first status information and the second status information.

[0086] Optionally, the second state information includes second scene information of the second user, the second scene information is determined by data collected by a position sensor and a motion sensor of a second terminal device of the second user, and the second preset condition of the second user includes that the second scene information matches any second preset scene information in a plurality of second preset scene information. The determination of the second state information and the second preset condition can refer to the determination of the first state information and the first preset condition in step 220, which will not be repeated here, but it should be noted that, since the living habits of the first user and the second user are different, the plurality of first preset scene information in the first preset condition and the plurality of second preset scene information in the second preset condition may also be different.

[0087] Step 340, obtain the voiceprint features of the target user, generate conversation voice according to the voiceprint features and the conversation content, and play the conversation voice.

[0088] In an embodiment of the present application, the first terminal device may generate dialogue content through a dialogue model based on the first state information, or through a dialogue model based on the first state information and the second state information, when detecting that the first state information satisfies the first preset condition, and the second terminal device may send the second state information to the first terminal device when detecting that the second state information satisfies the second preset condition, wherein the first state information may include first scene information and / or emotion information, and the second state information may include second scene information, and the first terminal device may generate dialogue content through a dialogue model based on the second state information, or through a dialogue model based on the first state information and the second state information, when receiving the second state information.

[0089] When implementing this embodiment, the first terminal device may generate conversation content based on the first scene information and / or emotion information, and may be combined with the second state information. The first terminal device may also generate conversation content based on the second scene information of the second user, and may be combined with the first state information. Therefore, the conversation content may be more targeted and personalized. The conversation content may be optimized according to the different scenes and needs of different users to provide a more customized voice conversation experience, which is equivalent to simulating a conversation and communication between the second user and the first user, making the AI ​​voice chat more realistic. The first terminal device may accompany the first user instead of the second user, thereby enhancing human-computer interactivity.

[0090] like Figure 4 As shown, Figure 4 : is a flowchart of another voice dialogue method disclosed in an embodiment of the present application. The voice dialogue method can be applied to the first terminal device in the above embodiment. The voice dialogue method may include the following steps:

[0091] Step 410, obtaining user status information.

[0092] Step 420: Determine a target user from among a plurality of friends and relatives of the first user based on the first status information and / or the second status information.

[0093] The first terminal device can generate a conversation index between the first user and each friend and relative user through a preset algorithm based on the first state information and / or the second state information. This application does not limit the preset algorithm. The conversation index can be used to characterize the first user's interest in having a conversation with the friend and relative user. The higher the conversation index, the greater the interest. The first terminal device can determine the friend and relative user with the highest conversation index as the target user. Optionally, the first terminal device can also randomly determine the target user from multiple friend and relative users who are friends and relatives of the first user.

[0094] Step 430: Use the first state information and / or the second state information, and the user information of the target user as input information of the dialogue model.

[0095] The user information of the target user may include the user identity information of the target user and the relationship between the target user and the first user.

[0096] Step 440, input the input information into the dialogue model, and obtain the dialogue features of the target user according to the user information through the dialogue model, and based on the dialogue features, generate dialogue content corresponding to the target user according to the first state information and / or the second state information.

[0097] The first terminal device may input the input information into the dialogue model, and the dialogue model may obtain the dialogue features of the target user according to the user identity information in the user information. Optionally, the dialogue features may include common dialogue words, dialogue order (e.g., inversion), etc., which are not limited. The dialogue model may generate dialogue content corresponding to the target user based on the dialogue features and the first state information and / or the second state information.

[0098] Step 450, obtain the voiceprint features of the target user, generate conversation voice according to the voiceprint features and the conversation content, and play the conversation voice.

[0099] In an embodiment of the present application, the first terminal device can determine the target user and obtain its conversation characteristics based on the first state information and / or the second state information, and based on the conversation characteristics, generate personalized conversation content corresponding to the target user through the conversation model according to the first state information and / or the second state information, thereby providing a voice accompaniment experience that better meets the needs of the first user.

[0100] like Figure 5 As shown, Figure 5: is a flowchart of another voice dialogue method disclosed in an embodiment of the present application. The voice dialogue method can be applied to the first terminal device in the above embodiment. The voice dialogue method may include the following steps:

[0101] Step 510, obtaining user status information.

[0102] Step 520: Determine a target user from among a plurality of friends and relatives of the first user based on the first status information and / or the second status information.

[0103] Step 530: Use the first state information and / or the second state information, and the user information of the target user as input information of the dialogue model.

[0104] The methods of steps 510 to 530 may refer to the methods of steps 410 to 430 in the above embodiment, and will not be described in detail here.

[0105] Step 540: If the input information is the same as the historical input information, then obtain the historical information corresponding to the historical input information.

[0106] The historical information may include historical input information, historical conversation content corresponding to the historical input information, and historical reply content of the first user. The first terminal device may compare the current input information with the historical input information included in each historical information. If the input information is the same as the historical input information, the first terminal device may obtain the historical information corresponding to the historical input information. The historical conversation content corresponding to the historical input information may be generated by the first terminal device inputting the historical input information into the conversation model, and the historical reply content of the first user may be determined based on the reply voice of the first user to the historical conversation voice corresponding to the historical conversation content.

[0107] Step 550, input the input information and historical information into the dialogue model, and obtain the dialogue characteristics of the target user according to the user information through the dialogue model, and based on the dialogue characteristics, generate dialogue content corresponding to the target user according to the first state information and / or the second state information, and the historical information.

[0108] The first terminal device can input the input information and the historical information into the dialogue model, and the dialogue model can also refer to the historical information when generating the dialogue content. Optionally, the dialogue model can determine the historical satisfaction of the first user with respect to the historical dialogue content based on the historical reply content in the historical information. When the satisfaction is greater than the satisfaction threshold, the historical dialogue content can be used as positive reference content to generate dialogue content similar to the historical dialogue content based on the dialogue feature, the first state information and / or the second state information. When the satisfaction is not greater than the satisfaction threshold, the historical dialogue content can be used as negative reference content to generate dialogue content dissimilar to the historical dialogue content based on the dialogue feature, the first state information and / or the second state information.

[0109] Step 560, obtain the voiceprint features of the target user, generate conversation voice according to the voiceprint features and the conversation content, and play the conversation voice.

[0110] Step 570: Detect the reply voice of the first user to the conversation voice.

[0111] After playing the conversation voice, the first terminal device can detect the reply voice of the first user to the conversation voice through the audio detection device.

[0112] Step 580: Perform content recognition on the reply voice to determine the reply content corresponding to the reply voice.

[0113] Among them, the first terminal device can perform content recognition on the reply voice through a voice recognition algorithm to determine the reply content corresponding to the reply voice. The embodiment of the present application does not limit the voice recognition algorithm. In one embodiment, after the first terminal device determines the reply content, the first terminal device can send the reply content and the conversation content to the terminal device of the target user, so that the target user can understand that the first terminal device simulates the conversation between the target user and the first user, avoiding information gap between the target user and the first user.

[0114] Step 590, save the reply content, conversation content, and input information as historical information.

[0115] In an embodiment of the present application, the first terminal device can also detect the reply voice of the first user to the conversation voice after playing the conversation voice, and perform content recognition on the reply voice to determine the reply content corresponding to the reply voice, and then save the reply content, conversation content, and input information as historical information accordingly, so that after each determination of the input information, the input information can be compared with the historical input information in the historical information, and when the input information is the same as the historical input information, the historical information corresponding to the historical input information can be obtained, so that the input information and the historical information can be input into the conversation model, so that the conversation model can generate conversation content corresponding to the target user based on the first state information and / or the second state information, as well as the historical information, and the accuracy of the conversation content is further improved by referring to the historical information.

[0116] like Figure 6 As shown, Figure 6 6 is a modular schematic diagram of a voice dialogue device disclosed in an embodiment of the present application. The voice dialogue device 600 can be applied to the first terminal device in the above embodiment. The voice dialogue device 600 may include an information acquisition module 610, a content generation module 620 and a voice playback module 630, wherein:

[0117] An information acquisition module, used to acquire user status information, the user status information including first status information corresponding to a first user of the first terminal device, and / or second status information corresponding to a second user, where the second user is a relative or friend of the first user;

[0118] A content generation module, used to generate dialogue content according to the first state information and / or the second state information through a dialogue model;

[0119] The voice playback module is used to obtain the voiceprint features of the target user, generate conversation voice according to the voiceprint features and the conversation content, and play the conversation voice; wherein the target user includes the second user, or other friends and relatives of the first user.

[0120] In one embodiment, the content generation module is also used to generate dialogue content through a dialogue model based on the first state information, or through a dialogue model based on the first state information and the second state information, if it is detected that the first state information satisfies a first preset condition; the first state information includes first scene information of the first user, and the first preset condition includes: the first scene information matches any first preset scene information of multiple first preset scene information; and / or, the first state information includes emotional information of the first user, and the first preset condition includes: the emotional information of the first user matches any preset emotional information of multiple preset emotional information.

[0121] In one embodiment, the content generation module is also used to generate dialogue content through a dialogue model based on the second state information, or through a dialogue model based on the first state information and the second state information, if second state information is received; wherein the second state information is sent when the second terminal device detects that the second state information meets a second preset condition; the second state information includes second scene information of the second user, and the second preset condition of the second user includes: the second scene information matches any second preset scene information among multiple second preset scene information.

[0122] In one embodiment, the content generation module is also used to determine a target user from a plurality of friends and relatives of the first user based on the first state information and / or the second state information; use the first state information and / or the second state information, and the user information of the target user as input information of the dialogue model; input the input information into the dialogue model, and obtain the dialogue features of the target user according to the user information through the dialogue model, and based on the dialogue features, generate dialogue content corresponding to the target user according to the first state information and / or the second state information.

[0123] In one embodiment, the content generation module is also used to obtain historical information corresponding to the historical input information if the input information is the same as the historical input information, the historical information including the historical input information, the historical conversation content corresponding to the historical input information, and the historical reply content of the first user; input the input information into the conversation model, and obtain the conversation features of the target user according to the user information through the conversation model, and based on the conversation features, generate the conversation content corresponding to the target user according to the first state information and / or the second state information, including: input the input information and the historical information into the conversation model, and obtain the conversation features of the target user according to the user information through the conversation model, and based on the conversation features, generate the conversation content corresponding to the target user according to the first state information and / or the second state information, and the historical information.

[0124] In one embodiment, the voice dialogue device also includes an information storage module, which is used to detect the reply voice of the first user to the dialogue voice; perform content recognition on the reply voice to determine the reply content corresponding to the reply voice; and save the reply content, dialogue content, and input information as historical information.

[0125] In one embodiment, the voice dialogue device further includes an information sending module, which is used to send the reply content and the dialogue content to the terminal device of the target user.

[0126] In an embodiment of the present application, the first terminal device can generate conversation content according to user status information through a conversation model, and generate conversation voice according to the voiceprint information of the target user and the conversation content. The personalized conversation content and conversation voice can provide the first user with a more intimate and comfortable voice conversation experience. Moreover, the first terminal device can actively initiate a voice conversation based on the user status information, thereby enhancing the companionship effect provided by AI voice chat to the user and improving the human-computer interactivity of the terminal device.

[0127] like Figure 7 As shown, in one embodiment, a terminal device is provided, which may include:

[0128] A memory 710 storing executable program codes;

[0129] a processor 720 coupled to the memory 710;

[0130] The processor 720 calls the executable program code stored in the memory 710 to implement the voice dialogue method provided in the above embodiments.

[0131] The memory 710 may include a random access memory (RAM) or a read-only memory (ROM). The memory 710 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 710 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc. The data storage area may also store data created by the terminal device during use, etc.

[0132] The processor 720 may include one or more processing cores. The processor 720 uses various interfaces and lines to connect various parts of the entire terminal device, and executes various functions of the terminal device and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 710, and calling data stored in the memory 710. Optionally, the processor 720 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 720 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 720, but may be implemented separately through a communication chip.

[0133] It is understandable that the terminal device may include more or fewer structural elements than those in the above structural block diagram, for example, a power module, physical buttons, a WiFi (Wireless Fidelity) module, a speaker, a Bluetooth module, a sensor, etc., and no limitation is made here.

[0134] An embodiment of the present application discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the methods described in the above embodiments.

[0135] In addition, an embodiment of the present application further discloses a computer program product. When the computer program product is run on a computer, the computer can execute all or part of the steps in any one of the voice dialogue methods described in the above embodiments.

[0136] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0137] The above is a detailed introduction to a voice conversation method, apparatus, terminal device and storage medium disclosed in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A voice dialogue method, characterized in that: Applied to a first terminal device, the method includes: Acquire user status information, where the user status information includes first status information corresponding to a first user of the first terminal device and / or second status information corresponding to a second user, where the second user is a relative or friend of the first user; generating dialogue content according to the first state information and / or the second state information through a dialogue model; Acquire the voiceprint features of the target user, generate a conversation voice according to the voiceprint features and the conversation content, and play the conversation voice; wherein the target user includes the second user, or other friends or relatives of the first user.

2. The method according to claim 1, characterized in that The generating the dialogue content according to the first state information and / or the second state information by using the dialogue model includes: If it is detected that the first state information satisfies a first preset condition, generating a dialogue content through a dialogue model according to the first state information, or through the dialogue model according to the first state information and the second state information; The first state information includes first scene information of the first user, and the first preset condition includes: the first scene information matches any first preset scene information among a plurality of first preset scene information; and / or, The first state information includes emotion information of the first user, and the first preset condition includes: the emotion information of the first user matches any preset emotion information among a plurality of preset emotion information.

3. The method according to claim 1, characterized in that: The generating the dialogue content according to the first state information and / or the second state information by using the dialogue model includes: If the second state information is received, generating a dialogue content through the dialogue model according to the second state information, or through the dialogue model according to the first state information and the second state information; wherein the second state information is sent when the second terminal device detects that the second state information meets a second preset condition; The second state information includes second scene information of the second user, and the second preset condition of the second user includes: the second scene information matches any second preset scene information among a plurality of second preset scene information.

4. The method according to claim 1, characterized in that: The generating the dialogue content according to the first state information and / or the second state information by using the dialogue model includes: Determine a target user from a plurality of friends and relatives of the first user according to the first status information and / or the second status information; using the first state information and / or the second state information, and the user information of the target user as input information of a dialogue model; The input information is input into a dialogue model, and a dialogue feature of the target user is acquired through the dialogue model according to the user information, and based on the dialogue feature, a dialogue content corresponding to the target user is generated according to the first state information and / or the second state information.

5. The method according to claim 4, characterized in that After taking the first state information and / or the second state information and the user information of the target user as input information of the dialogue model, the method further includes: If the input information is the same as the historical input information, obtaining historical information corresponding to the historical input information, wherein the historical information includes the historical input information, historical conversation content corresponding to the historical input information, and historical reply content of the first user; The inputting the input information into a dialogue model, acquiring a dialogue feature of the target user according to the user information through the dialogue model, and generating dialogue content corresponding to the target user based on the dialogue feature and the first state information and / or the second state information, includes: The input information and the historical information are input into the dialogue model, and the dialogue features of the target user are obtained according to the user information through the dialogue model, and based on the dialogue features, according to the first state information and / or the second state information, and the historical information, the dialogue content corresponding to the target user is generated.

6. The method according to claim 5, characterized in that After playing the conversation voice, the method further includes: detecting a reply voice of the first user to the conversation voice; Performing content recognition on the reply voice to determine the reply content corresponding to the reply voice; The reply content, the conversation content, and the input information are correspondingly saved as historical information.

7. The method according to claim 6, characterized in that After performing content recognition on the reply voice and determining the reply content corresponding to the reply voice, the method further includes: The reply content and the conversation content are sent to the terminal device of the target user.

8. A voice dialogue device, characterized in that: Applied to a first terminal device, the apparatus comprises: an information acquisition module, configured to acquire user status information, wherein the user status information includes first status information corresponding to a first user of the first terminal device and / or second status information corresponding to a second user, wherein the second user is a relative or friend of the first user; a content generation module, configured to generate dialogue content according to the first state information and / or the second state information through a dialogue model; A voice playback module is used to obtain the voiceprint features of the target user, generate conversation voice according to the voiceprint features and the conversation content, and play the conversation voice; wherein the target user includes the second user, or other relatives and friends of the first user.

9. A terminal device, characterized in that: include: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 7.

Images