Voice interaction method and electronic equipment

Abstract

The invention provides a voice interaction method and electronic equipment, and the method comprises the steps: obtaining the voice of a speaker in a voice interaction scene of a voice assistant, and obtaining the voiceprint features of the speaker according to the voice data of the speaker; according to the voiceprint features of the speaker, role recognition of the speaker is carried out in real time; the method comprises the following steps: acquiring voice data of a speaker, acquiring associated multi-user memory content in combination with the role of the speaker, determining a reply text for the voice data according to the voice data of the speaker and the associated multi-user memory content, and playing the reply text to the speaker. Therefore, the voice conversation has a better interaction effect, the intelligent degree and flexibility of the voice assistant are improved, and the voice interaction experience of the user is improved.

Description

Technical Field

[0001] The present application relates to the field of electronic technology, specifically to the field of voice assistants, and more particularly to a voice interaction method and electronic device. Background Art

[0002] Voice assistant (or "smart assistant") is one of the important functions of electronic devices such as electronic devices. Controlling electronic devices such as electronic devices through voice commands has become a common form of human-computer interaction. After the user starts the voice assistant on the electronic device, he can use voice commands to control multiple third-party applications (applications, Apps) on the electronic device to perform corresponding actions, such as playing music, setting alarms, searching for information, etc. This process can free the user's hands, facilitate the user to control the electronic device, and improve the user experience.

[0003] In recent years, with the development of electronic technology and equipment, users are no longer satisfied with voice assistants that can only assist in the operation and chatting functions of electronic devices. Users have more demands on the capabilities of voice assistants. For example, users may want voice assistants to be more intelligent and more flexible in the voice interaction process with voice assistants. At present, the intelligence and flexibility of voice assistants are still insufficient. Summary of the invention

[0004] The present application provides a voice interaction method and electronic device, which can improve the intelligence and flexibility of a voice assistant and enhance the user's voice interaction experience.

[0005] In a first aspect, a method for voice interaction is provided, which is applied to an electronic device equipped with a voice assistant, the method comprising: acquiring voice data of a first user, and determining an identity of the first user based on the voice data of the first user; when the first user is a registered user in a preset database, acquiring memory content associated with the identity of the first user from the preset database based on the identity of the first user, the memory content associated with the identity of the first user being content extracted by the voice assistant based on the acquired voice data emitted by the first user; determining a first reply text for the voice data of the first user based on the memory content associated with the identity of the first user, and presenting the first reply text to the first user.

[0006] During the conversation between the user and the voice assistant, the above method can identify the user's identity based on the voice emitted by the user. When the user is identified as a registered user in the preset database, the present application can obtain the memory content associated with the result of the user's identity identification in the preset database, and based on the associated memory content, determine the reply text of the voice emitted by the user and present it to the user. This method can determine the voice reply text or content based on the memory content associated with the user, so that the voice conversation has a better interactive effect, improves the intelligence and flexibility of the voice assistant, and thus improves the user's voice interaction experience.

[0007] Especially for the situation where multiple people share the same voice assistant, this application can identify the user based on the voice data sent by the user to determine whether the current user is a historical user or a new user. When the current user is determined to be a historical user, the memory content of the historical user is directly read, and the appropriate reply content is output, keeping the reply content consistent with the user's historical memory, personality style, etc., and finally converting the text reply content into audio and outputting it to the user.

[0008] According to a second aspect, an electronic device is provided, comprising: an acquisition unit for acquiring voice data of a first user; a processing unit for determining an identity of the first user based on the voice data of the first user; when the first user is a registered user in a target preset database, the processing unit is further used to acquire memory content associated with the identity of the first user from the target preset database based on the identity of the first user, wherein the memory content associated with the identity of the first user is content extracted by the voice assistant based on all voice data acquired from the first user; determining a first reply text for the voice data instruction of the first user based on the memory content associated with the identity of the first user; and a response unit for presenting the first reply text to the first user.

[0009] According to a third aspect, a server is provided, characterized in that it comprises: a receiving module for receiving memory content associated with a user identifier and sent by a voice assistant of an electronic device, and a storage module for storing the memory content.

[0010] In a fourth aspect, a device is provided. The device is included in an electronic device, and the device has the function of implementing the behavior of the electronic device in the first aspect.

[0011] The function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions. For example, a display module or unit, a detection module or unit, a processing module or unit, etc.

[0012] In a fifth aspect, the present application provides an electronic device, comprising: a display screen; one or more processors; a memory; a plurality of applications; and one or more computer programs. The one or more computer programs are stored in the memory, and the one or more computer programs include instructions. When the instructions are executed by the electronic device, the electronic device executes the method in the first aspect.

[0013] In a sixth aspect, the present application provides a graphical user interface system on an electronic device, wherein the electronic device has a display screen, one or more memories, and one or more processors, wherein the one or more processors are used to execute one or more computer programs stored in the one or more memories, and the graphical user interface system includes a graphical user interface displayed when the electronic device executes the method of the first aspect.

[0014] In a seventh aspect, the present application provides a computer storage medium, comprising computer instructions, which, when executed on an electronic device, enables the electronic device to execute the method of the first aspect.

[0015] In an eighth aspect, the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the method in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a user interface diagram of a user turning on the voice assistant function of an electronic device through a settings application.

[0017] Figure 2 This is a diagram of an interface for human-computer interaction between a user and a voice assistant.

[0018] Figure 3 It is a schematic flowchart of a method for voice interaction provided in an embodiment of the present application.

[0019] Figure 4 It is a schematic block diagram of a voice interaction method provided in an embodiment of the present application.

[0020] Figure 5 This is a schematic flowchart of a voiceprint recognition process provided in an embodiment of the present application.

[0021] Figure 6 It is a schematic diagram of a voiceprint comparison solution provided in an embodiment of the present application.

[0022] Figure 7 This is a schematic diagram of the processing process of the deep neural network EPACA-TDNN.

[0023] Figure 8This is a schematic diagram of an interface for human-computer interaction between a user and a voice assistant provided in an embodiment of the present application.

[0024] Fig. 9 It is a schematic block diagram of another example of a human-computer interaction device provided in an embodiment of the present application.

[0025] Fig.10 It is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] To facilitate understanding, the following will take an electronic device equipped with a voice assistant as an example and provide a detailed explanation in conjunction with the accompanying drawings.

[0027] It should be understood that voice assistants have the "wake-up and use" coverage capability, can realize a new type of human-computer interaction of "liberating users' hands and achieving efficient operation", have a variety of wake-up methods, and can easily realize direct voice access to content.

[0028] Optionally, in an embodiment of the present application, a user may enable the voice assistant function of an electronic device in a variety of possible ways. Several possible operation processes are described below.

[0029] In one possible implementation, a user may enable the voice assistant function of an electronic device through a settings application.

[0030] Figure 1 This is a user interface diagram of a user turning on the voice assistant function of an electronic device through a settings application.

[0031] For example, Figure 1 Figure (a) in the figure shows an interface 101 displayed by the electronic device in unlock mode, and the interface 101 displays a weather clock component and multiple applications (applications, Apps), etc. Among them, the applications may include phone, navigation, music, and settings, etc. It should be understood that the interface 101 may also include other more applications, which is not limited in the embodiment of the present application.

[0032] The user executes Figure 1 In the operation shown in Figure (a), click the icon of the setting application, and in response to the user's click operation, the electronic device displays Figure 1 FIG. 100 shows a main interface 102 of the setting application, which may include multiple operation options. The user clicks the "Xiaobu Assistant" option on the main interface 102 of the setting application. In response to the user's click operation, the electronic device displays the following Figure 1 (c) of the diagram shows the Xiaobu assistant interface 103; the user clicks the "voice wake-up" option on the Xiaobu assistant interface 103, and in response to the user's click operation, the electronic device displays the following Figure 1Interface 104 is shown in FIG.

[0033] Optionally, a "voice wake-up" switch is included on the interface 104, and the user can perform the following operations: Figure 1 As shown in Figure (d), click the "Voice Wake-up" switch to keep the "Voice Wake-up" switch in the "ON" state. Figure 1 In the operation process described, the user has enabled the voice assistant function of the electronic device. In the subsequent use process, the user can wake up the voice assistant of the electronic device through the wake-up word "Xiaobu Xiaobu" and enter the voice interaction mode or dialogue mode.

[0034] In another possible implementation, in addition to the above operation process, the user can also wake up the voice assistant of the electronic device by long pressing the power button for 0.5 seconds to enter the voice interaction mode or conversation mode.

[0035] Alternatively, the user can also wake up the voice assistant of the electronic device by pressing the operation button of the floating ball on the interface of the electronic device. Alternatively, the user can also wake up the voice assistant of the electronic device by pressing the operation button on accessories such as headphones connected to the electronic device and enter the voice interaction mode or conversation mode. The embodiments of the present application do not limit the user to waking up the voice assistant of the electronic device.

[0036] When the user wakes up the voice assistant of the electronic device and enters the conversation mode, the voice assistant of the electronic device can obtain the user's voice content and make corresponding feedback based on the recognized voice content.

[0037] Specifically, some current voice assistants can help users record meetings, schedules, etc. based on the user's voice content, but cannot provide users with accurate answers; or, based on the user's voice content, they search for relevant content for the user by calling the browser and pop up the searched content for the user to view.

[0038] Figure 2 This is a schematic diagram of an interface for human-computer interaction between a user and a voice assistant. For example, after the voice assistant function of an electronic device is turned on and the conversation mode is entered, the user may interact with the voice assistant of the electronic device as follows: Figure 2 The dialogue shown in the interface 201 of FIG. (a) is as follows:

[0039] User: Xiaobu Xiaobu;

[0040] Voice assistant robot (bot): I am here;

[0041] User: Video conference will be held at 9:00 am tomorrow;

[0042] Voice assistant robot (bot): has searched for you for content related to "video conferencing".

[0043] After the above conversation occurs, the electronic device may pop up a message such as Figure 2 The interface 202 shown in FIG. 8 (b) shows a browser searching for related content for a user.

[0044] However, the content that the browser displayed on the interface 202 searches for the user may not be the content that the user expects to obtain. The chat style of the voice assistant robot is the same for everyone. During the conversation, it cannot chat with the user flexibly like a friend. It is not intelligent enough and cannot provide users with personalized voice interaction services and meet users' needs, resulting in a poor user experience.

[0045] The embodiment of the present application provides a method for voice interaction. During the voice interaction process, the method can improve the intelligence and flexibility of the voice assistant and improve the user's voice interaction experience.

[0046] The following will describe the voice interaction method provided in the embodiments of the present application in detail with reference to the accompanying drawings in the embodiments of the present application.

[0047] Figure 3 It is a schematic flowchart of a method for voice interaction provided in an embodiment of the present application.

[0048] It should be understood that Figure 3 The method 300 and Figure 4 The process 400 shown can be applied to any electronic device equipped with a voice assistant or having voice interaction function, such as a mobile phone, a smart speaker, a smart screen, a tablet, etc. The embodiment of the present application does not limit the type of electronic device.

[0049] For example, Figure 3 As shown, the method 300 includes the following steps:

[0050] Step 310: Acquire voice data of the first user.

[0051] Optionally, "voice data of the first user" can be understood as audio (query) emitted by the first user, that is, audio (query) emitted after the user wakes up the voice assistant of the electronic device. The audio may include one or more information such as voice wake-up words, voice commands, chat content, user demands, etc., which is not limited to this embodiment of the present application.

[0052] Exemplarily, the voice data of the first user may not include a voice wake-up word; or, the audio (query) issued by the first user includes both a voice wake-up word and other content such as an event that the user wants to execute, for example, "Xiaobu Xiaobu, remind me that I have a music class at 2pm", where "Xiaobu Xiaobu" is the wake-up word and "Remind me that I have a music class at 2pm" is the event that the user wants to execute, which will not be repeated here.

[0053] Step 320: Determine the identity of the first user based on the voice data of the first user.

[0054] Optionally, after the intelligent assistant receives the voice data or audio (query) of the first user, the voice or audio (query) signal can be converted into text (query) through an automatic speech recognition (ASR) module.

[0055] It should be understood that the voice assistant can provide an ASR module with speech recognition, a large-scale language model (LLM) module, and a text-to-speech (TTS) module, etc., wherein the ASR module can convert the user's speech into text information; the LLM module can convert natural language into a language that the machine can understand and judge; the TTS module can convert the text into a voice signal and output it to the user. For simplicity, the functions and principles of each module in the voice assistant are not repeated in the embodiments of the present application.

[0056] Step 330: When the first user is a registered user in a preset database, the memory content associated with the identity identifier of the first user is obtained from the preset database according to the identity identifier of the first user.

[0057] Optionally, the voice assistant can identify the user based on the voiceprint features of the first user.

[0058] For example, after the voice assistant obtains the user's voice, it can input the collected user voice into the voiceprint characterization model, extract and output the voiceprint features of the current user, and then perform voiceprint recognition based on the voiceprint features of the current user, and then perform different operations based on the voiceprint recognition results.

[0059] It should be understood that in the embodiment of the present application, the "preset database" can be used to store the voiceprint features and memory contents of multiple different users.

[0060] Optionally, the "memory content in the preset database" may include the content or information contained in the voice or audio input by the user during the use of the voice assistant, which can be called "non-relational data". For example, the user's voice words, names, event elements, etc., where the event elements may include all elements related to the event, such as the time, place, and person of the event, which are not limited in the embodiments of the present application.

[0061] Exemplarily, taking the first user as an example, "the memory content associated with the identity identifier of the first user" is the content extracted by the voice assistant based on the voice data emitted by the first user obtained in a certain period of time.

[0062] Optionally, the "memory content associated with the identity of the first user" may include one or more information of the first user's name, personalized preferences, and the first user's to-be-executed schedule, activities, and event elements. Among them, "personalized preferences" may include one or more contents that can characterize the user's behavior habits, such as the first user's historical behavior habits, personality traits, speech tone, and pitch, etc., which are not limited in this embodiment of the application.

[0063] Alternatively, the "memory content associated with the identity identifier of the first user" may also include the role information of the first user. Optionally, the role information of the first user may include one or more of the identity, gender, occupation, age, language, and nationality of the first user.

[0064] In a possible implementation, the method 300 also includes: obtaining identity information of each user among multiple users, voiceprint characteristics of each user, and memory content associated with each user; generating the preset database based on the identity information of each user, the voiceprint characteristics of each user, and the memory content associated with each user.

[0065] Optionally, the "preset database" can use a database solution such as MongoDB for persistent deployment, for example, in the form of "key-value pairs" to store data generated during the user's conversation with the voice assistant.

[0066] Optionally, in an embodiment of the present application, the "user account" may be used as the primary key (or referred to as the "primary identifier").

[0067] It should be understood that for each voice assistant, a unique user account can be associated, and the user account can be a login account of the electronic device. In addition, the same user account can be logged in on multiple different electronic devices at the same time.

[0068] For example, in a home scenario, the same user account can be logged in simultaneously on multiple electronic devices of different types, such as mobile phones, smart speakers, smart screens, smart watches, tablets, computers, etc.

[0069] It should also be understood that the same user account can serve a relatively fixed group of users. For example, in a family scenario, all family members can be associated with the same user account; or, in a work scenario, members of the same company can be associated with the same user account. The embodiments of the present application do not limit this.

[0070] In another possible implementation, for each user account, the voice assistant can store the voiceprint features of one or more users and all memory content associated with each user.

[0071] Table 1 lists the storage content and information of a possible preset database.

[0072] For example, take the smart speaker in the home scene as an example, as shown in Table 1, assuming that the smart speaker in the current home scene is logged in to the first user account (referred to as user-1), and multiple users in the home scene (such as Xiao Ming, Xiao Ming's mother, Xiao Ming's father, Xiao Ming's sister...) have used the conversation function of the voice assistant of the smart speaker. In the process of establishing the preset database, the voiceprint key value of Xiao Ming (user 1) can be marked as 01 (referred to as speaker-1), and the voiceprint feature vector of Xiao Ming (user 1) is vector (or embedding-1), Xiao Ming’s words include “I have music class at 2pm”, “I will get up at 8am tomorrow and there will be a parent-teacher meeting at 2pm”, “My mother’s birthday is on the 16th”, etc. In other words, all the memories that Xiao Ming has communicated with the smart speaker can be stored in the database, and all the memories are associated or bound to Xiao Ming’s identity.

[0073] Similarly, the voiceprint key value of Xiao Ming's mother (user 2) is marked as 02 (referred to as speaker-2), and the voiceprint feature vector of Xiao Ming's mother (user 2) is vector (or recorded as embedding-2), and all the content said by Xiao Ming's mother is recorded, and all the content is associated or bound to the identity of Xiao Ming's mother. And so on, the identities and content of all users in all family scenes, such as Xiao Ming's father (user 3) and Xiao Ming's sister (user 4), are recorded.

[0074] It should be noted here that the determination of the “identity of the corresponding speaker” in Table 1 may be determined by extracting the speech content input by the user.

[0075] For example, if user 1 can say "Xiaobu Xiaobu, I am Xiaoming" during the conversation, then the voice assistant can mark user 1 as "Xiaoming", and user 1's voiceprint key "01 (speaker-1)" and all memory content are associated with Xiaoming. User 2 can say "Xiaobu Xiaobu, I am Xiaoming's mother" during the conversation, then the voice assistant can mark user 2's identity as "Xiaoming's mother", and user 2's voiceprint key "02 (speaker-2)" and all memory content are associated with Xiaoming's mother, and so on.

[0076] Table 1

[0077]

[0078]

[0079] In another possible implementation, the "preset database" also stores voiceprint features of multiple registered users. The voiceprint features can be used to identify the user. Specifically, determining the identity of the first user based on the voice data of the first user includes: extracting the voiceprint features of the first user based on the voice data of the first user; matching the voiceprint features of the first user with the voiceprint features of multiple registered users in the preset database to determine the identity of the first user.

[0080] Optionally, the matching processing of the voiceprint features of the first user with the voiceprint features of multiple registered users in the preset database includes: when the degree of matching between the voiceprint features of the first user and the voiceprint features of a target user among the multiple registered users is greater than or equal to a preset value, determining that the first user is the target user registered in the preset database; or, when the degree of matching between the voiceprint features of the first user and the voiceprint features of any one of the multiple registered users is less than the preset value, determining that the first user is not a registered user in the preset database.

[0081] For example, the voice assistant described in the aforementioned S330 can identify the user's identity based on the voiceprint features of the first user. After acquiring the user's voice, the voice assistant can input the collected user voice into the voiceprint representation model, extract and output the voiceprint features of the current user, and then perform voiceprint recognition in combination with the voiceprint features of the current user, and then perform different operations according to the voiceprint recognition results.

[0082] Optionally, the "voiceprint feature" in the embodiment of the present application can be represented by a multi-dimensional vector, called a "voiceprint feature vector", which is not limited in the embodiment of the present application.

[0083] Optionally, in an embodiment of the present application, members of the same group are members who trust each other, that is, each user account can share the historical memory content of all users in the group in the preset database.

[0084] For example, assume that Table 1 is the preset database information registered and stored by the voice assistant after the smart speaker logs in to Xiao Ming's user account. The preset database includes Xiao Ming's user account, registered users associated with the account, and memory content associated with each registered user. When Xiao Ming's user account is logged in on any other electronic device such as a smart screen or tablet, the preset database information in Table 1 associated with Xiao Ming's user account can also be directly obtained.

[0085] Alternatively, in this family scenario, Xiao Ming, Xiao Ming's mother, Xiao Ming's father, Xiao Ming's sister... are members of the same group, and the members can trust each other. Then, when the smart speaker logs out of Xiao Ming's user account and logs in to Xiao Ming's mother's user account, it can also obtain the database information of Table 1 associated with Xiao Ming's user account.

[0086] Alternatively, in this home scenario, Xiao Ming's mobile phone and the smart screen at home are mutually trusted devices in this home scenario, then Xiao Ming's mobile phone and the smart screen at home can obtain the database information of Table 1 associated with Xiao Ming's user account. The embodiment of the present application is not limited to this.

[0087] In a possible implementation, the method further includes: storing the voiceprint features of each user in the preset database in the local storage space of the electronic device, and storing the memory content associated with each user in the server of the voice assistant.

[0088] Exemplarily, the voice assistant stores all the voiceprint feature vectors and memory contents shown in Table 1 locally on the electronic device where the voice assistant is installed, for example, performing localized storage of the electronic device.

[0089] The above storage method is suitable for electronic devices with large local storage space, which facilitates and quickly obtains the voiceprint feature vectors and memory content contained in the database from the local area, thereby improving the conversation response speed.

[0090] In another possible implementation, for each user account, the voice assistant can store the identification of one or more users, the voiceprint characteristics of each user, and all memory content associated with each user.

[0091] Optionally, the voice assistant can store the voiceprint features of each registered user in the local storage space of the electronic device, and store the memory content associated with each registered user in the server of the voice assistant. In other words, during the process of building and storing the database, the voice assistant can store the content of the database locally on the electronic device where the voice assistant is installed, or the voice assistant can also store the content of the database in a distributed manner.

[0092] For example, the voice assistant converts the voiceprint key value 01 (speaker-1) of Xiao Ming (user 1) in the database shown in Table 1, the voiceprint feature vector of Xiao Ming (user 1) Stored locally on the electronic device with the voice assistant installed, all the memory contents Xiao Ming has communicated with the smart speaker, including "I have music class at 2 pm", "I will get up at 8 am tomorrow and there will be a parent-teacher meeting at 2 pm", "My mother's birthday is on the 16th", etc., can be stored on the server, and all the memory contents on the server side are associated or bound to Xiao Ming's identity. For example, all the memory contents Xiao Ming has communicated with the smart speaker carry Xiao Ming's identity (user 1) or the registered voiceprint key 01 (speaker-1).

[0093] In other words, for the registered voiceprint part, the data is organized in the manner of [speaker: voiceprint feature vector] in Table 1. For example, speaker-1, speaker-2, ..., speaker-n represent the identifiers of the speakers who have had a conversation with the voice assistant, and embedding-1, embedding-2, ..., embedding-n represent the voiceprint feature vectors corresponding to different speakers.

[0094] For the memory content of multiple users, the data is organized in the manner of [speaker-X: memory List] in Table 1, where the speaker (speaker) is consistent with the identification of the speaker in the registered voiceprint part, memoryList-1, memoryList-2,..., memoryList-n are memory content lists of different speakers. For example, Xiao Ming's conversation memory content is recorded as memory List-1 = [memory-1-1, memory-1-2, memory-1-3,...], and Xiao Ming's mother's conversation memory content is recorded as memory List-2 = [memory-2-1, memory-2-2, memory-2-3,...]. The memory content list can record all conversation contents in text form. The embodiment of the present application does not specifically limit the data structure of the memory content.

[0095] The distributed storage method of the above-mentioned database can save local storage space of electronic devices, use the server corresponding to the voice assistant to store a large amount of memory content, meet the large-scale storage of user memory content data, improve the flexibility of data storage, meet the storage of large amounts of memory content of multiple users, and improve data reading and writing performance.

[0096] Alternatively, the voice assistant stores all the voiceprint feature vectors and memory contents shown in Table 1 in the cloud, for example, in a server corresponding to the voice assistant for cloud storage. This storage method can utilize the larger storage space in the cloud to increase the storage capacity of data.

[0097] After the user's identity is identified based on the voiceprint feature through the process of steps 310 to 330, a multi-user memory mechanism needs to be called.

[0098] Step 340: Determine a first reply text for the voice data of the first user based on the memory content associated with the identity identifier of the first user, and present the first reply text to the first user.

[0099] In one possible implementation, the memory content associated with the identity identifier of the first user includes one or more information of the first user's name, personalized preferences, and event elements associated with the first user, and determining the first reply text for the first user's voice data based on the memory content associated with the identity identifier of the first user includes: extracting target information that can respond to the first user's voice data from one or more information of the first user's name, personalized preferences, and event elements associated with the first user; and splicing the target information to generate the first reply text.

[0100] Optionally, the first reply text contains the name of the first user.

[0101] In another possible implementation, the memory content associated with the identity identification of the first user also includes the role information of the first user, and the method further includes: determining a dialogue mode that matches the first user based on the role information of the first user, the dialogue mode including one or more of language techniques, language types, language styles, dialogue intonation, and dialogue tone; and presenting the first reply text to the first user includes: presenting the first reply text to the first user based on the dialogue mode that matches the first user.

[0102] Optionally, the role information of the first user may include one or more of the identity, gender, occupation, age, language, and nationality of the first user.

[0103] Specifically, after the ASR module of the voice assistant converts the first user's audio into text, it reads the corresponding memory content of the first user based on the result of the first user's voiceprint feature recognition. Then, based on the first user's memory content, it determines the text prompt of the answer; then, based on the input of the Large Language Model (LLM), the LLM outputs two results:

[0104] 1) Determine whether the current text query contains memory content. If so, write it into the preset database.

[0105] Optionally, "personalized memory content" may include the name, role information, personalized preferences, and one or more information of event elements associated with the first user, wherein the role information of the first user may include one or more of the identity, gender, occupation, age, language, and nationality of the first user, which is not limited to this embodiment of the present application.

[0106] 2) Outputting a suitable reply text according to the text prompt, wherein the content of the reply text is consistent with the historical memory of the first user and the personalized preferences of the current user.

[0107] Finally, the text-to-speech (TTS) module converts the reply text into audio and outputs it to the user.

[0108] It should be understood that with the rapid development of the LLM model, the dialogue generation model has also greatly improved the effect of the dialogue. Different from the traditional dialogue system framework that adopts a pipeline architecture, the dialogue generation model of the embodiment of the present application can directly realize end-to-end reply text, eliminating the processing of independent modules such as natural language understanding (NLU), dialogue management (DM) and natural language generation (NLG). Since the LLM model has a powerful in-context learning capability, in the memory scenario, it only needs to care whether the content of the constructed prompt is sufficient and whether the reply text (response) output by the LLM meets the requirements.

[0109] In another possible implementation, after obtaining the voice data of the first user, the method 300 includes: parsing the voice data of the first user to obtain the intention corresponding to the voice data of the first user; and determining the first reply text for the voice data of the first user based on the memory content associated with the identity identifier of the first user, including: determining the first reply text based on the memory content associated with the identity identifier of the first user and the intention corresponding to the voice data of the first user.

[0110] It should be understood that in this implementation, when a user is conversing with a voice assistant robot (bot), the user can also combine the existing dialogue scheme to parse the acquired voice data of the first user to obtain the intention instruction corresponding to the voice data of the first user, further combine the intention instruction with the memory content associated with the identity identification of the first user, and finally obtain the first reply text, and present the first reply text to the first user.

[0111] Figure 4 is a schematic block diagram of a method for voice interaction provided in an embodiment of the present application. For example, Figure 4 The method shown can correspond to the aforementioned Figure 3 The implementation process, such as Figure 4 As shown, the method 400 includes:

[0112] Step 1: Obtain user voice and input the user voice into the voiceprint representation model;

[0113] Step 2: Output the user's voiceprint features;

[0114] Step 3, performing voiceprint feature comparison based on the output voiceprint features;

[0115] Step 4, obtaining a voiceprint recognition result; when the voiceprint recognition result determines that the current user is a registered user in a preset database, the content contained in the user's voice is stored in the preset database.

[0116] Step 5, when the voiceprint recognition result determines that the current user is not a registered user in the preset database, register the user's voiceprint to the preset database, and execute step 9 to store the content of the current user's voice as user memory content associated with the user's voiceprint feature.

[0117] In addition, you can also execute the following steps while in step 1:

[0118] Step 6, performing speech recognition on the user's speech based on the user's speech to obtain a speech recognition text;

[0119] Step 7, when the voiceprint recognition result determines that the current user is a registered user in the preset database, read the memory content from the preset database according to the user identity;

[0120] Step 8, determining prompt information according to the user speech recognition text and the memory content associated with the user's identity identifier;

[0121] Step 10, generating a reply text of the user's voice data;

[0122] Step 11, performing speech synthesis based on TTS;

[0123] Step 12, presenting the reply text to the user, i.e. replying to the user.

[0124] The above process introduces the registration and information storage process of the "preset database" shown in Table 1. After the "preset database" including the voiceprint features and memory contents of multiple users is generated, when the user wakes up the voice assistant of the electronic device and has a conversation, the voice assistant can first obtain the user's audio, extract the voiceprint features of the current user from the audio, and judge the identity of the current user according to the extracted voiceprint features according to the method in S330 above, that is, identify the user's identity.

[0125] In S330, the intelligent assistant can extract the voiceprint features of the first user based on the voice data of the first user; match the voiceprint features of the first user with the voiceprint features of multiple registered users in the preset database to determine the identity of the first user.

[0126] The above process is called "the process of voiceprint identification of users". It should be understood that "voiceprint identification" refers to identifying the identity of the speaker through sound signals. The main working forms of "voiceprint identification" include 1:1 verification and 1:N retrieval. Among them, "1:1 verification" can be used to verify whether the voice of the current speaker belongs to a specific speaker. For example, based on the audio information of user 1 speaking, it is determined whether user 1 is the owner of the electronic device: Xiao Ming.

[0127] Optionally, the "1:1 verification" process can be widely used in the wake-up and security authentication of smart assistants. For example, when an electronic device or other electronic device is awakened in the screen-off state, it detects whether it is the owner of the electronic device who wakes it up, that is, determines whether the electronic device needs to respond, wake up, light up the screen and unlock it. The embodiment of the present application does not elaborate on this process.

[0128] "1:N search" can be used to search which of the N specific users the current speaker's voice belongs to. In the embodiment of the present application, the identity and / or identity identifier of the current user can be determined based on the voiceprint features by extracting the voiceprint features in the voice of the first user, thereby realizing the identification of different users in a scenario where multiple people share a voice assistant.

[0129] In one possible implementation, voiceprint recognition can use a deep learning model.

[0130] When the degree of match between the voiceprint features of the first user and the voiceprint features of a target user among multiple registered users is greater than or equal to a preset value, it is determined that the first user is the target user registered in the preset database; or, when the degree of match between the voiceprint features of the first user and the voiceprint features of any one of the multiple registered users is less than the preset value, it is determined that the first user is not a registered user in the preset database.

[0131] It should be understood that the voiceprint recognition module can extract voiceprint features from the audio of the speaker (eg, the first user), and then compare them with the registered voiceprints to determine the identity of the speaker.

[0132] Figure 5 FIG. 1 is a schematic flow chart of a voiceprint recognition process provided in an embodiment of the present application. Figure 5 As shown, the voiceprint recognition process 500 includes the following steps:

[0133] S501, obtaining the voice data of the first user; accordingly, S501 corresponds to Figure 4 Follow step 1 in the

[0134] S502, inputting the voice data of the first user into the deep neural network EPACA-TDNN for processing to obtain the voiceprint feature vector of the first user; accordingly, S502 corresponds to Figure 4 Follow step 2 in the

[0135] S503, reading the registered voiceprint;

[0136] S504, calculating the cosine distance according to the voiceprint feature vector and the registered voiceprint of the first user, and taking the target user with the highest probability value as a candidate;

[0137] S505: Determine whether the highest probability value is greater than a preset threshold.

[0138] S506: When the highest probability value is less than or equal to the preset threshold, determine that the first user is not the candidate user registered in the preset database, register the first user as a new user, and write the voiceprint feature of the first user. Figure 4 Follow the steps 3, 4, and 5 in the procedure.

[0139] Specifically, when it is determined that the first user is not the candidate user registered in the preset database, a new user registration can be performed, and the voiceprint features of the first user and the content of the first user's voice data are newly stored in the preset database, and the voiceprint features of the first user and the content of the first user's voice data are associated with the identity of the first user.

[0140] S507: When the highest probability value is greater than a preset threshold, it is determined that the first user is the candidate user registered in the preset database, and the voiceprint feature of the registered user is refreshed.

[0141] Specifically, when the highest probability value is greater than a preset threshold, that is, when it can be determined that the first user is the candidate user registered in the preset database, the voiceprint features of the first user currently extracted and the voiceprint features of the registered target user can be fitted, and the voiceprint features of the target user stored in the preset database can be updated with the result of the fitting process.

[0142] In a possible implementation, in the process of determining in real time whether the current first user is a user who has registered a voiceprint, a solution for voiceprint comparison based on cosine distance may be adopted.

[0143] Figure 6 It is a schematic diagram of a voiceprint comparison solution provided in an embodiment of the present application.

[0144] For example, Figure 6 As shown, the process includes: step 1, extracting the voiceprint features of the current user in real time; step 2, calculating the cosine distance between each registered voiceprint and the voiceprint feature vector of the current user.

[0145] Optionally, under a certain user account, for the current real-time user audio, Mel-scale Frequency Cepstral Coefficients (MFCC) processing may be performed on the user audio first.

[0146] It should be understood that the MFCC processing process may include audio preprocessing, fast Fourier transform, filter bank, logarithmic operation, discrete cosine transform, dynamic feature extraction and other steps. After a series of processing, the user audio is converted into MFCC features, and the MFCC features are input into the deep neural network EPACA-TDNN model to obtain the current voiceprint feature vector embedding, denoted as e_x.

[0147] Figure 7 This is a schematic diagram of the processing process of the deep neural network EPACA-TDNN.

[0148] It should be understood that after the x-vector is proposed, it can be used to describe the embeddings feature vector extracted from the TDNN network. Figure 7 As shown, the audio (user's voice data) is input, and the MFCC features converted from the user's audio are input into the deep neural network. Figure 7 The multi-layer convolutional neural network shown in Figure (a) is preprocessed and finally outputs a multi-dimensional vector of fixed length, namely the "voiceprint feature vector".

[0149] Figure 7 Figure (b) shows the specific processing process of the second step "SE-Res2Block" of the multi-layer convolutional neural network. For the sake of simplicity, the processing process of the deep neural network EPACA-TDNN is not repeated in detail in this embodiment of the application.

[0150] It should be understood that the multi-layer convolutional network's processing of vectors, by introducing residual connections and attention mechanisms, greatly reduces the convergence problem of training and improves the recognition effect.

[0151] Specifically, the deep neural network EPACA-TDNN model can be trained and tested on the open source VoxCeleb dataset. The training data can contain 5994 different speakers, and the ACC on the test set reaches 99%. The audio quality of the open source VoxCeleb dataset is relatively high. Although the deep neural network EPACA-TDNN model introduces data enhancement schemes such as noise and reverberation during the training process, the voice assistant faces the real user environment, and the online audio distribution is still different from the open source dataset. Therefore, it is necessary to fine-tune the model for the voice assistant scenario. By collecting online audio data and using manual annotation to construct a fine-tuning dataset, during the annotation process, it can be assumed that the audio under different accounts must belong to different speakers. The annotator only needs to determine whether the audio under the same account belongs to different speakers. The final data format is (audio 1, audio 2, label), and the label indicates whether audio 1 and audio 2 belong to the same speaker. Under the same account, most of the audio belongs to the same speaker, so it is relatively easy to construct positive samples. For the problem of scarce negative samples, a large number of negative samples can be constructed by mixing audio data between different accounts. For the sake of simplicity, the training and labeling process of the deep neural network EPACA-TDNN model is not described in detail in the embodiment of the present application.

[0152] It should be understood that the voiceprint recognition module can extract voiceprint features from the audio of the speaker (eg, the first user), and then compare them with the registered voiceprints to determine the identity of the speaker.

[0153] Specifically, after the voiceprint feature vector embedding(e_x) of the current user is obtained, all registered voiceprints E=[embedding-1, embedding-2, ..., embedding-n] under the user account are obtained from the preset database.

[0154] Next, follow Figure 6 In step 2, the cosine distance between each registered voiceprint and the current user's voiceprint feature vector embedding(e_x) is calculated to obtain a cosine distance vector S = [s1, s2, ..., sn]. Where s1, s2, ..., sn represent the distance between the current speaker's voiceprint feature vector embedding(e_x) and the n registered speakers' voiceprint feature vectors.

[0155] For ease of calculation, a softmax operation is performed on the cosine distance vector S, and the distance is normalized to [0,1] to obtain a vector P = [p1, p2, ..., pn], where p1, p2, ..., pn can represent the probability distribution of the current speaker's voiceprint feature vector embedding (e_x) belonging to the n registered speakers.

[0156] Finally, the speaker m with the highest probability among p1, p2, …, pn is selected as a candidate, m∈[1,n].

[0157] In the probability determination process, a preset threshold pt is set. If the probability pm of the speaker m with the highest probability is greater than or equal to pt, the current speaker is considered to be a registered user m. If the probability pm of the speaker m with the highest probability is less than pt, the current speaker is considered to be a new user, that is, an unregistered user.

[0158] As described in step S507, in order to enable the voiceprint recognition module to better fit the real environment, an adaptive mechanism for registering voiceprints is introduced.

[0159] In another possible implementation, when the first user is the target user registered in the preset database, the voiceprint features of the first user currently extracted and the voiceprint features of the target user are fitted, and the voiceprint features of the target user stored in the preset database are updated with the results of the fitting.

[0160] Alternatively, when the first user is not a registered user in the preset database, the voiceprint features of the first user and the content of the first user's voice data are newly stored in the preset database, and the voiceprint features of the first user and the content of the first user's voice data are associated with the identity of the first user.

[0161] Exemplarily, if the current speaking user is determined to be a new user, a new user is registered in the voiceprint part of the preset database, and {speaker-n+1:e_x} is written. If the current speaking user is determined to be a registered user m, that is, speaker-m, the registered voiceprint in the preset database is refreshed according to the following formula (1).

[0162] embedding_m = α * embedding_m + (1-α) * e_x formula (1)

[0163] Among them, α is the smoothing coefficient, and its value range is [0,1].

[0164] It should be understood that when α is larger, it proves that the user who is currently speaking is more likely to maintain the historical state, that is, the registered voiceprint features in the preset database are more accurate.

[0165] It should be noted that since voiceprint recognition is judged in each round of dialogue, and the use of memory content is only triggered in a few rounds of dialogue, the number of read and write requests for registering voiceprints in the preset database is much higher than the number of requests for storing the user's memory content. In order to improve the read and write performance, the distributed storage solution introduced above is adopted to store the user's registered voiceprint and memory content separately, and provide the interface of the preset database to the outside, including:

[0166] 1. Voiceprint feature reading interface, given the user account primary key user-id, read and return the registered voiceprints of all speakers under the user account.

[0167] For example, a voiceprint feature reading interface is provided by means of the program language "voiceprint_read(user-id=string)->list(speaker:embedding)" to read the registered voiceprints of all speakers under the user account.

[0168] 2. Voiceprint feature writing interface. Given the user account primary key user-id, speaker speaker-id, and voiceprint feature vector embedding, when you need to write the speaker's voiceprint feature, you can use the following example to write it and return the status of whether the writing is successful.

[0169] For example, a voiceprint feature writing interface is provided through the programming language "voiceprint_write(user_id=string, speaker_id=string, embedding=tensor)->boolean(0 / 1)" to write the speaker's voiceprint features into a preset database.

[0170] 3. The reading interface of the memory content, given the user account primary key user-id and the speaker speaker-id, returns the memory content of the speaker.

[0171] For example, a memory content reading interface is provided by means of a program language "memoryList_read(user_id=string, speaker_id=string)->list(memory)" to obtain the memory content of the speaker from a preset database.

[0172] 4. The memory content writing interface, given the user account primary key user-id, speaker speaker-id, writes the memory content.

[0173] For example, a memory content write interface is provided through the program language "memoryList_write(user_id=string, speaker_id=string, memoryList=list)->boolean(0 / 1)" to write the memory content associated with the speaker into the preset database and return the status of whether the writing is successful.

[0174] In another possible implementation, the method 300 also includes: obtaining voice data of a second user, determining an identity identifier of the second user based on the voice data of the second user, and the voice data of the second user is associated with the first user; when the second user is a mutually trusted user associated with the first user, obtaining memory content associated with the identity identifier of the first user from the preset database based on the identity identifier of the first user; determining a second reply text for the voice data of the second user based on the memory content associated with the identity identifier of the first user, and presenting the second reply text to the second user.

[0175] Optionally, in the embodiment of the present application, “the voice data of the second user is associated with the first user” may have any one or more of the following possible meanings, for example:

[0176] 1) The identity of the second user is associated with the identity of the first user. For example, the first user and the second user are trusted users in a mutual trust group, such as Xiao Ming and Xiao Ming's mother, Xiao Ming's father, etc. in the family scenario listed in Table 1;

[0177] 2) The voice data of the second user includes content associated with the "first user". Exemplarily, as listed in Table 1, the voice data of Xiao Ming's mother (the second user) is "Does Xiao Ming (the first user) have class today?" The voice data of the second user includes content related to "Xiao Ming (the first user)", which will not be repeated here. Optionally, the method 300 also includes: obtaining the role relationship between the first user and the second user, and determining the title of the second user based on the role relationship between the first user and the second user; and determining the second reply text for the voice data of the second user based on the memory content associated with the identity identifier of the first user, including: determining the second reply text for the voice data of the second user based on the memory content associated with the title of the second user and the identity identifier of the first user.

[0178] For the above scenario, for multiple registered users in the same preset database, the embodiment of the present application can identify each user based on their voice content. When the second user's voice content is associated with other registered users (such as the first user), the identity of the associated registered user (such as the first user) can be obtained, and the memory content associated with the identity of the first user in the preset database can be further obtained based on the identity of the associated registered user (such as the first user), and the reply text for the second user's voice data can be determined. This method can obtain more associated information content based on the second user's voice, so that the voice dialogue has a better interactive effect, improve the intelligence and flexibility of the voice assistant, and thus improve the user's voice interaction experience.

[0179] For example, in order to allow LLM to fully understand the context and historical memory information of the current dialogue turn, the prompt information can be constructed through the following examples:

[0180] 1. Describe the role of LLM: "You are an intelligent assistant that has the ability to communicate with users and help them remember information";

[0181] 2. Describe the user role and memory information: "The user you are talking to now is {speaker}. The following is his memory information: At {time}, {speaker} said {text}; At {time}, {speaker} said {text}; ...; At {time}, {speaker} said {text}. The current time is {time}";

[0182] Among them, {time} is the date and time slot to be filled in, {speaker} is the name of the current speaker, and {text} is the conversation content that the speaker once said and was recorded by the intelligent assistant.

[0183] 3. Describe the current conversation context: "The context of the current conversation is [user:{query_1}, bot:{response_1}, user:{query_2}, bot:{response_2}, user:{query_t}]";

[0184] {query} and {response} are the conversation contents between the user and the intelligent assistant respectively.

[0185] 4. Description instruction: "Output according to this information in the following format: {'have_memory':boolean,'response':string}. If the current conversation contains the user's personalized information, have_memory output is True, otherwise it outputs False; response is the content you want to answer to the user. Please answer"

[0186] The above text is concatenated into prompt as the input of LLM, and LLM gives the output as required, for example:

[0187] Prompt = 'You are an intelligent assistant that can talk to users and help them remember information. The user you are talking to now is Xiao Ming. The following is his memory information: On March 1, Xiao Ming said "I put the comic book on the second layer of the drawer"; on March 5, Xiao Ming said "Remember that there is a music class at 2 pm the day after tomorrow". The current time is 15:00 on March 6. The context of the current conversation is:

[0188] Xiao Ming: Hello;

[0189] Voice assistant robot (bot): Hello Xiao Ming;

[0190] Xiao Ming: When is my music class? Remind me to bring my sheet music half an hour in advance.

[0191] The voice assistant can output according to this information in the following format:

[0192] {'have_memory':boolean,'response':string}. If the current conversation contains the user's personalized information, have_memory output is True, otherwise it outputs False; response is the content you want to answer to the user. Please answer:

[0193] Response = '{'have_memory':True,'response':'The music class is at 2 pm tomorrow. I will remind you to bring the music at 1:30 pm tomorrow.'}'.

[0194] Figure 8 This is a schematic diagram of an interface for human-computer interaction between a user and a voice assistant provided in an embodiment of the present application.

[0195] For example, based on the solution provided in the above embodiments of the present application, the user may have the following conversation with the voice assistant of the electronic device: Figure 8 The dialogue shown in the interface 801 of FIG. (a) is as follows:

[0196] User 1: Xiaobu Xiaobu;

[0197] Voice assistant robot (bot): I am here;

[0198] User 1: Help me remember that I have a music class at 2pm;

[0199] Voice assistant robot (bot): OK, Xiao Ming, I will remind you 30 minutes in advance.

[0200] Through the above dialogue, the voice assistant can obtain the user's voiceprint features based on the voice when the user first speaks (for example, "Xiaobu Xiaobu"), and judge that the current speaker is "Xiaoming", and Xiaoming is a registered user under the current user account, that is, the preset database stores Xiaoming's user identification, memory content, etc. Therefore, the voice assistant can obtain Xiaoming's user identification, memory content, etc. stored in the preset database, and may find that Xiaoming often prepares for class 30 minutes in advance in the stored historical memory, that is, Xiaoming's historical habits can be obtained. Then, a personalized reply will be provided to "Xiaoming", such as "OK, Xiaoming, I will remind you 30 minutes in advance". In the reply text, it can be said that the current user identified is Xiaoming, and it can also be combined with Xiaoming's historical class habits to actively customize a personalized reminder service for Xiaoming. From the user's perspective, this dialogue method behaves more like a human friend, can adapt to dialogues under different roles, and improve the user experience.

[0201] For example, based on the solution provided in the above embodiments of the present application, the user may have the following conversation with the voice assistant of the electronic device: Figure 8 The dialogue shown in the interface 802 of FIG. (b) is as follows:

[0202] User 2: Xiaobu Xiaobu;

[0203] bot: I am...Xiaoming's mother;

[0204] User 2: Video conference at 9 am tomorrow;

[0205] bot: OK, I'll remind you;

[0206] User 2: Get up at 7 o'clock tomorrow morning to make breakfast for Xiao Ming;

[0207] Bot: OK, the alarm is calling you, Xiao Ming wants to eat a sandwich;

[0208] bot: Also, the 16th is your birthday, I wish you a happy birthday in advance.

[0209] Through the above dialogue, the voice assistant can obtain the user's voiceprint features based on the voice when the user utters a voice for the first time (for example, "Xiaobu Xiaobu"), and perform user recognition based on the current user's voiceprint features, and determine that the current speaker is "Xiao Ming's mother", and that Xiao Ming's mother is a registered user under the current user account, that is, the preset database stores Xiao Ming's mother's user identification, memory content, etc.

[0210] In addition, the voice assistant can determine that Xiao Ming's mother and Xiao Ming are mutually trusted users in the same family scenario. At this time, the voice assistant can also read Xiao Ming's memory content stored in the preset database, such as obtaining Xiao Ming's words "My mother's birthday is on the 16th" and Xiao Ming's query "delicious sandwiches" listed in Table 1. Then, after obtaining Xiao Ming's mother's words "Get up at 7 o'clock tomorrow morning to make breakfast for Xiao Ming" in the conversation, it can be determined that Xiao Ming likes to eat sandwiches based on the information of "delicious sandwiches" that Xiao Ming has queried, and the memory information is provided to Xiao Ming's mother.

[0211] Furthermore, the voice assistant can also obtain the part of the memory content associated with Xiao Ming that is related to his mother based on the memory content associated with Xiao Ming stored in the preset database. For example, the 16th is Xiao Ming’s mother’s birthday. The voice assistant can extract this information and add prompts such as "Happy birthday to you" in the reply message, which can bring users a warmer experience and deepen the emotions between family members to a certain extent.

[0212] In summary, during the conversation between the user and the voice assistant, the above method can identify the user's identity based on the voice emitted by the user. When the user is identified as a registered user in the preset database, the present application can obtain the memory content associated with the result of the user's identity identification in the preset database, and based on the associated memory content, determine the reply text of the voice emitted by the user and present it to the user. This method can determine the voice reply text or content based on the memory content associated with the user, so that the voice conversation has a better interactive effect, improves the intelligence and flexibility of the voice assistant, and thus improves the user's voice interaction experience.

[0213] The embodiment of the present application aims at the scenario where multiple people share a voice assistant. By acquiring the user's voice data and registering the user according to the voiceprint features in the voice data, the personalized content in the voice data such as the user's name, identity, personalized preferences, personality, personality, schedule, behavioral habits, etc. is stored as memory content associated with the user to generate a preset database. In other words, the preset database can store the voiceprint features and memory content of different users, and provide external reading and writing interfaces for the voiceprint features and memory content.

[0214] In addition, during real-time user conversations, the voiceprint recognition module can perform voiceprint recognition based on the user's input and output voice data, and convert the input voice data into the user's voiceprint features through the voiceprint feature model. The voiceprint comparison module reads the existing registered voiceprints from the preset database, compares the current voiceprint with the registered voiceprint, and determines which historical user or new user the current voiceprint belongs to.

[0215] When it is determined that the current user is a historical user, the memory content of the historical user is directly read, and appropriate reply content is output, keeping the reply content consistent with the identity, role, personalized preferences, personality style, etc. related to the user's memory content. Finally, the TTS module converts the text reply content into audio and outputs it to the user.

[0216] Furthermore, when the current user is determined to be a historical user, the voiceprint features of the current user and the stored voiceprint features of the historical user can be fitted, and the results of the fitting process can be used to update the voiceprint features of the historical user stored in the preset database, thereby improving the stability of the voiceprint features of the historical user and thereby improving the accuracy of speech recognition.

[0217] When it is determined that the current user is not a historical user but a new user, the new user is registered, and the voiceprint features of the user and the memory content associated with the new user are stored.

[0218] In a possible implementation, in addition to the conversation content described in the above embodiment, it can be more in line with the identity, role, personality, hobbies, personality, behavioral habits, etc. of the current speaker. When the memory content in the preset database also includes the user's favorite desktop style, such as desktop wallpaper, application layout, etc., then Figure 8 The pop-up windows in Figures (a) and (b) may also match different styles according to the desktop style changed by the user, and this embodiment of the present application does not limit this.

[0219] Fig. 9 It is a schematic block diagram of another example of a human-computer interaction device 900 provided in an embodiment of the present application.

[0220] like Fig. 9 As shown, the device 900 can be the electronic device described in the above embodiments, such as Fig. 9 As shown, the device 900 may include: an acquisition unit 910 , a processing unit 920 and a response unit 930 .

[0221] An acquiring unit 910 is configured to acquire voice data of a first user;

[0222] The processing unit 920 is used to determine the identity of the first user according to the voice data of the first user; when the first user is a registered user in a preset database, it is used to obtain the memory content associated with the identity of the first user from the preset database according to the identity of the first user, and the memory content associated with the identity of the first user is the content extracted by the voice assistant based on the acquired voice data sent by the first user;

[0223] The response unit 930 is used to determine a first reply text for the voice data of the first user based on the memory content associated with the identity identifier of the first user, and play the first reply text to the first user.

[0224] In a possible implementation, the memory content associated with the identity identifier of the first user includes one or more information of the name of the first user, the historical behavior habits of the first user, and the event elements to be executed by the first user. The acquisition unit 910 is specifically used to: extract target information that can respond to the voice data of the first user from one or more information of the name of the first user, the historical behavior habits of the first user, and the event elements to be executed by the first user;

[0225] The processing unit 920 is specifically used to: splice the target information to generate the first reply text.

[0226] Optionally, the first reply text contains the name of the first user.

[0227] In one possible implementation, the memory content associated with the identity identification of the first user also includes the role information of the first user, and the processing unit 920 is further used to: determine a dialogue mode that matches the first user based on the role information of the first user, the dialogue mode including one or more of language skills, language type, language style, dialogue intonation, and dialogue tone; and the response unit 930 is further used to: play the first reply text to the first user based on the dialogue mode that matches the first user.

[0228] Optionally, the role information of the first user includes one or more of the identity, gender, occupation, age, language, and nationality of the first user.

[0229] In another possible implementation, the preset database also stores voiceprint features of multiple registered users, and the processing unit 920 is specifically used to: extract the voiceprint features of the first user based on the voice data of the first user; match the voiceprint features of the first user with the voiceprint features of multiple registered users in the preset database to determine the identity of the first user.

[0230] In another possible implementation, the processing unit 920 is specifically configured to: when a matching degree between the voiceprint feature of the first user and a voiceprint feature of a target user among multiple registered users is greater than or equal to a preset value, determine that the first user is the target user registered in the preset database;

[0231] Alternatively, when the matching degree between the voiceprint feature of the first user and the voiceprint feature of any one of the multiple registered users is less than the preset value, it is determined that the first user is not a registered user in the preset database.

[0232] In another possible implementation, when the first user is the target user registered in the preset database, the processing unit 920 may further perform fitting processing on the currently extracted voiceprint feature of the first user and the voiceprint feature of the target user, and update the voiceprint feature of the target user stored in the preset database with the result of the fitting processing;

[0233] Alternatively, when the first user is not a registered user in the preset database, the voiceprint features of the first user and the content of the first user's voice data are newly stored in the preset database, and the voiceprint features of the first user and the content of the first user's voice data are associated with the identity of the first user.

[0234] In a possible implementation, the acquisition unit 910 is further configured to: acquire voice data of a second user, and determine an identity of the second user according to the voice data of the second user;

[0235] When the second user is a mutually trusted user associated with the first user, the processing unit 920 may further be configured to:

[0236] According to the identity identifier of the first user, memory content associated with the identity identifier of the first user is obtained from the preset database; according to the memory content associated with the identity identifier of the first user, a second reply text for the voice data of the second user is determined, and the second reply text is played to the first user.

[0237] In a possible implementation, the acquisition unit 910 is further configured to: acquire a role relationship between the first user and the second user, and determine a title of the second user according to the role relationship between the first user and the second user;

[0238] Furthermore, the processing unit 920 may also be configured to determine a second reply text for the voice data of the second user based on the memory content associated with the title of the second user and the identity identifier of the first user.

[0239] Optionally, the preset database stores identity information of multiple registered users, voiceprint features of each registered user, and memory content associated with each registered user. The processing unit 920 can also be used to: store the voiceprint features of each registered user in the local storage space of the electronic device, and store the memory content associated with each registered user in the server of the voice assistant.

[0240] In another possible implementation, after acquiring the voice data of the first user, the acquisition unit 910 is further used to: parse the voice data of the first user to obtain the intention corresponding to the voice data of the first user; and determine the first reply text based on the memory content associated with the identity identifier of the first user and the intention corresponding to the voice data of the first user.

[0241] Specifically, the specific process of each unit executing the above corresponding steps has been described in detail in the above embodiments, and for the sake of brevity, it will not be repeated here.

[0242] The voice interaction method provided in the embodiments of the present application can be applied to electronic devices, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), and other electronic devices. The embodiments of the present application do not impose any restrictions on the specific types of electronic devices.

[0243] Fig.10 It is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application.

[0244] It should be understood that in the embodiment of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware. It should be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100.

[0245] For example, Fig.10 As shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a button 130, a camera 140, a display screen 150, a sensor module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a mobile communication module 180, a wireless communication module 190, an antenna 1, an antenna 2, etc.

[0246] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0247] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0248] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0249] In the embodiment of the present application, the electronic device 100 can be an electronic device equipped with a "voice assistant", which has the function of obtaining user voice data, processing the user's voice data, and then responding, such as replying to the user's voice. Specifically, the processor 110 of the electronic device 100 stores a computer program or instruction corresponding to the method for voice interaction provided in the present application.

[0250] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 180, the wireless communication module 190, the modem processor and the baseband processor.

[0251] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands.

[0252] The mobile communication module 180 can provide solutions for wireless communications including 2G / 3G / 4G / 5G / 6G, etc., applied to the electronic device 100. The mobile communication module 180 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc.

[0253] The wireless communication module 190 can provide wireless communication solutions for application in the electronic device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc.

[0254] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 180, and antenna 2 is coupled to wireless communication module 190, so that electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0255] The electronic device 100 implements the display function through a GPU, a display screen 150, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 150 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0256] The display screen 150 is used to display images, videos, etc. The display screen 150 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), MiniLED, MicroLED, Micro-OLED, quantum dot light-emitting diodes (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 150, where N is a positive integer greater than 1.

[0257] In an embodiment of the present application, the display screen 150 of the electronic device can receive user operations and display corresponding interfaces on the display screen 150 of the electronic device, such as a setting interface, a dialogue interface between a voice assistant and the user, etc.

[0258] The electronic device 100 can realize the shooting function through the ISP, the camera 140, the video codec, the GPU, the display screen 150 and the application processor.

[0259] The camera 140 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. In some embodiments, the electronic device 100 may include 1 or N cameras 140, where N is a positive integer greater than 1.

[0260] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0261] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0262] In the embodiment of the present application, the electronic device 100 can store the contents of the preset database shown in Table 1 through the internal memory 121 or an external memory card connected to the external memory interface 120, etc.

[0263] Exemplarily, the voice assistant of the electronic device 100 can store the voiceprint features of each registered user in the local storage space of the electronic device 100 (such as the internal memory 121), and store the memory content associated with each registered user in the server of the voice assistant to realize the distributed storage of the content of the preset database.

[0264] The sensor module 160 may include a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc. Exemplarily, in the embodiment of the present application, the touch sensor may detect operations such as touch and click of the user, generate an operation event, and transmit the operation event to the processor 110, which determines that the operation event includes relevant parameters such as the coordinates of the touch point and the touch state, and responds according to the operation event.

[0265] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0266] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0267] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0268] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the human ear.

[0269] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to microphone 170C to input the sound signal into microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0270] The earphone jack 170D is used to connect a wired earphone.

[0271] In an embodiment of the present application, the electronic device 100 can obtain the user's voice signal through the microphone 170C, and input the voice signal for voice recognition and other processing; when the processor 110 determines the reply text for the user's voice signal based on the memory content associated with the user's identity identification, the reply text can be played to the user through the speaker 170A, which will not be repeated here.

[0272] In addition, the software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. The embodiments of the present application do not limit the software architecture of the electronic device.

[0273] It should be noted that, in the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0274] It should be understood that the methods, situations, categories and divisions of the embodiments in the present application are only for the convenience of description and should not constitute special limitations. The features of various methods, categories, situations and embodiments can be combined without contradiction.

[0275] It should also be understood that the terms "first", "second" and "third" in the embodiments of the present application are only for distinction and should not constitute any limitation to the present application. For example, the "first user", "second user" and the like in the embodiments of the present application represent different users.

[0276] It should also be understood that in the various embodiments of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0277] It should also be noted that in the embodiments of the present application, "preset", "fixed value", etc. can be implemented by pre-saving corresponding codes, tables or other methods that can be used to indicate relevant information in electronic devices (such as transmitting end devices and receiving end devices, etc.), and the present application does not limit the specific implementation method. For example, in the embodiments of the present application, "preset threshold" etc.

[0278] It is understandable that, in order to realize the above functions, the electronic device includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of the present application.

[0279] In this embodiment, the electronic device can be divided into functional modules according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0280] In the case of dividing each functional module according to each function, a possible composition diagram of the electronic device involved in the above embodiment, the electronic device may include: a display unit, a detection unit and a processing unit. The display unit, the detection unit and the processing unit cooperate with each other to support the electronic device to perform the above steps, etc., and / or other processes of the technology described in this article.

[0281] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0282] The electronic device provided in this embodiment is used to execute the above-mentioned video playback method, and thus can achieve the same effect as the above-mentioned implementation method.

[0283] In the case of using integrated units, the electronic device may include a processing module, a storage module and a communication module. Among them, the processing module can be used to control and manage the actions of the electronic device, for example, it can be used to support the electronic device to execute the steps performed by the above-mentioned display unit, detection unit and processing unit. The storage module can be used to support the electronic device to execute stored program codes and data, etc. The communication module can be used to support the communication between the electronic device and other devices.

[0284] Among them, the processing module can be a processor or a controller. It can implement or execute various exemplary logic boxes, modules and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module can be a memory. The communication module can specifically be a device that interacts with other electronic devices, such as a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, etc.

[0285] In one embodiment, when the processing module is a processor and the storage module is a memory, the electronic device involved in this embodiment may be a Figure 8 Device of the structure shown.

[0286] This embodiment also provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the voice interaction method in the above-mentioned embodiment.

[0287] This embodiment also provides a computer program product. When the computer program product runs on a computer, it enables the computer to execute the above-mentioned related steps to implement the voice interaction method in the above-mentioned embodiment.

[0288] In addition, an embodiment of the present application also provides a device, which may specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor may execute the computer-executable instructions stored in the memory so that the chip executes the voice interaction method in the above-mentioned method embodiments.

[0289] Among them, the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above and will not be repeated here.

[0290] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0291] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0292] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0293] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0294] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0295] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A voice interaction method, characterized in that: Applied to an electronic device equipped with a voice assistant, the method comprises: Acquire voice data of a first user, and determine an identity of the first user according to the voice data of the first user; When the first user is a registered user in a preset database, memory content associated with the identity identifier of the first user is obtained from the preset database according to the identity identifier of the first user, and the memory content associated with the identity identifier of the first user is content extracted by the voice assistant based on the acquired voice data sent by the first user; A first reply text for the voice data of the first user is determined based on the memory content associated with the identity identifier of the first user, and the first reply text is presented to the first user.

2. The method according to claim 1, characterized in that The memory content associated with the identity identifier of the first user includes one or more information of the name of the first user, personalized preferences, and event elements associated with the first user. The determining, based on the memory content associated with the identity identifier of the first user, a first reply text for the voice data of the first user includes: Extracting target information that can respond to the voice data of the first user from one or more of the name of the first user, personalized preferences, and event elements associated with the first user; The target information is concatenated to generate the first reply text.

3. The method according to claim 1 or 2, characterized in that: The memory content associated with the identity identifier of the first user also includes the role information of the first user, and the method further includes: Determining a conversation method that matches the first user according to the role information of the first user, the conversation method including one or more of language skills, language type, language style, conversation tone, and conversation tone; And, presenting the first reply text to the first user comprises: The first reply text is presented to the first user based on a conversation pattern that matches the first user.

4. The method according to claim 1 or 2, characterized in that: The preset database also stores voiceprint features of multiple registered users, and determining the identity of the first user according to the voice data of the first user includes: Extracting a voiceprint feature of the first user according to the voice data of the first user; The voiceprint feature of the first user is matched with the voiceprint features of multiple registered users in the preset database to determine the identity of the first user.

5. The method according to claim 4, characterized in that The matching process of the voiceprint feature of the first user with the voiceprint features of a plurality of registered users in the preset database includes: When the matching degree between the voiceprint feature of the first user and the voiceprint feature of the target user among the multiple registered users is greater than or equal to a preset value, determining that the first user is the target user registered in the preset database; or, When the matching degree between the voiceprint feature of the first user and the voiceprint feature of any one of the multiple registered users is less than the preset value, it is determined that the first user is not a registered user in the preset database.

6. The method according to claim 5, characterized in that The method further comprises: When the first user is the target user registered in the preset database, fitting the currently extracted voiceprint features of the first user and the voiceprint features of the target user, and updating the voiceprint features of the target user stored in the preset database with the fitting result; or, When the first user is not a registered user in the preset database, the voiceprint features of the first user and the content of the first user's voice data are newly stored in the preset database, and the voiceprint features of the first user and the content of the first user's voice data are associated with the identity of the first user.

7. The method according to claim 1 or 2, characterized in that: The method further comprises: Acquire voice data of a second user, and determine an identity of the second user according to the voice data of the second user, wherein the voice data of the second user is associated with the first user; When the second user is a mutually trusted user associated with the first user, acquiring memory content associated with the identity identifier of the first user from the preset database according to the identity identifier of the first user; Based on the memory content associated with the identity identifier of the first user, a second reply text for the voice data of the second user is determined, and the second reply text is presented to the second user.

8. The method according to claim 7, characterized in that The method further comprises: Acquire a role relationship between the first user and the second user, and determine a title of the second user according to the role relationship between the first user and the second user; And, determining a second reply text for the voice data of the second user according to the memory content associated with the identity identifier of the first user includes: The second reply text for the voice data of the second user is determined according to the memory content associated with the title of the second user and the identity identifier of the first user.

9. The method according to claim 1 or 2, characterized in that: The method further comprises: Obtaining identity information of each user among a plurality of users, a voiceprint feature of each user, and memory content associated with each user; The preset database is generated according to the identity identification information of each user, the voiceprint characteristics of each user and the memory content associated with each user.

10. The method according to claim 9, characterized in that The method further comprises: The voiceprint features of each user in the preset database are stored in the local storage space of the electronic device, and the memory content associated with each user is stored in the server of the voice assistant.

11. The method according to claim 1 or 2, characterized in that: After acquiring the voice data of the first user, the method includes: parsing the voice data of the first user to obtain the intent corresponding to the voice data of the first user; And, determining a first reply text for the voice data of the first user according to the memory content associated with the identity identifier of the first user includes: The first reply text is determined based on the memory content associated with the identity identifier of the first user and the intention corresponding to the voice data of the first user.

12. A voice interaction device, characterized in that: The device is equipped with a voice assistant, and the device includes: An acquiring unit, configured to acquire voice data of a first user; a processing unit, configured to determine an identity of the first user according to the voice data of the first user; When the first user is a registered user in the target preset database, the processing unit is further used to obtain the memory content associated with the identity identifier of the first user from the target preset database according to the identity identifier of the first user, and the memory content associated with the identity identifier of the first user is the content extracted by the voice assistant based on all voice data emitted by the first user; Determining a first reply text for the voice data of the first user according to the memory content associated with the identity identifier of the first user; A response unit is used to present the first reply text to the first user.

13. An electronic device, characterized in that: include: one or more processors; one or more memories; Modules with multiple applications installed; The memory stores one or more programs, and when the one or more programs are executed by the processor, the electronic device executes the method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 11.

15. A computer program product, characterized in that When the computer program product is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 11.

Images