Digital human voice interaction processing method and device, electronic equipment and medium

Abstract

The invention provides a digital human voice interaction processing method and device, electronic equipment and a medium, and relates to the technical field of natural language processing, in particular to the technical field of voice recognition, semantic recognition, intelligent agents and generative search. According to the implementation scheme, in response to received first audio data, first voiceprint data of a digital person is obtained, the first audio data indicates a user to initiate a voice interaction request to the digital person, and the first voiceprint data indicates voiceprint data used by the digital person for currently performing voice broadcasting; in response to the received second audio data, first noise reduction processing is carried out on the second audio data based on the first voiceprint data to obtain third audio data, and the time for receiving the second audio data is after the time for receiving the first audio data; and in response to a first instruction which is sent by the user and indicates to interrupt voice interaction and is determined according to the third audio data, generating a stop instruction and sending the stop instruction to the digital person to control the digital person to stop voice broadcasting.

Description

Technical Field

[0001] The present disclosure relates to the field of natural language processing technology, in particular to the field of speech recognition, semantic recognition, intelligent agents and generative search technology, and specifically to a method, device, electronic device, computer-readable storage medium and computer program product for processing digital human speech interaction. Background Art

[0002] When the user is interacting with the digital human by voice, he or she may issue voice commands or click on the display screen to interrupt the current voice interaction, causing the digital human to stop the voice broadcast and restart the next voice interaction.

[0003] The methods described in this section are not necessarily methods that have been previously conceived or employed. Unless otherwise indicated, it should not be assumed that any method described in this section is considered to be prior art simply because it is included in this section. Similarly, unless otherwise indicated, the issues mentioned in this section should not be considered to have been recognized in any prior art. Summary of the invention

[0004] The present disclosure provides a method, device, electronic device, computer-readable storage medium and computer program product for processing digital human voice interaction.

[0005] According to one aspect of the present disclosure, a method for processing digital human voice interaction is provided, comprising: in response to receiving first audio data, obtaining first voiceprint data of the digital human, wherein the first audio data indicates that a user initiates a voice interaction request to the digital human, and the first voiceprint data indicates the voiceprint data currently used by the digital human for voice broadcasting; in response to receiving second audio data, performing a first noise reduction process on the second audio data based on the first voiceprint data to obtain third audio data, wherein the time when the second audio data is received is after the time when the first audio data is received; and in response to determining, according to the third audio data, that the user issues a first instruction to interrupt the voice interaction, generating a stop instruction and sending it to the digital human to control the digital human to stop voice broadcasting.

[0006] According to another aspect of the present disclosure, a processing device for digital human voice interaction is provided, comprising: an acquisition module, configured to acquire first voiceprint data of the digital human in response to receiving first audio data, wherein the first audio data indicates that the user initiates a voice interaction request to the digital human, and the first voiceprint data indicates the voiceprint data currently used by the digital human for voice broadcasting; a first noise reduction module, configured to perform a first noise reduction process on the second audio data based on the first voiceprint data in response to receiving second audio data to obtain third audio data, wherein the time when the second audio data is received is after the time when the first audio data is received; and a processing module, configured to generate a stop instruction in response to determining, according to the third audio data, that the user issues a first instruction to interrupt the voice interaction, and send the stop instruction to the digital human to control the digital human to stop voice broadcasting.

[0007] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above method.

[0008] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the above method.

[0009] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the above method when executed by a processor.

[0010] According to one or more embodiments of the present disclosure, a method for processing digital human voice interaction is provided. When a user issues a voice interaction request, the voiceprint data currently used by the digital human for voice broadcast is obtained, and during the voice interaction process, noise reduction processing is performed on the digital human's own broadcast voice based on the voiceprint data, thereby making the user's voice in the audio data used for recognition clearer and improving the accuracy of recognizing user interruption commands during the interaction process.

[0011] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0013] Figure 1 is a schematic diagram illustrating an example system in which the various methods described herein may be implemented according to an exemplary embodiment;

[0014] Figure 2 A flowchart of a method for processing digital human voice interaction according to an embodiment of the present disclosure is shown;

[0015] Figure 3 A partial flow chart of another method for processing digital human voice interaction according to an embodiment of the present disclosure is shown;

[0016] Figure 4 A partial flow chart of another method for processing digital human voice interaction according to an embodiment of the present disclosure is shown;

[0017] Figure 5 A partial flow chart of another method for processing digital human voice interaction according to an embodiment of the present disclosure is shown;

[0018] Figure 6 A partial flow chart of another method for processing digital human voice interaction according to an embodiment of the present disclosure is shown;

[0019] Figure 7 A partial flow chart of another method for processing digital human voice interaction according to an embodiment of the present disclosure is shown;

[0020] Figure 8 A structural block diagram of a digital human voice interaction processing device according to an embodiment of the present disclosure is shown; and

[0021] Fig. 9 A structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0022] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0023] In the present disclosure, unless otherwise specified, the use of the terms "first", "second", etc. to describe various elements is not intended to limit the positional relationship, temporal relationship, or importance relationship of these elements, and such terms are only used to distinguish one element from another element. In some examples, the first element and the second element may refer to the same instance of the element, and in some cases, based on the description of the context, they may also refer to different instances.

[0024] The terms used in the description of various examples in this disclosure are only for the purpose of describing specific examples and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element can be one or more. In addition, the term "and / or" used in this disclosure covers any one of the listed items and all possible combinations.

[0025] In the related art, when a user is interacting with a digital human through voice, he or she may issue a voice command to interrupt the current voice interaction so that the digital human stops the voice broadcast and starts to recognize and answer the topic content of the next voice interaction proposed by the user. However, there is a lot of noise in the user's voice audio collected during the voice interaction process, which will have a significant impact on the recognition accuracy of the user's interruption command.

[0026] To solve the above problems, the present disclosure provides a method for processing digital human voice interaction. When a user issues a voice interaction request, the voiceprint data currently used by the digital human for voice broadcast is obtained, and during the voice interaction process, noise reduction processing is performed on the digital human's own broadcast voice based on the voiceprint data, thereby making the user's voice in the audio data used for recognition clearer and improving the accuracy of identifying user interruption commands during the interaction process.

[0027] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0028] Figure 1 FIG. 1 is a schematic diagram of an exemplary system 100 in which various methods and apparatuses described herein may be implemented according to an embodiment of the present disclosure. Figure 1 , the system 100 includes one or more client devices 101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120. The client devices 101, 102, 103, 104, 105, and 106 may be configured to execute one or more applications.

[0029] In an embodiment of the present disclosure, the server 120 may run one or more services or software applications that enable execution of a processing method for digital human voice interaction.

[0030] In some embodiments, server 120 may also provide other services or software applications that may include non-virtualized environments and virtualized environments. In some embodiments, these services may be provided as web-based services or cloud services, such as provided to users of client devices 101, 102, 103, 104, 105, and / or 106 under a software as a service (SaaS) model.

[0031] exist Figure 1 In the configuration shown, the server 120 may include one or more components that implement the functions performed by the server 120. These components may include software components, hardware components, or a combination thereof that can be executed by one or more processors. Users operating client devices 101, 102, 103, 104, 105, and / or 106 may in turn utilize one or more client applications to interact with the server 120 to utilize the services provided by these components. It should be understood that a variety of different system configurations are possible, which may differ from the system 100. Therefore, Figure 1 is an example of a system for implementing the processing method of digital human voice interaction described in this article and is not intended to be limiting.

[0032] The user may use client devices 101, 102, 103, 104, 105 and / or 106 to perform the digital human voice interaction processing method. The client device may provide an interface that enables the user of the client device to interact with the client device. The client device may also output information to the user via the interface. Figure 1 Only six client devices are depicted, but one skilled in the art will appreciate that the present disclosure may support any number of client devices.

[0033] Client devices 101, 102, 103, 104, 105 and / or 106 may include various types of computer devices, such as portable handheld devices, general-purpose computers (such as personal computers and laptop computers), workstation computers, wearable devices, smart screen devices, self-service terminal devices, service robots, game systems, thin clients, various messaging devices, sensors or other sensing devices, etc. These computer devices may run various types and versions of software applications and operating systems, such as MICROSOFT Windows, APPLE iOS, UNIX-like operating systems, Linux or Linux-like operating systems (such as GOOGLE Chrome OS); or include various mobile operating systems, such as MICROSOFT Windows Mobile OS, iOS, Windows Phone, Android. Portable handheld devices may include cellular phones, smart phones, tablet computers, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays (such as smart glasses) and other devices. Game systems may include various handheld game devices, Internet-enabled game devices, etc. Client devices are capable of executing various different applications, such as various Internet-related applications, communication applications (such as email applications), short message service (SMS) applications, and may use various communication protocols.

[0034] The network 110 may be any type of network known to those skilled in the art that may support data communications using any of a variety of available protocols, including but not limited to TCP / IP, SNA, IPX, etc. By way of example only, the one or more networks 110 may be a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and / or any combination of these and / or other networks.

[0035] Server 120 may include one or more general purpose computers, dedicated server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that may be virtualized to maintain a server's virtual storage device). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0036] The computing units in the server 120 may run one or more operating systems including any of the above operating systems and any commercially available server operating systems. The server 120 may also run any of a variety of additional server applications and / or middle-tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.

[0037] In some implementations, server 120 may include one or more applications to analyze and consolidate data feeds and / or event updates received from users of client devices 101, 102, 103, 104, 105, and 106. Server 120 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105, and 106.

[0038] In some embodiments, the server 120 may be a server of a distributed system, or a server combined with a blockchain. The server 120 may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. A cloud server is a host product in a cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and virtual private servers (VPS) services.

[0039] The system 100 may also include one or more databases 130. In some embodiments, these databases may be used to store audio data and other information. For example, one or more of the databases 130 may be used to store audio data such as collected by a microphone. The database 130 may reside in various locations. For example, the database used by the server 120 may be local to the server 120, or may be remote from the server 120 and may communicate with the server 120 via a network-based or dedicated connection. The database 130 may be of different types. In some embodiments, the database used by the server 120 may be, for example, a relational database. One or more of these databases may store, update, and retrieve data from the database in response to a command.

[0040] In some embodiments, one or more of the databases 130 may also be used by applications to store application data. The databases used by the applications may be different types of databases, such as a key-value store, an object store, or a conventional store backed by a file system.

[0041] Figure 1 The system 100 may be configured and operated in various ways to enable the application of various methods and apparatuses described in the present disclosure.

[0042] Figure 2 A flowchart of a method for processing digital human voice interaction according to an embodiment of the present disclosure is shown.

[0043] like Figure 2 As shown, the method 200 for processing digital human voice interaction includes:

[0044] Step 210: in response to receiving the first audio data, obtaining the first voiceprint data of the digital human, wherein the first audio data indicates that the user initiates a voice interaction request to the digital human, and the first voiceprint data indicates the voiceprint data currently used by the digital human for voice broadcasting;

[0045] Step 220: in response to receiving the second audio data, performing a first noise reduction process on the second audio data based on the first voiceprint data to obtain third audio data, wherein the second audio data is received after the first audio data is received; and

[0046] Step 230: In response to determining, based on the third audio data, that the user issues a first instruction to interrupt the voice interaction, a stop instruction is generated and sent to the digital human to control the digital human to stop the voice broadcast.

[0047] During the digital human voice interaction process, the user usually issues an interruption command when the digital human is performing a voice broadcast, so that a new voice interaction can be restarted for the next topic content. At this time, the digital human's own broadcast voice becomes the main noise in the audio data collected by the microphone of the digital human, and the digital human's broadcast voice is highly related to the current topic content, which will interfere with the recognition of the user's interruption command.

[0048] Therefore, by performing targeted noise reduction processing based on the voiceprint data of the digital person's own voice during the voice interaction process, the user's voice in the audio data used for recognition can be made clearer, and the accuracy of identifying the user's interruption command during the interaction process is improved. In particular, when the user's voiceprint and the voiceprint of the digital person's own voice are highly similar, this method of user command recognition based on the noise-reduced audio data is more effective.

[0049] In step 210, the first audio data may be, for example, audio data collected by the microphone of the digital person, and the first voiceprint data may be directly obtained from the digital person. In an example, the default voiceprint data of the digital person may be directly obtained. In another example, if the user has made specific settings for the broadcast voice of the digital person, the voiceprint data of the digital person after the settings may be obtained as the first voiceprint data.

[0050] Figure 3A partial flow chart of another method for processing digital human voice interaction according to an embodiment of the present disclosure is shown.

[0051] According to some embodiments, Figure 3 As shown, step 210 includes:

[0052] Step 311: In response to receiving the first audio data, use a first speech recognition model to perform speech recognition on the first audio data to obtain a first recognized text; and

[0053] Step 312: In response to determining, based on the first recognition text, that the user initiates a voice interaction request to the digital human, obtaining first voiceprint data.

[0054] In step 311, the first speech recognition model can be constructed based on neural networks such as DNN (Deep Neural Network), CNN (Convolutional Neural Network), LSTM (Long short-term memory), Conformer (a hybrid network structure), TDNN (Time-Delay Neural Network), etc. It is understandable that the first speech recognition model can adopt other types of network structures, which are not limited here.

[0055] In step 312, the judgment condition for determining whether the user initiates a voice interaction request may be that the first recognition text includes specific keywords indicating the initiation of a voice interaction request, such as the name of the digital person, etc.; or, the judgment condition may also be that the number of words in the recognized first recognition text exceeds the target threshold.

[0056] Therefore, it is possible to convert the first audio data into text based on voice recognition to determine whether the user has initiated a voice interaction request, and the implementation method is simple and has high accuracy.

[0057] In step 220, the second audio data may be, for example, audio data collected by the microphone of the digital human. In the example, all audio data collected by the microphone after the user initiates a voice interaction request may be subjected to noise reduction processing to avoid missing the user's interruption instruction due to noise interference.

[0058] Figure 4 A partial flow chart of another method for processing digital human voice interaction according to an embodiment of the present disclosure is shown.

[0059] According to some embodiments, Figure 4 As shown, the second audio data at least includes the user's voice audio. In addition to steps 210 to 230, method 200 also includes:

[0060] Step 410: Determine the second voiceprint data of the user according to the first audio data; and

[0061] Step 420: Determine whether the second audio data is received according to the second voiceprint information.

[0062] Therefore, only when the user's voice data is recognized will it be used as the second audio data for noise reduction processing, which can effectively save resource overhead.

[0063] In the example, if this is not the first time that the user interacts with the digital human and the digital human side or the server side has stored the user's voiceprint data, the stored user voiceprint data can be directly called as the second audio data to further reduce the processing difficulty.

[0064] Figure 5 A partial flow chart of another method for processing digital human voice interaction according to an embodiment of the present disclosure is shown.

[0065] According to some embodiments, Figure 5 As shown, step 230 includes:

[0066] Step 531: Use the second speech recognition model to perform speech recognition on the third audio data to obtain a second recognition text;

[0067] Step 532: In response to determining that the user issues a first instruction according to the second recognition text, a stop instruction is generated and sent to the digital human to control the digital human to stop voice broadcasting.

[0068] In step 531, the second speech recognition model can be constructed based on a neural network such as DNN, CNN, LSTM, Conformer, TDNN, etc. It is understandable that the second speech recognition model can adopt other types of network structures, which are not limited here.

[0069] In step 532, the judgment condition for determining that the user issued the first instruction may be that the second recognized text includes specific keywords indicating a request to interrupt voice interaction, such as "please wait a moment", "next question", or "interrupt", etc.; or, the judgment condition may also be that the number of words in the recognized second recognized text exceeds the target threshold. For example, if the recognized user voice only includes interjections such as "um", "ah", or "oh", it may be ignored as invalid audio data.

[0070] Therefore, it is possible to convert the second audio data into text based on voice recognition to determine whether the user has initiated a request to interrupt the voice interaction, and the implementation method is simple and has high accuracy.

[0071] Figure 6A partial flow chart of another method for processing digital human voice interaction according to an embodiment of the present disclosure is shown.

[0072] According to some embodiments, before the user issues an instruction to interrupt the voice interaction, the number of interactions between the two may be greater than one. Figure 5 As shown, in addition to steps 210 to 230, method 200 further includes:

[0073] Step 610: In response to determining, based on the second recognition text, that the user issues a second instruction to continue voice interaction, using a deep learning network model to process the second recognition text to obtain a target output text, wherein the target output text indicates the digital human's reply content to the second instruction; and

[0074] Step 620: Convert the target output text into fourth audio data for the digital human to perform voice broadcasting.

[0075] In step 610 , the deep learning network model may be, for example, a large language model to generate incremental reply content as target output text based on the user's second instruction.

[0076] In step 620, exemplarily, the large language model usually outputs the target output text in the form of segments, so when the digital human performs voice broadcasting, it can instantly convert each segment of the target output subtext into audio data for broadcasting when the segment is received, or it can simultaneously convert multiple target output subtexts into audio data for broadcasting when the number or size of the received target output subtexts is greater than a specific threshold, and it can also convert the completed target output text into audio data for broadcasting when it is received.

[0077] According to some embodiments, the method 200 further includes: sending the target output text to a display device to display it to the user. Exemplarily, the display device may be, for example, a display screen of the digital human. Based on this, even if the interruption instruction issued by the user causes the digital human to stop the voice broadcast, the complete text content corresponding to the voice broadcast will be displayed to the user in text form, which can provide the user with a better interactive experience.

[0078] Figure 7 A partial flow chart of another method for processing digital human voice interaction according to an embodiment of the present disclosure is shown.

[0079] According to some embodiments, Figure 7 Before step 230, method 200 further includes:

[0080] Step 710: Perform environmental noise reduction processing on the third audio data to update the third audio data; and

[0081] Step 720: Determine whether the user issues a first instruction to interrupt the voice interaction based on the updated third audio data.

[0082] Exemplarily, the noise targeted by the environmental noise reduction processing in step 710 may be, for example, other human voices, traffic sounds, wind and rain sounds, etc. around the digital human except for the voice of the user who is performing voice interaction. Thus, the noise interference in the audio data used for user interruption command recognition can be further reduced, and the accuracy of user command recognition can be further improved.

[0083] Figure 8 A structural block diagram of a digital human voice interaction processing device according to an embodiment of the present disclosure is shown.

[0084] According to another aspect of the present disclosure, Figure 8 As shown, a processing device 800 for digital human voice interaction is provided, comprising: an acquisition module 810, configured to acquire first voiceprint data of the digital human in response to receiving first audio data, wherein the first audio data indicates that the user initiates a voice interaction request to the digital human, and the first voiceprint data indicates the voiceprint data currently used by the digital human for voice broadcasting; a first noise reduction module 820, configured to perform first noise reduction processing on the second audio data based on the first voiceprint data in response to receiving second audio data to obtain third audio data, wherein the time when the second audio data is received is after the time when the first audio data is received; and a processing module 830, configured to generate a stop instruction in response to determining that the user issues a first instruction to interrupt the voice interaction according to the third audio data, and send it to the digital human to control the digital human to stop voice broadcasting.

[0085] According to some embodiments, the acquisition module 810 includes: a first recognition module 811, configured to, in response to receiving first audio data, perform speech recognition on the first audio data using a first speech recognition model to obtain a first recognition text; and an acquisition sub-module 812, configured to, in response to determining, based on the first recognition text, that a user initiates a voice interaction request to a digital human, obtain first voiceprint data.

[0086] According to some embodiments, the second audio data includes at least the user's voice audio, and the device 800 also includes: a first determination module 840, configured to determine the user's second voiceprint data based on the first audio data; and a second determination module 850, determining whether the second audio data is received based on the second voiceprint information.

[0087] According to some embodiments, the processing module 830 includes: a second recognition module 831, configured to use a second speech recognition model to perform speech recognition on the third audio data to obtain a second recognition text; and a generation submodule 832, configured to generate the stop instruction and send it to the digital human to control the digital human to stop voice broadcasting in response to determining that the user has issued the first instruction based on the second recognition text.

[0088] According to some embodiments, the device 800 also includes: an output module 860, which is configured to, in response to determining that the user issues a second instruction to continue the voice interaction based on the second recognition text, use a deep learning network model to process the second recognition text to obtain a target output text, wherein the target output text indicates the digital human's reply content to the second instruction; and a conversion module 870, which is configured to convert the target output text into fourth audio data for the digital human to perform voice broadcasting.

[0089] According to some embodiments, the apparatus 800 further includes: a display module 880 configured to send the target output text to a display device for displaying to a user.

[0090] According to some embodiments, the device 800 also includes: a second noise reduction module 890, configured to perform environmental noise reduction processing on the third audio data to update the third audio data; and a third determination module 8100, configured to determine whether the user issues a first instruction to interrupt the voice interaction based on the updated third audio data.

[0091] According to another aspect of the present disclosure, an electronic device is also provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the aforementioned method.

[0092] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is further provided, wherein the computer instructions are used to enable the computer to execute the aforementioned method.

[0093] According to another aspect of the present disclosure, a computer program product is also provided, including a computer program, wherein the computer program implements the aforementioned method when executed by a processor.

[0094] like Fig. 9As shown, the electronic device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the electronic device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0095] Multiple components in the electronic device 900 are connected to the I / O interface 905, including: an input unit 906, an output unit 907, a storage unit 908, and a communication unit 909. The input unit 906 can be any type of device that can input information to the electronic device 900. The input unit 906 can receive input digital or character information, and generate key signal input related to user settings and / or function control of the electronic device, and can include but is not limited to a mouse, a keyboard, a touch screen, a trackpad, a trackball, a joystick, a microphone and / or a remote control. The output unit 907 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator and / or a printer. The storage unit 908 can include but is not limited to a disk and an optical disk. The communication unit 909 allows the electronic device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver and / or a chipset, such as Bluetooth TM devices, 802.11 devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0096] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as a matrix calculation method based on a GPU. For example, in some embodiments, a matrix calculation method based on a GPU may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 900 via a ROM 902 and / or a communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the matrix calculation method based on the GPU described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform a matrix calculation method based on a GPU in any other appropriate manner (e.g., by means of firmware).

[0097] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0098] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0099] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0100] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0101] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0102] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0103] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0104] Although the embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it should be understood that the above-mentioned methods, systems and devices are merely exemplary embodiments or examples, and the scope of the present invention is not limited by these embodiments or examples, but only by the claims after authorization and their equivalent scope. Various elements in the embodiments or examples can be omitted or replaced by their equivalent elements. In addition, each step can be performed in an order different from that described in the present disclosure. Further, the various elements in the embodiments or examples can be combined in various ways. It is important that with the evolution of technology, many elements described herein can be replaced by equivalent elements that appear after the present disclosure.

Claims

1. A method for processing digital human voice interaction, comprising: In response to receiving the first audio data, obtaining first voiceprint data of the digital human, wherein the first audio data indicates that the user initiates a voice interaction request to the digital human, and the first voiceprint data indicates the voiceprint data currently used by the digital human for voice broadcasting; In response to receiving second audio data, performing a first noise reduction process on the second audio data based on the first voiceprint data to obtain third audio data, wherein the second audio data is received after the first audio data is received; and In response to determining, based on the third audio data, that the user issues a first instruction indicating interruption of voice interaction, a stop instruction is generated and sent to the digital human to control the digital human to stop voice broadcasting.

2. The method according to claim 1, wherein: The step of obtaining first voiceprint data of a digital person in response to receiving the first audio data includes: In response to receiving the first audio data, performing speech recognition on the first audio data using a first speech recognition model to obtain a first recognized text; and In response to determining, based on the first recognition text, that the user initiates a voice interaction request to the digital human, the first voiceprint data is acquired.

3. The method according to claim 1 or 2, wherein: The second audio data at least includes the voice audio of the user, and the method further includes: Determining second voiceprint data of the user according to the first audio data; and Determine whether the second audio data is received according to the second voiceprint information.

4. The method according to any one of claims 1 to 3, wherein: In response to determining, according to the third audio data, that the user issues a first instruction to interrupt the voice interaction, generating a stop instruction and sending it to the digital human to control the digital human to stop voice broadcasting, comprising: Performing speech recognition on the third audio data using a second speech recognition model to obtain a second recognized text; and In response to determining, based on the second recognition text, that the user issues the first instruction, the stop instruction is generated and sent to the digital human to control the digital human to stop voice broadcasting.

5. The method according to claim 4, further comprising: In response to determining, based on the second recognized text, that the user issues a second instruction to continue voice interaction, processing the second recognized text using a deep learning network model to obtain a target output text, wherein the target output text indicates the digital human's reply content to the second instruction; and The target output text is converted into fourth audio data for the digital human to perform voice broadcasting.

6. The method according to claim 5, further comprising: The target output text is sent to a display device to be displayed to the user.

7. The method according to any one of claims 1 to 5, wherein: Before the step of determining, according to the third audio data, that the user issues a first instruction to interrupt the voice interaction, the method further includes: performing environmental noise reduction processing on the third audio data to update the third audio data; and Determine whether the user issues the first instruction to interrupt the voice interaction based on the updated third audio data.

8. A processing device for digital human voice interaction, comprising: an acquisition module, configured to acquire first voiceprint data of the digital human in response to receiving first audio data, wherein the first audio data indicates that the user initiates a voice interaction request to the digital human, and the first voiceprint data indicates the voiceprint data currently used by the digital human for voice broadcasting; A first noise reduction module is configured to, in response to receiving second audio data, perform a first noise reduction process on the second audio data based on the first voiceprint data to obtain third audio data, wherein the second audio data is received after the first audio data is received; and The processing module is configured to generate a stop instruction and send it to the digital human to control the digital human to stop voice broadcasting in response to determining, based on the third audio data, that the user issues a first instruction to interrupt voice interaction.

9. The device according to claim 8, wherein: The acquisition module comprises: A first recognition module is configured to, in response to receiving the first audio data, perform speech recognition on the first audio data using a first speech recognition model to obtain a first recognition text; and The acquisition submodule is configured to acquire the first voiceprint data in response to determining, based on the first recognition text, that the user initiates a voice interaction request to the digital human.

10. The device according to claim 8 or 9, wherein: The second audio data at least includes the user's voice audio, and the device further includes: A first determining module is configured to determine second voiceprint data of the user according to the first audio data; and The second determination module determines whether the second audio data is received according to the second voiceprint information.

11. The device according to any one of claims 8 to 10, wherein: The processing module comprises: A second recognition module is configured to perform speech recognition on the third audio data using a second speech recognition model to obtain a second recognition text; and The generating submodule is configured to generate the stop instruction and send it to the digital human to control the digital human to stop voice broadcasting in response to determining that the user issues the first instruction according to the second recognition text.

12. The apparatus according to claim 11, further comprising: an output module, configured to, in response to determining, based on the second recognized text, that the user issues a second instruction to continue voice interaction, process the second recognized text using a deep learning network model to obtain a target output text, wherein the target output text indicates the digital human's reply content to the second instruction; and The conversion module is configured to convert the target output text into fourth audio data for the digital human to perform voice broadcasting.

13. The apparatus according to claim 12, further comprising: The display module is configured to send the target output text to a display device to display it to the user.

14. The apparatus according to any one of claims 8 to 13, further comprising: a second noise reduction module, configured to perform environmental noise reduction processing on the third audio data to update the third audio data; as well as The third determination module is configured to determine whether the user issues the first instruction to interrupt the voice interaction based on the updated third audio data.

15. An electronic device, comprising: at least one processor; as well as a memory communicatively coupled to the at least one processor; in The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-7.

17. A computer program product comprising a computer program, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images