Voice interaction method and device, electronic equipment, computer readable storage medium and computer program product

Abstract

The invention provides a voice interaction method and device, electronic equipment, a computer readable storage medium and a computer program product. The method comprises the steps of performing first recognition on first voice to obtain a first recognition result; when the first recognition result represents that the first voice comprises the to-be-recognized voice, performing second recognition on the to-be-recognized voice to obtain a second recognition result; when the second recognition result represents that the first execution instruction matched with the to-be-recognized voice exists, executing an operation indicated by the first execution instruction; when the first recognition result represents that the first voice does not include the to-be-recognized voice or the second recognition result represents that a first execution instruction matched with the to-be-recognized voice does not exist, performing third recognition on the first voice in response to an awakening instruction of the voice interaction function to obtain a third recognition result; and when the third recognition result represents that the first voice comprises the effective semantics, executing an operation indicated by the first voice. According to the invention, the voice interaction efficiency can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a voice interaction method, device, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] With the continuous advancement of artificial intelligence and speech recognition technologies, intelligent voice interaction systems have become an indispensable part of daily life. A voice interaction system is an intelligent application that interacts with users through voice recognition and natural language processing technologies. Initially, intelligent voice interaction systems could only perform basic functions such as voice input and text messaging, while current systems can perform more operations based on user needs, such as voice search, voice translation, and voice news broadcasting. Furthermore, voice interaction systems can interconnect with other smart devices to enable the control and management of smart homes.

[0003] In related technologies, it is necessary to first say a wake word to the voice interaction system to wake it up, and then give a voice command to the voice interaction system before it will respond and execute the voice command. It does not support the mode of giving a voice command first and then waking up the voice interaction system, which results in low efficiency of voice interaction. Summary of the Invention

[0004] This application provides a voice interaction method, device, electronic device, computer-readable storage medium, and computer program product, which can improve the efficiency of voice interaction.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides a voice interaction method, the method comprising:

[0007] A first speech is subjected to a first recognition to obtain a first recognition result, wherein the first speech does not include a wake word used to activate the voice interaction function;

[0008] When the first recognition result indicates that the first speech includes the speech to be recognized, the speech to be recognized is subjected to a second recognition to obtain a second recognition result.

[0009] When the second recognition result indicates the existence of a first execution instruction that matches the speech to be recognized, the operation indicated by the first execution instruction is executed;

[0010] When the first recognition result indicates that the first speech does not include the speech to be recognized, or the second recognition result indicates that there is no first execution instruction matching the speech to be recognized, in response to the wake-up instruction of the voice interaction function, the first speech is subjected to third recognition to obtain the third recognition result;

[0011] When the third recognition result indicates that the first speech contains valid semantics, the operation indicated by the first speech is executed.

[0012] This application provides a voice interaction device, including:

[0013] The first recognition module is used to perform a first recognition on the first speech and obtain a first recognition result, wherein the first speech does not include a wake word for activating the voice interaction function;

[0014] The second recognition module is used to perform a second recognition on the speech to be recognized when the first recognition result indicates that the first speech includes the speech to be recognized, and obtain a second recognition result.

[0015] The first execution module is configured to execute the operation indicated by the first execution instruction when the second recognition result indicates that there is a first execution instruction that matches the speech to be recognized;

[0016] The third recognition module is used to perform third recognition on the first speech in response to the wake-up command of the voice interaction function when the first recognition result indicates that the first speech does not include the speech to be recognized, or the second recognition result indicates that there is no first execution instruction matching the speech to be recognized; and obtain the third recognition result.

[0017] The second execution module is used to execute the operation indicated by the first speech when the third recognition result indicates that the first speech includes valid semantics.

[0018] This application provides an electronic device, the electronic device comprising:

[0019] Memory is used to store executable instructions or computer programs.

[0020] The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the voice interaction method provided in the embodiments of this application.

[0021] This application provides a computer-readable storage medium storing computer-executable instructions or computer programs, which, when executed by a processor, implement the voice interaction method provided in this application.

[0022] This application provides a computer program product, including computer-executable instructions or a computer program, which, when executed by a processor, implements the voice interaction method provided in this application.

[0023] The embodiments of this application have the following beneficial effects:

[0024] Applying the embodiments of this application, a first speech is first recognized to obtain a first recognition result. The first speech does not include a wake-up word for activating the voice interaction function. Then, when the first recognition result indicates that the first speech includes the speech to be recognized, a second recognition is performed on the speech to be recognized to obtain a second recognition result. Then, when the second recognition result indicates that there is a first execution instruction matching the speech to be recognized, the operation indicated by the first execution instruction is executed. When the first recognition result indicates that the first speech does not include the speech to be recognized, or the second recognition result indicates that there is no first execution instruction matching the speech to be recognized, in response to the wake-up instruction of the voice interaction function, a third recognition is performed on the first speech to obtain a third recognition result. Then, when the third recognition result indicates that the first speech includes valid semantics, the operation indicated by the first speech is executed. Thus, 1) when there is a first execution instruction that matches the first speech to be recognized, the operation indicated by the first execution instruction can be executed without using a wake-up word, thereby realizing a voice interaction mode without a wake-up word; 2) when there is no first execution instruction that matches the first speech to be recognized, in response to the wake-up instruction of the voice interaction function, when the first speech is recognized to include valid semantics, the operation indicated by the first speech is executed, thereby realizing a voice interaction mode that supports issuing voice commands first and then waking up; based on this, the voice interaction mode is added through the embodiments of this application, thereby improving the efficiency of voice interaction. Attached Figure Description

[0025] Figure 1 This is a schematic diagram illustrating the application mode of the voice interaction method provided in the embodiments of this application;

[0026] Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0027] Figure 3A This is a first flowchart illustrating the voice interaction method provided in an embodiment of this application;

[0028] Figure 3B This is a second flowchart illustrating the voice interaction method provided in the embodiments of this application;

[0029] Figure 3C This is a third flowchart illustrating the voice interaction method provided in the embodiments of this application;

[0030] Figure 3D This is a schematic diagram of the fourth process of the voice interaction method provided in the embodiments of this application;

[0031] Figure 4 This is a schematic diagram of the first process of the wake-up-free voice interaction system provided in the embodiments of this application;

[0032] Figure 5This is a schematic diagram of the second process of the voice interaction system without wake-up provided in the embodiments of this application.

[0033] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0035] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0036] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0037] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0038] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0039] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0040] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0041] 1) Voice wake-up: Detects the wake word in the voice input to wake up the voice interaction system of the electronic device.

[0042] 2) Wake-up-free: No need to say a wake-up word, just give the voice command to be executed.

[0043] 3) Vocal range: The area where the user speaks.

[0044] 4) Vocal range localization: Determine the vocal range of the user based on the user's vocal output.

[0045] 5) Effective semantics: Instructions given to the voice interaction system of electronic devices.

[0046] 6) Invalid semantics: Not a command given to the voice interaction system of an electronic device.

[0047] 7) Voice Activity Detection (VAD): Detects whether the current speech signal contains a speech signal, that is, judges the input speech signal and distinguishes the speech signal from various background noise signals.

[0048] 8) Automatic Speech Recognition (ASR): A speech recognition model that converts speech information into a text representation that machines can understand.

[0049] 9) Natural Language Understanding (NLU): It is an important component of the field of natural language processing. By analyzing and processing text, it enables computers to understand human language.

[0050] 10) Single-trigger command: Voice commands that include a wake word and an execution command.

[0051] This application provides a voice interaction method, device, electronic device, computer-readable storage medium, and computer program product, which can improve the efficiency of voice interaction.

[0052] The following describes exemplary applications of the electronic devices provided in the embodiments of this application. These electronic devices can be implemented as various types of terminals such as laptops, tablets, desktop computers, set-top boxes, smartphones, smart speakers, smartwatches, smart TVs, and in-vehicle terminals, or as servers. The following will describe exemplary applications when the device is implemented as a server.

[0053] See Figure 1 , Figure 1 This is a schematic diagram illustrating the application mode of the voice interaction method provided in the embodiments of this application, for example. Figure 1 The system involves server 200, network 300, and terminal 400. Terminal 400 is connected to server 200 through network 300, which can be a wide area network, a local area network, or a combination of both.

[0054] During voice interaction, terminal 400 responds to the user's voice interaction command by sending a voice interaction request to server 200. Server 200 responds to the voice interaction request by performing a first recognition on the first voice, obtaining a first recognition result. When the first recognition result indicates that the first voice includes the voice to be recognized, a second recognition is performed on the voice to be recognized, obtaining a second recognition result. When the second recognition result indicates that there is a first execution command matching the voice to be recognized, the operation indicated by the first execution command is executed. When the first recognition result indicates that the first voice does not include the voice to be recognized, or the second recognition result indicates that there is no first execution command matching the voice to be recognized, a third recognition is performed on the first voice in response to the wake-up command of the voice interaction function, obtaining a third recognition result. When the third recognition result indicates that the first voice includes valid semantics, the operation indicated by the first voice is executed, and the voice interaction result of the first voice is returned to terminal 400. Terminal 400 receives the voice interaction result and outputs the voice interaction result. In this way, the user can understand the voice interaction result through terminal 400.

[0055] In some embodiments, the server (e.g., server 200) can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminals and servers can be connected directly or indirectly via wired or wireless communication, which is not limited in this embodiment.

[0056] The embodiments of this application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart home, smart wearable devices, virtual assistants, smart speakers, smart transportation, assisted driving, autonomous driving, autonomous driving, drones, virtual humans, robots, conversational interaction, intelligent customer service, etc.

[0057] See Figure 2 , Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may be a terminal or a server. Figure 2 The illustrated electronic device 600 includes at least one processor 410, a memory 450, and at least one network interface 420. The various components in the electronic device 600 are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 440.

[0058] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0059] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0060] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0061] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0062] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, and driver layer, for implementing various basic business functions and handling hardware-based tasks.

[0063] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including Bluetooth, WiFi, and Universal Serial Bus (USB).

[0064] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 A voice interaction device 455 stored in memory 450 is shown. It can be software in the form of programs and plug-ins, including the following software modules: a first recognition module 4551, a second recognition module 4552, a first execution module 4553, a third recognition module 4554, and a second execution module 4555. These modules are logical and can therefore be arbitrarily combined or further divided according to the functions they implement. The functions of each module will be described below.

[0065] In some embodiments, the terminal or server can implement the voice interaction method provided in this application by running various computer-executable instructions or computer programs. For example, computer-executable instructions can be microprogram-level commands, machine instructions, or software instructions. Computer programs can be native programs or software modules in an operating system; they can be native applications (APPs), i.e., programs that need to be installed in the operating system to run, such as voice assistant APPs; or they can be applets that can be embedded in any APP, i.e., programs that only need to be downloaded to a browser environment to run. In summary, the aforementioned computer-executable instructions can be any form of instruction, and the aforementioned computer programs can be any form of application, module, or plugin.

[0066] The voice interaction method provided in this application will be described in conjunction with exemplary applications and implementations of the server devices provided in the embodiments of this application.

[0067] The following describes the voice interaction method provided in the embodiments of this application. As mentioned above, the electronic device implementing the voice interaction method of the embodiments of this application can be a terminal, a server, or a combination of both. Next, taking an electronic device as a server as an example, the voice interaction method provided in the embodiments of this application will be described. See also... Figure 3A , Figure 3A This is a first flowchart illustrating the voice interaction method provided in this application embodiment, which will be combined with... Figure 3A The steps shown are explained.

[0068] In step 301, the first speech is subjected to first recognition to obtain a first recognition result.

[0069] Here, the first voice input does not include the wake word used to activate the voice interaction function. This voice interaction function can be provided by a voice assistant (i.e., a voice interaction system). The first voice input refers to the currently input user voice, and the wake word is the word used to activate the voice interaction function. For example, the wake word can be the name of the voice interaction system or any preset word.

[0070] The first recognition refers to inputting the first speech into the rejection model for recognition processing. The first recognition result indicates whether the first speech contains the speech to be recognized. The rejection model can be an offline Natural Language Understanding (NLU) model. The rejection model is used to reduce the false wake-up rate. In voice assistants (i.e., voice interaction systems), the rejection model is used to distinguish the user's real speech from random sounds or noise that may trigger wake-up word detection. The rejection model can analyze the features of the first speech, such as frequency, intensity, speech rate, and pitch, to distinguish the speech to be recognized from noise within the first speech.

[0071] In some embodiments, see Figure 3B , Figure 3B This is a schematic diagram of the second process of the voice interaction method provided in the embodiments of this application. Figure 3A Before step 301 shown, the following steps can also be performed: Figure 3B Steps 3001 to 3002 are explained in detail below.

[0072] In step 3001, the device status of the electronic device including the voice interaction function is determined.

[0073] Here, device status refers to the current operating status of electronic devices, including those with voice interaction capabilities.

[0074] In step 3002, when the device state is not the target device state, it is determined to perform the first recognition operation on the first voice.

[0075] Here, the target device state refers to a device state where the voice interaction function is unavailable. In this state, the first voice recognition operation is not performed, meaning the device does not respond to the user's first voice input. For example, taking an electronic device as an in-vehicle terminal, the target device state could be: the in-vehicle terminal is parked; the in-vehicle terminal is in Bluetooth call mode; the in-vehicle terminal is in emergency call mode; the in-vehicle terminal is running a voice interaction game; the in-vehicle terminal is in voiceprint recording mode; the in-vehicle terminal is in driving navigation broadcast mode; the in-vehicle terminal is in voice input mode; or the in-vehicle terminal's central control screen has its audio zone enabled.

[0076] In this embodiment of the application, when the electronic device including the voice interaction function is not in the target device state, the operation of performing the first recognition of the first voice is performed, and the conditions for performing the first recognition of the first voice can be determined according to the device state.

[0077] In some embodiments, when the device state is not the target device state, the following steps may be performed: performing voice activity detection on the first speech to obtain a detection result; when the detection result indicates the presence of voice activity, performing voice recognition on the first speech to obtain the first speech text of the first speech; and performing first recognition on the first speech text to obtain a first recognition result.

[0078] Here, speech activity detection refers to detecting whether the first speech contains user speech. Speech activity detection can be achieved through Voice Activation Detection (VAD), a signal processing technique that analyzes the characteristics of the first speech, such as its energy level, frequency content, and temporal characteristics, to determine whether it contains speech produced by a speaking user. When the detection result indicates the presence of speech activity, i.e., when the first speech contains user speech, an Automatic Speech Recognition (ASR) model can be used to recognize the first speech. An ASR model is a machine learning model that converts input audio into text. Whether operating offline or in a mixed online / offline state, an ASR model converts the first speech into easily processed text, obtaining the first speech text. Then, a rejection model performs initial recognition on the first speech text, outputting the recognized text content as the initial recognition result.

[0079] In this embodiment of the application, by performing voice activity detection on the first speech and performing first recognition based on the voice text of the first speech, the accuracy of the first speech recognition is improved.

[0080] In some embodiments, the first speech text can be first recognized by performing the following steps to obtain a first recognition result: performing a fourth recognition on the first speech based on the first speech text to obtain a fourth recognition result; when the fourth recognition result indicates that the first speech is not invalid speech, performing a first recognition on the first speech text to obtain a first recognition result; when the fourth recognition result indicates that the first speech is invalid speech, discarding the first speech.

[0081] Here, the fourth recognition refers to the identification of invalid speech in the first speech. The fourth recognition result indicates whether the first speech is invalid speech. Invalid speech refers to short speech with a duration lower than a preset duration threshold. Based on the first speech text, the fourth recognition is performed on the first speech. When the first speech is not invalid speech, the first speech text is recognized by the rejection model, and the recognized first speech text content is output to obtain the first recognition result. When the first speech is invalid speech, the first speech is discarded.

[0082] In this embodiment of the application, by performing a fourth recognition on the first speech text, when the fourth recognition result indicates that the first speech is not invalid speech, a first recognition is performed on the first speech text, thereby avoiding the first recognition of invalid speech and improving the accuracy of the first recognition of the first speech.

[0083] Continue to refer to Figure 3AIn step 302, when the first recognition result indicates that the first speech includes the speech to be recognized, the speech to be recognized is subjected to a second recognition to obtain a second recognition result.

[0084] Here, the first speech includes the speech to be recognized, meaning the first speech has passed the recognition process of the rejection model. The second recognition is to identify the execution instruction that matches the speech to be recognized, and the result of the second recognition indicates whether there is an execution instruction that matches the speech to be recognized.

[0085] In some embodiments, see Figure 3C , Figure 3C This is a schematic diagram of the third process of the voice interaction method provided in the embodiments of this application. Figure 3A The illustrated step 302 can be achieved through Figure 3C Steps 3021 to 3024 are implemented, and will be explained in detail below.

[0086] In step 3021, the voice commands included in the speech to be recognized are determined, and M execution commands are obtained.

[0087] Here, M is a positive integer, and a whitelist of execution instructions is constructed, consisting of M execution instructions from the whitelist. A pre-trained speech recognition model (such as a deep neural network) is used to detect words or phrases in the speech to be recognized, thereby determining the speech instructions contained in the speech.

[0088] In step 3022, the voice command is matched with each execution command.

[0089] Here, words or phrases in the voice commands are standardized, for example: removing unnecessary modifiers from phrases in the voice commands, converting phrases in the voice commands and phrases in each executed command to the same grammatical structure, and ignoring capitalization differences between words in the voice commands and words in each executed command. A matching algorithm is used to match the voice commands with each executed command. The matching algorithm can be a simple string matching of words or phrases in the voice commands and words or phrases in each executed command, or it can be a pattern matching of words or phrases in the voice commands and words or phrases in each executed command based on grammatical structure and contextual information. When the matching degree between a voice command and an executed command exceeds a preset matching degree threshold, the executed command is determined to be a matching executed command.

[0090] In step 3023, when there is an execution instruction among the M execution instructions that matches the voice instruction, a second recognition result is obtained, which represents the existence of a first execution instruction that matches the voice to be recognized.

[0091] Here, when the matching degree between the voice instructions included in the speech to be recognized and a certain execution instruction exceeds a preset matching degree threshold, it is determined that there is an execution instruction among the M execution instructions that matches the speech to be recognized. At this time, a second recognition result representing the existence of a first execution instruction that matches the speech to be recognized is obtained.

[0092] In step 3024, when there is no execution instruction among the M execution instructions that matches the voice instruction, a second recognition result is obtained, which represents that there is no first execution instruction that matches the voice to be recognized.

[0093] Here, when the matching degree between the voice instructions included in the speech to be recognized and each execution instruction is lower than a preset matching degree threshold, it is determined that there is no execution instruction among the M execution instructions that matches the speech to be recognized. At this time, a second recognition result is obtained, which represents that there is no first execution instruction that matches the speech to be recognized.

[0094] Continue to refer to Figure 3A In step 303, when the second recognition result indicates the existence of a first execution instruction that matches the speech to be recognized, the operation indicated by the first execution instruction is executed.

[0095] Here, when a first execution instruction matching the speech to be recognized exists, semantic analysis, such as part-of-speech tagging and syntactic analysis, is performed on the first execution instruction to determine its specific meaning. The context in which the first execution instruction appears is also considered to more accurately understand its meaning. Combining the semantic analysis and contextual understanding of the first execution instruction, machine learning models, such as classifiers or clustering algorithms, are used to identify the intent of the first execution instruction. Based on the identified intent of the first execution instruction, the operation indicated by the first execution instruction is executed.

[0096] In this embodiment, when there is a first execution instruction that matches the first speech to be recognized, the operation indicated by the first execution instruction can be executed without using a wake word, thus improving the efficiency of voice interaction.

[0097] Continue to refer to Figure 3A In step 304, when the first recognition result indicates that the first speech does not include the speech to be recognized, or the second recognition result indicates that there is no first execution instruction matching the speech to be recognized, the first speech is subjected to third recognition in response to the wake-up instruction of the voice interaction function to obtain the third recognition result.

[0098] Here, the wake-up command refers to the voice command that activates the voice interaction function. The third recognition is the recognition of the effective semantics of the first speech. Effective semantics refers to the voice commands that the voice interaction function can support (or execute). The result of the third recognition indicates whether the first speech includes effective semantics.

[0099] In some embodiments, in response to a wake-up command of a voice interaction function, a third recognition is performed on the first voice. Before obtaining the third recognition result, the wake-up command can be received by performing the following steps: determining a first time point for acquiring the first voice; detecting a second voice within a first duration after the first time point, the second voice including a wake-up word; and determining that a wake-up command has been received when the second voice is detected within the first duration after the first time point.

[0100] Here, the first time point refers to the time point at which the first voice is acquired, and the first duration can be preset. Within the first duration after the first time point, if a second voice containing a wake-up word is detected, it is determined that a wake-up command for activating the voice interaction function has been received. For example, the first duration can be 2 seconds. If a second voice containing a wake-up word is detected within 2 seconds after the first time point of acquiring the first voice, it is determined that a wake-up command has been received.

[0101] In this embodiment, when there is no first execution instruction that matches the first speech to be recognized, the detection of the second speech, including the wake-up word, is performed within a first duration after the first time point of the first speech, thus realizing a mode of waking up the voice interaction function by first saying the voice instruction and then saying the wake-up word.

[0102] In some embodiments, when the second voice is detected within a first duration after the first time point, the wake-up command can be determined to be received by performing the following steps: when the second voice is detected within a first duration after the first time point, the second voice is subjected to a fifth recognition to obtain a fifth recognition result; when the fifth recognition result indicates that the second voice does not include a voice command, it is determined that a wake-up command has been received; when the fifth recognition result indicates that the second voice includes a voice command, the operation indicated by the voice command is executed.

[0103] Here, the fifth recognition step involves recognizing the voice command in the second speech. The result of the fifth recognition indicates whether the second speech includes a voice command. When the second speech does not include a voice command, i.e., it only includes a wake-up word, it is determined that a wake-up command for activating the voice interaction function has been received. In response to the wake-up command, the voice interaction function is activated to execute the operation indicated by the voice command in the first speech. When the second speech includes a voice command, i.e., it includes both a wake-up word and a voice command, the voice interaction function is activated to execute the operation indicated by the voice command in the second speech.

[0104] In this embodiment, the voice interaction function is activated by a voice command that includes a second voice word to execute the operation indicated by the voice command in the second voice word, thus realizing a mode of activating the voice interaction function by first saying the voice command and then saying the wake word.

[0105] In some embodiments, when the fifth recognition result indicates that the second speech does not include a voice command, the following steps can be performed to determine that a wake-up command has been received: when the fifth recognition result indicates that the second speech does not include a voice command, determine a second time point for acquiring the second speech; perform speech activity detection within a second duration after the second time point; when no speech activity is detected within the second duration after the second time point, determine that a wake-up command has been received; when speech activity is detected within the second duration after the second time point, acquire a third speech that generated the speech activity; and execute the operation indicated by the third speech.

[0106] Here, the second time point refers to the time point at which the second voice is acquired, and the second duration can be preset. When the second voice does not include a voice command, voice activity detection is performed within the second duration after the second time point. When no voice activity is detected within the second duration after the second time point, it is determined that a wake-up command for activating the voice interaction function has been received. In response to the wake-up command, the voice interaction function is activated to execute the operation indicated by the voice command in the first voice. When voice activity is detected within the second duration after the second time point, the third voice generated by the voice activity is acquired, and the voice interaction function is activated to execute the operation indicated by the voice command in the third voice.

[0107] For example, the second duration can be 1000 milliseconds. When the second voice does not include a voice command, voice activity detection is performed within 1000 milliseconds after the second time point. If no voice activity is detected within 1000 milliseconds after the second time point, it is determined that a wake-up command has been received; if voice activity is detected within 1000 milliseconds after the second time point, the third voice that generated the voice activity is obtained, and the operation indicated by the third voice is executed.

[0108] In this embodiment, voice activity detection is performed within a second duration after the second voice acquisition time point, and a third voice that generates voice activity is acquired. The operation indicated by the third voice is executed. Through the voice activity detection of the second voice, a mode of waking up the voice interaction function by first speaking the voice command and then speaking the wake-up word is realized.

[0109] In some embodiments, in response to a wake-up command of a voice interaction function, a third recognition is performed on the first speech. Before obtaining the third recognition result, the following steps may be performed: caching the first speech; and deleting the cached first speech when the third recognition result indicates that the first speech does not contain valid semantics.

[0110] Here, when the first recognition result indicates that the first speech does not include the speech to be recognized, or the second recognition result indicates that there is no first execution instruction matching the speech to be recognized, the speech text of the first speech is cached. When the first speech does not include valid semantics, that is, when the first speech does not include voice instructions for voice interaction, the cached speech text of the first speech is deleted.

[0111] In this embodiment of the application, the first speech is cached before the third recognition is performed on the first speech, so as to facilitate the subsequent detection and processing of wake words and voice commands on the cached first speech.

[0112] Continue to refer to Figure 3A In step 305, when the third recognition result indicates that the first speech contains valid semantics, the operation indicated by the first speech is performed.

[0113] Here, when the first speech includes valid semantics, that is, when the first speech includes voice instructions for performing voice interaction, the operation indicated by the first speech is executed.

[0114] In some embodiments, see Figure 3D , Figure 3D This is a schematic diagram of the fourth process of the voice interaction method provided in the embodiments of this application, which can also execute... Figure 3D Steps 306 and 307 are explained in detail below.

[0115] In step 306, multiple fourth voice messages are received, each of which does not include a wake word.

[0116] Here, the fourth voice refers to the currently input user voice. Multiple currently input user voices are received, and each user voice does not include a wake word used to activate the voice interaction function.

[0117] In step 307, the following processing is performed serially for each fourth speech:

[0118] In step 3071, the fourth speech is subjected to first recognition to obtain the sixth recognition result.

[0119] Here, the specific implementation process of step 3071 is the same as that of step 301 above, and will not be repeated here.

[0120] In step 3072, when the sixth recognition result indicates that the fourth speech includes the target speech to be recognized, the target speech to be recognized is subjected to a second recognition to obtain a seventh recognition result.

[0121] Here, the specific implementation process of step 3072 is the same as that of step 302 above, and will not be repeated here.

[0122] In step 3073, when the seventh recognition result indicates the existence of a second execution instruction that matches the target speech to be recognized, the operation indicated by the second execution instruction is executed.

[0123] Here, the specific implementation process of step 3073 is the same as that of step 303 above, and will not be repeated here.

[0124] In step 3074, when the sixth recognition result indicates that the first speech does not include the target speech to be recognized, or the seventh recognition result indicates that there is no second execution instruction matching the target speech to be recognized, in response to the wake-up instruction, the fourth speech is recognized in the third way to obtain the eighth recognition result.

[0125] Here, the specific implementation process of step 3074 is the same as that of step 304 above, and will not be repeated here.

[0126] In step 3075, when the eighth recognition result characterizes the fourth speech as including valid semantics, the operation indicated by the fourth speech is performed.

[0127] Here, the specific implementation process of step 3075 is the same as that of step 305 above, and will not be repeated here.

[0128] Based on steps 306 and 307, multiple fourth voices without wake words are received. For each fourth voice, when there is a second execution instruction that matches the target voice to be recognized, the operation indicated by the second execution instruction can be executed without using a wake word. When there is no second execution instruction that matches the target voice to be recognized, in response to the wake-up instruction of the voice interaction function, when the fourth voice is recognized to include valid semantics, the operation indicated by the fourth voice is executed. This allows for simultaneous voice interaction operations of multiple voices, improving the efficiency of voice interaction between multiple voices.

[0129] In some embodiments, when a fifth voice command is received during the execution of an operation, a prompt message is output to indicate that an operation is being performed.

[0130] Here, after the voice interaction function is activated, upon receiving a fifth voice command, the operation indicated by the fifth voice command is executed, and a prompt message for the operation indicated by the fifth voice command is output. The prompt message can be a voice broadcast indicating that the operation indicated by the fifth voice command is being executed. Alternatively, multiple fifth voice commands can be received, and the operation indicated by each fifth voice command can be executed sequentially, with a prompt message for each operation being output sequentially.

[0131] In this embodiment of the application, when a fifth voice command is received, prompt information indicating the operation indicated by the fifth voice command is output, which can improve the efficiency of obtaining voice interaction content.

[0132] In some embodiments, the voice interaction method provided in this application can be applied to the voice interaction system of an in-vehicle terminal. In the in-vehicle terminal's voice interaction system, when a user inputs a first voice (e.g., "Please play song XX"), the first voice is first recognized to obtain a first recognition result. The first voice does not include a wake-up word for activating the voice interaction function. Then, when the first recognition result indicates that the first voice includes the voice to be recognized, the voice to be recognized is second recognized to obtain a second recognition result. Furthermore, when the second recognition result indicates that a first execution instruction matching the voice to be recognized exists, the operation indicated by the first execution instruction is executed. When the first recognition result indicates that the first voice does not include the voice to be recognized, or the second recognition result indicates that no first execution instruction matching the voice to be recognized exists, in response to the wake-up instruction of the voice interaction function, the first voice is third recognized to obtain a third recognition result. Furthermore, when the third recognition result indicates that the first voice includes valid semantics, the operation indicated by the first voice is executed. Thus, 1) when there is a first execution instruction that matches the first speech to be recognized, the operation indicated by the first execution instruction can be executed without using a wake-up word, thereby realizing a voice interaction mode without a wake-up word; 2) when there is no first execution instruction that matches the first speech to be recognized, in response to the wake-up instruction of the voice interaction function, when the first speech is recognized to include valid semantics, the operation indicated by the first speech is executed, thereby realizing a voice interaction mode that supports issuing voice commands first and then waking up; based on this, the voice interaction mode is added through the embodiments of this application, thereby improving the voice interaction efficiency of the vehicle terminal.

[0133] Applying the above embodiments of this application, a first recognition is performed on the first speech to obtain a first recognition result. The first speech does not include a wake-up word for activating the voice interaction function. Then, when the first recognition result indicates that the first speech includes the speech to be recognized, a second recognition is performed on the speech to be recognized to obtain a second recognition result. Then, when the second recognition result indicates that there is a first execution instruction matching the speech to be recognized, the operation indicated by the first execution instruction is executed. When the first recognition result indicates that the first speech does not include the speech to be recognized, or the second recognition result indicates that there is no first execution instruction matching the speech to be recognized, in response to the wake-up instruction of the voice interaction function, a third recognition is performed on the first speech to obtain a third recognition result. Then, when the third recognition result indicates that the first speech includes valid semantics, the operation indicated by the first speech is executed. Thus, 1) when there is a first execution instruction that matches the first speech to be recognized, the operation indicated by the first execution instruction can be executed without using a wake-up word, thereby realizing a voice interaction mode without a wake-up word; 2) when there is no first execution instruction that matches the first speech to be recognized, in response to the wake-up instruction of the voice interaction function, when the first speech is recognized to include valid semantics, the operation indicated by the first speech is executed, thereby realizing a voice interaction mode that supports issuing voice commands first and then waking up; based on this, the voice interaction mode is added through the embodiments of this application, thereby improving the efficiency of voice interaction.

[0134] The following will describe an exemplary application of the voice interaction method provided in the embodiments of this application in an in-vehicle voice interaction system scenario.

[0135] Currently, when a user issues a request command to the voice interaction system of an in-vehicle device, they need to say a wake-up word to activate the voice interaction system before issuing the request command. Furthermore, only one user is allowed to interact with the voice interaction system at a time. Related technologies only support the mode of activating the voice interaction system first and then issuing the request command; they do not support the mode of issuing the request command first and then activating the voice interaction system.

[0136] This application proposes a voice interaction method to address the problems existing in related technologies, which includes the following improvements compared to related technologies:

[0137] (1) The voice command executed through whitelist management (i.e. the voice command in the first voice above) If the first voice command issued by the user passes the rejection model (the first recognition process above), it is then determined whether the voice command in the first voice is a whitelist command (the second recognition process above). If the voice command in the first voice is a whitelist command (i.e. the first execution command matched with the voice to be recognized in the first voice above), the voice command in the first voice is executed. It supports waking up the voice interaction system without saying a wake word, and the voice interaction system can automatically respond to the voice command issued by the user.

[0138] (2) When multiple people issue voice commands simultaneously, the voice interaction system will execute the voice commands in the voice commands sequentially. If the voice interaction system is currently playing a voice command, it will output a prompt tone to execute the voice command upon receiving the voice command. It supports a multi-person wake-up-free voice interaction system (wake-up-free means not saying a wake-up word, but waking up the voice interaction system by directly issuing voice commands).

[0139] (3) If the first speech fails to pass the rejection model or the speech command in the first speech is not in the whitelist, the speech command in the first speech is cached first. In response to the second speech including the wake word, the speech command in the cached first speech is detected as a valid command. When the speech command in the cached first speech is a valid command issued to the voice interaction system, the speech command in the cached first speech is executed, so as to issue the speech command first and then say the wake word to wake up the voice interaction system.

[0140] Example, reference Figure 4 , Figure 4 This is a schematic diagram of the first process of the wake-up-free voice interaction system provided in the embodiments of this application. Combined with... Figure 4 The following steps will be used to explain the wake-up-free implementation scheme of the voice interaction system provided in the embodiments of this application.

[0141] In step 401, input voice.

[0142] The user inputs a voice command to begin the wake-up-free process.

[0143] In step 402, it is determined whether there is a conflict scenario where voice is unavailable.

[0144] If the currently input first voice is in a voice unavailability conflict scenario (the target device state mentioned above), then proceed to step 403 and do not respond to the command, i.e., do not respond to the voice command in the first voice. If the currently input first voice is not in a voice unavailability conflict scenario, then proceed to step 405 for multi-channel voice activation detection, i.e., voice activity detection. Examples of voice unavailability conflict scenarios are shown in area 404: 1. Parking; 2. Bluetooth phone call; 3. Emergency call system; 4. Voice interactive game; 5. Voiceprint recording process; 6. Driving navigation broadcast; 7. Voice input method in operation; 8. The central control screen's audio zone is on.

[0145] In step 406, the multi-channel automatic speech recognition model is processed.

[0146] The automatic speech recognition model can be a hybrid offline / online speech recognition model (ASR). The first speech is input into the multi-channel speech recognition model (ASR), and the first speech is converted into speech text (the process of speech recognition of the first speech mentioned above).

[0147] In step 407, it is determined whether the short audio file is invalid.

[0148] If the first speech is an invalid short audio file, proceed to step 408 and discard it; if the first speech is not an invalid short audio file, proceed to step 409 and input the rejection model. For example, the rejection model can be an offline Natural Language Understanding (NLU) model. The rejection model is mainly used to reject background noise, non-human-computer interaction speech content, or cluttered speech content.

[0149] In step 430, it is determined whether recognition is rejected.

[0150] If the voice command is invalid or irrelevant (as described in the first recognition process above), proceed to step 418 to cache the audio; for example, cache the voice text of the first voice command. If the first voice command is not invalid or irrelevant, proceed to step 411 to determine if it is on the whitelist.

[0151] In step 411, it is determined whether the item is on the whitelist.

[0152] If the voice command in the first speech is not in the whitelist (as described in the second recognition process above), proceed to step 418 to cache the audio, or proceed to step 406 for multi-channel automatic speech recognition model processing. If the voice command in the first speech is in the whitelist, proceed to step 412 to activate the voice interaction system and enter the voice command response state, i.e., the voice command response state in the first speech. Then proceed to step 413 to begin executing the command, i.e., begin executing the voice command in the first speech.

[0153] In step 414, the user enters a wake-up free dialogue state.

[0154] After being woken up, the voice interaction system can engage in free conversation with the user. After the conversation ends, it proceeds to step 415, where the voice exits, and then proceeds to step 416, entering a non-wake-up standby state. The wake-up-free process of the voice interaction system ends.

[0155] Continue to refer to Figure 4 This application embodiment also provides a post-wake-up judgment logic (the third recognition process described above), as shown in region 417. If the first voice is invalid voice or irrelevant voice content, or if the voice command in the first voice is not in the whitelist, then the post-wake-up judgment logic is performed on the first voice.

[0156] In step 418, the audio is cached; for example, the audio text of the first speech is cached.

[0157] In step 419, it is determined whether the user is woken up within 2 seconds.

[0158] If the user does not use a second voice, including a wake-up word, to wake up the voice interaction system within the 2-second interval after issuing the first voice (the second voice detection process described above), then proceed to step 431 to automatically clear the cached audio; if the user uses a second voice, including a wake-up word, to wake up the voice interaction system within the 2-second interval after issuing the first voice, then proceed to step 421 to determine whether it is a pre-triggered single-trigger wake-up.

[0159] In step 421, it is determined whether it is a pre-triggered single-trigger wake-up.

[0160] If the voice interaction system is woken up using a pre-triggered single-trigger command (the second voice includes a voice command) (as described in the fifth recognition process above), then proceed to step 422, respond to the single-trigger command, and automatically clear the cached audio. Then proceed to step 413 to begin executing the command, i.e., begin executing the voice command in the first voice. If the voice interaction system is not woken up using a pre-triggered single-trigger command, then proceed to step 423, determining that voice activation was triggered within 1000ms after wake-up, i.e., the process of voice activity detection within the second duration after the second time point.

[0161] In step 423, it is determined that a voice activation detection was triggered within 1000ms after wake-up.

[0162] If the voice interaction system triggers voice activation detection within 1000ms after wake-up, it proceeds to step 424 to begin normal command recognition and automatically clears the cached audio, i.e., it begins recognizing the third voice generated by voice activation detection. Then it proceeds to step 413 to begin executing the command, i.e., it begins executing the voice command in the third voice. If the voice interaction system does not trigger voice activation detection within 1000ms after wake-up, it proceeds to step 425 to determine whether the semantics are invalid.

[0163] In step 425, it is determined whether the semantics are invalid.

[0164] If the voice command in the cached first audio recording is not a valid voice command issued to the voice interaction system, proceed to step 426 to automatically clear the cached audio. If the voice command in the cached first audio recording is a valid voice command issued to the voice interaction system, proceed to step 427 to respond to the post-wake-up command, i.e., the wake-up command, and then proceed to step 428 to activate the voice interaction system and perform a wake-up recognition process.

[0165] Example, reference Figure 5 , Figure 5 This is a schematic diagram of the second process of the voice interaction system without wake-up provided in the embodiments of this application.

[0166] In step 501, front-end multi-channel audio processing is performed.

[0167] Turn on the 6-channel microphone device of the vehicle terminal, acquire the first voice input from the user, and perform usability voice discrimination processing on the first voice input from each channel.

[0168] In step 502, front-end multi-channel voice activation detection is performed.

[0169] For each input speech, voice signal detection is performed to distinguish the speech to be recognized from various background noise signals.

[0170] In step 503, the multi-channel speech recognition model performs recognition.

[0171] The first voice input from each channel is fed into the offline speech recognition model ASR, which converts the first voice input into speech-to-text.

[0172] In step 504, the rejection model is identified.

[0173] The rejection model can be an offline Natural Language Understanding (NLU) model that identifies the speech text of the first speech.

[0174] In step 505, the natural language understanding model identifies.

[0175] Using an offline Natural Language Understanding (NLU) model, semantic results are identified in the speech text of the first speech.

[0176] In step 506, voice command whitelist management is performed.

[0177] The voice command whitelist includes voice commands to be executed. The semantic result is used to determine the command intent contained in the first voice command, and thus determine whether the voice command in the first voice command is a voice command to be executed.

[0178] In step 507, dialogue management is performed.

[0179] The voice command in the first voice prompt is the voice command to be executed. The voice interaction system is activated, and the voice command in the first voice prompt is executed, thus entering the voice interaction state.

[0180] In step 508, the voice command is executed and broadcast.

[0181] The voice interaction system receives the voice command from the user's first voice command, executes the voice command, and plays the prompt tone indicating the operation indicated by the voice command.

[0182] For example, when multiple users issue voice commands simultaneously, the voice interaction system can execute each voice command in sequence and play the prompt tone for each executed voice command. Multiple users can interact with the voice interaction system via voice.

[0183] In the aforementioned application scenarios of the in-vehicle voice interaction system without wake-up, the voice commands issued by the user to the in-vehicle terminal can be activated without saying a wake-up word by judging through the rejection model and whitelist commands. The voice interaction system directly executes the voice commands issued by the user. Through the post-wake-up judgment logic, the voice command can be issued first, and then the wake-up word can be said to activate the voice interaction system. When multiple people issue voice commands at the same time, the voice interaction system will execute the voice commands in sequence and play the prompt tone, thus supporting a multi-person wake-up-free voice interaction system.

[0184] The following description continues to illustrate the exemplary structure of the voice interaction device 455 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the voice interaction device 455 in the memory 450 may include: a first recognition module 4551, used to perform a first recognition on a first voice to obtain a first recognition result, wherein the first voice does not include a wake-up word for activating the voice interaction function; a second recognition module 4552, used to perform a second recognition on the voice to be recognized when the first recognition result indicates that the first voice includes the voice to be recognized, to obtain a second recognition result; a first execution module 4553, used to execute the operation indicated by the first execution instruction when the second recognition result indicates that there is a first execution instruction matching the voice to be recognized; a third recognition module 4554, used to perform a third recognition on the first voice in response to the wake-up instruction of the voice interaction function to obtain a third recognition result when the first recognition result indicates that the first voice does not include the voice to be recognized, or the second recognition result indicates that there is no first execution instruction matching the voice to be recognized; and a second execution module 4555, used to execute the operation indicated by the first voice when the third recognition result indicates that the first voice includes valid semantics.

[0185] In some embodiments, the first recognition module 4551 is further configured to perform a first recognition on the first voice, and before obtaining the first recognition result, determine the device state of the electronic device including the voice interaction function; when the device state is the target device state, determine not to perform the first recognition on the first voice.

[0186] In some embodiments, when the device state is not the target device state, the first recognition module 4551 is further configured to perform voice activity detection on the first speech and obtain a detection result; when the detection result indicates that there is voice activity, perform voice recognition on the first speech to obtain the first speech text of the first speech; and perform first recognition on the first speech text to obtain a first recognition result.

[0187] In some embodiments, the first recognition module 4551 is further configured to perform a fourth recognition on the first speech based on the first speech text to obtain a fourth recognition result; when the fourth recognition result indicates that the first speech is not invalid speech, perform a first recognition on the first speech text to obtain a first recognition result; when the fourth recognition result indicates that the first speech is invalid speech, discard the first speech.

[0188] In some embodiments, the second recognition module 4552 is further configured to determine the voice instructions included in the speech to be recognized and obtain M execution instructions, where M is a positive integer; match the voice instructions with each execution instruction; when there is an execution instruction among the M execution instructions that matches the voice instructions, determine a second recognition result that represents the existence of a first execution instruction that matches the speech to be recognized; when there is no execution instruction among the M execution instructions that matches the voice instructions, determine a second recognition result that represents the absence of a first execution instruction that matches the speech to be recognized.

[0189] In some embodiments, the third recognition module 4554 is further configured to, in response to a wake-up command of the voice interaction function, perform a third recognition on the first voice, and before obtaining the third recognition result, determine a first time point for acquiring the first voice; detect the second voice within a first duration after the first time point, the second voice including a wake-up word; and determine that a wake-up command has been received when the second voice is detected within the first duration after the first time point.

[0190] In some embodiments, the third recognition module 4554 is further configured to perform a fifth recognition on the second speech when the second speech is detected within a first duration after the first time point, and obtain a fifth recognition result; when the fifth recognition result indicates that the second speech does not include a voice command, determine that a wake-up command has been received; when the fifth recognition result indicates that the second speech includes a voice command, execute the operation indicated by the voice command.

[0191] In some embodiments, the third recognition module 4554 is further configured to: determine a second time point for acquiring the second speech when the fifth recognition result indicates that the second speech does not include a speech instruction; perform speech activity detection within a second duration after the second time point; determine that a wake-up instruction has been received when no speech activity is detected within the second duration after the second time point; acquire a third speech that generates the speech activity when speech activity is detected within the second duration after the second time point; and execute the operation indicated by the third speech.

[0192] In some embodiments, the third recognition module 4554 is further configured to perform a third recognition on the first speech in response to a wake-up command of the voice interaction function, and cache the first speech before obtaining the third recognition result; when the third recognition result indicates that the first speech does not include valid semantics, the cached first speech is deleted.

[0193] In some embodiments, the second execution module 4555 is further configured to receive a plurality of fourth voices, each fourth voice not including a wake-up word; and to serially perform the following processing on each fourth voice: perform a first recognition on the fourth voice to obtain a sixth recognition result; when the sixth recognition result indicates that the fourth voice includes a target voice to be recognized, perform a second recognition on the target voice to be recognized to obtain a seventh recognition result; when the seventh recognition result indicates that there is a second execution instruction matching the target voice to be recognized, execute the operation indicated by the second execution instruction; when the sixth recognition result indicates that the first voice does not include the target voice to be recognized, or the seventh recognition result indicates that there is no second execution instruction matching the target voice to be recognized, in response to a wake-up instruction, perform a third recognition on the fourth voice to obtain an eighth recognition result; and when the eighth recognition result indicates that the fourth voice includes valid semantics, execute the operation indicated by the fourth voice.

[0194] In some embodiments, the second execution module 4555 is further configured to output a prompt message when a fifth voice command including a voice instruction is received during the execution of an operation. The prompt message is used to indicate that an operation is being performed.

[0195] This application provides a computer program product, which includes computer-executable instructions or a computer program stored in a computer-readable storage medium. The processor of an electronic device reads the computer-executable instructions or computer program from the computer-readable storage medium and executes the computer-executable instructions or computer program, causing the electronic device to perform the voice interaction method described above in this application.

[0196] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the voice interaction method provided in this application. For example, ... Figure 3A The voice interaction method is shown.

[0197] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0198] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0199] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hypertext Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0200] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0201] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A voice interaction method, characterized in that, The method includes: A first speech is subjected to a first recognition to obtain a first recognition result, wherein the first speech does not include a wake word used to activate the voice interaction function; When the first recognition result indicates that the first speech includes the speech to be recognized, the speech to be recognized is subjected to a second recognition to obtain a second recognition result. When the second recognition result indicates the existence of a first execution instruction that matches the speech to be recognized, the operation indicated by the first execution instruction is executed; When the first recognition result indicates that the first speech does not include the speech to be recognized, or the second recognition result indicates that there is no first execution instruction matching the speech to be recognized, in response to the wake-up instruction of the voice interaction function, the first speech is subjected to third recognition to obtain the third recognition result; When the third recognition result indicates that the first speech contains valid semantics, the operation indicated by the first speech is executed.

2. The method according to claim 1, characterized in that, Before performing the first recognition on the first speech and obtaining the first recognition result, the method further includes: Determine the device status of the electronic device that includes the voice interaction function; When the device state is not the target device state, the operation of performing the first recognition on the first voice is determined to be executed.

3. The method according to claim 2, characterized in that, When the device state is not the target device state, the method further includes: Speech activity detection is performed on the first speech, and the detection result is obtained; When the detection result indicates the presence of speech activity, speech recognition is performed on the first speech to obtain the first speech text of the first speech. The first recognition of the first speech to obtain the first recognition result includes: The first speech text is subjected to a first recognition, and a first recognition result is obtained.

4. The method according to claim 3, characterized in that, The first recognition of the first speech text to obtain a first recognition result includes: Based on the first speech text, the first speech is subjected to a fourth recognition to obtain a fourth recognition result; When the fourth recognition result indicates that the first speech is not invalid speech, the first speech text is subjected to first recognition to obtain the first recognition result. The method further includes: When the fourth recognition result indicates that the first speech is invalid speech, the first speech is discarded.

5. The method according to claim 1, characterized in that, The second recognition of the speech to be recognized, to obtain a second recognition result, includes: Determine the voice commands included in the speech to be recognized, and obtain M execution commands, where M is a positive integer; Match the voice command with each of the execution commands; When there is an execution instruction among the M execution instructions that matches the voice instruction, a second recognition result is determined, which indicates that there is a first execution instruction that matches the voice to be recognized; When there is no execution instruction among the M execution instructions that matches the voice instruction, a second recognition result is obtained, which indicates that there is no first execution instruction that matches the voice to be recognized.

6. The method according to claim 1, characterized in that, Before performing a third recognition on the first voice in response to the wake-up command of the voice interaction function and obtaining the third recognition result, the method further includes: Determine the first time point at which the first voice recording is acquired; The detection of the second speech, including the wake word, is performed within a first duration after the first time point. When the second voice is detected within a first duration after the first time point, it is determined that the wake-up command has been received.

7. The method according to claim 6, characterized in that, The step of determining that the wake-up command has been received when the second voice is detected within a first duration after the first time point includes: When the second speech is detected within a first duration after the first time point, the second speech is subjected to a fifth recognition to obtain a fifth recognition result; When the fifth recognition result indicates that the second voice does not include a voice command, it is determined that the wake-up command has been received; The method further includes: When the fifth recognition result indicates that the second speech includes a voice command, the operation indicated by the voice command is executed.

8. The method according to claim 7, characterized in that, When the fifth recognition result indicates that the second voice does not include a voice command, determining that the wake-up command has been received includes: When the fifth recognition result indicates that the second speech does not include a speech command, a second time point for collecting the second speech is determined. Voice activity detection is performed within a second duration following the second time point; If no voice activity is detected within a second duration after the second time point, it is determined that the wake-up command has been received; The method further includes: When voice activity is detected within a second duration after the second time point, the third voice that generated the voice activity is acquired; Perform the operation indicated by the third voice.

9. The method according to claim 1, characterized in that, Before performing a third recognition on the first voice in response to the wake-up command of the voice interaction function and obtaining the third recognition result, the method further includes: Cache the first audio message; The method further includes: When the third recognition result indicates that the first speech does not contain valid semantics, the cached first speech is deleted.

10. The method according to claim 1, characterized in that, The method further includes: Multiple fourth voice messages are received, each of which does not include the wake word; The following processing is performed sequentially on each of the fourth speech segments: The fourth speech is subjected to a first recognition, resulting in a sixth recognition result; When the sixth recognition result indicates that the fourth speech includes the target speech to be recognized, the target speech to be recognized is subjected to a second recognition to obtain a seventh recognition result. When the seventh recognition result indicates the existence of a second execution instruction that matches the target speech to be recognized, the operation indicated by the second execution instruction is executed; When the sixth recognition result indicates that the first speech does not include the target speech to be recognized, or the seventh recognition result indicates that there is no second execution instruction matching the target speech to be recognized, in response to the wake-up instruction, the fourth speech is subjected to a third recognition to obtain an eighth recognition result; When the eighth recognition result indicates that the fourth speech contains valid semantics, the operation indicated by the fourth speech is executed.

11. The method according to claim 1, characterized in that, The method further includes: When a fifth voice command is received during the execution of the operation, a prompt message is output to indicate that the operation is being performed.

12. A voice interaction device, characterized in that, The device includes: The first recognition module is used to perform a first recognition on the first speech and obtain a first recognition result, wherein the first speech does not include a wake word for activating the voice interaction function; The second recognition module is used to perform a second recognition on the speech to be recognized when the first recognition result indicates that the first speech includes the speech to be recognized, and obtain a second recognition result. The first execution module is configured to execute the operation indicated by the first execution instruction when the second recognition result indicates that there is a first execution instruction that matches the speech to be recognized; The third recognition module is used to perform third recognition on the first speech in response to the wake-up command of the voice interaction function when the first recognition result indicates that the first speech does not include the speech to be recognized, or the second recognition result indicates that there is no first execution instruction matching the speech to be recognized; and obtain the third recognition result. The second execution module is used to execute the operation indicated by the first speech when the third recognition result indicates that the first speech includes valid semantics.

13. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the voice interaction method according to any one of claims 1 to 11.

14. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, the voice interaction method according to any one of claims 1 to 11 is implemented.

15. A computer program product comprising computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, the voice interaction method according to any one of claims 1 to 11 is implemented.

Images