Voice interaction method and electronic device

Abstract

The application provides a voice interaction method and an electronic device. For a first user voice, a voice wake-up module acquires a pre-generated wake-up response audio when a target voice front end point is detected, inserts a target marker frame before the wake-up response audio to form a target audio, sends the target audio to an audio detection management module, and performs audio playback initialization processing. After the audio detection management module detects the target marker frame, the level state of a second register module is set and an interrupt is triggered. After the voice wake-up module detects a target voice rear end point and confirms that the target voice rear end point includes target wake-up information, the level state of a first register module is set and an interrupt is triggered. When the audio detection management module detects interrupts of the two register modules and the level states of the two register modules are both in a first state, the wake-up response audio is sent to an audio playback processing module for playback. In this way, the wake-up response delay is reduced through the wake-up response audio pre-generation, target marker frame identification, and double register module interrupt synchronization mechanism.

Description

Technical Field

[0001] This invention relates to the field of voice interaction technology, and in particular to a voice interaction method and electronic device. Background Technology

[0002] Voice services are now standard features in electronic devices such as vehicles, mobile phones, and watches. These services are primarily based on functions like voice wake-up, speech recognition, text-to-speech (TTS), natural language understanding, and large language models. The voice wake-up function involves multiple processing steps, including acquiring the user's voice, performing acoustic processing such as noise reduction, detecting wake-up based on the user's voice, and generating and playing the wake-up response audio. Since each processing step consumes time, the entire voice wake-up response process is relatively long, resulting in a voice wake-up response latency. The performance metric of voice wake-up response latency has always been a crucial indicator of the quality of voice services. Therefore, reducing voice wake-up response latency is a problem that is currently being explored in the field of voice interaction technology. Summary of the Invention

[0003] The purpose of this invention is to solve the problem of reducing voice wake-up response latency during voice interaction. To address this technical problem, this invention provides a voice interaction method, a voice service system, and an electronic device that can reduce voice wake-up response latency.

[0004] To address the aforementioned technical problems, in a first aspect, embodiments of the present invention disclose a voice interaction method applied to a voice service system. The voice service system includes a voice wake-up module, a text-to-speech module, an audio detection and management module, a first register module, a second register module, and an audio playback processing module.

[0005] The method includes: a voice wake-up module performing voice front-end point detection processing on the first user's voice; if a target voice front-end point is detected, determining that the first user's voice is the voice service wake-up voice; obtaining the wake-up response audio from the text-to-speech module and performing audio playback initialization processing; inserting a target marker frame before the wake-up response audio to obtain the target audio; sending the target audio to the audio detection management module; and the wake-up response audio being pre-generated and stored by the text-to-speech module based on the target response information after the voice service system is started.

[0006] The voice wake-up module performs voice trailing point detection processing on the first user's voice. If the first user's voice includes target wake-up information, the module sets the level state corresponding to the first register module to the first state and triggers an interrupt in the first register module.

[0007] The audio detection and management module performs marker frame detection processing on the target audio. If it is determined that the target audio includes the target marker frame, it determines that the target audio is the audio corresponding to the wake-up response, stores the wake-up response audio included in the target audio, sets the level state corresponding to the second register module to the first state, and triggers an interrupt in the second register module.

[0008] When the audio detection and management module detects an interrupt triggered by the first register module, it obtains the corresponding level state of the first register module. When it detects an interrupt triggered by the second register module, it obtains the corresponding level state of the second register module. If the level state of both the first and second register modules is in the first state, the stored wake-up response audio is sent to the audio playback processing module, and the level states of both the first and second register modules are set to the second state.

[0009] The audio playback processing module plays the wake-up response audio.

[0010] By adopting the above technical solution, the wake-up response audio is pre-generated and stored by the text-to-speech module after the voice service system starts. The voice wake-up module performs voice front-end point detection processing on the first user's voice. If the target voice front-end point is detected, the pre-generated wake-up response audio can be directly obtained from the text-to-speech module, avoiding the need to generate the wake-up response audio after the voice wake-up module has completed its detection, thus reducing the computational overhead of generating the wake-up response audio. Therefore, using pre-generated wake-up response audio significantly reduces wake-up response time, thereby reducing wake-up response latency and enabling more timely wake-up responses, thus improving the user experience.

[0011] Furthermore, a target marker frame is inserted before the wake-up response audio to obtain the target audio. This target audio is then sent to the audio detection management module for marker frame detection, enabling the module to accurately identify and verify the wake-up response audio and prevent misplay or incorrect responses. Simultaneously, the voice wake-up module detects the beginning and end points of the user's voice, ensuring that only complete user voice commands are processed, reducing false triggering or false wake-ups caused by factors such as background noise interference.

[0012] Furthermore, an event-driven asynchronous processing mechanism is implemented using the dual-register interrupt mechanism corresponding to the first register module and the second register module. The wake-up response audio is played only when both conditions are met simultaneously: the voice wake-up information detection is complete and the voice tag frame audio is ready. This ensures that all steps of the operation are executed completely, avoids errors caused by partial execution, and reduces false triggering or false wake-up.

[0013] In summary, based on the voice interaction method provided in this application, the above-mentioned wake-up response audio pre-generation, target marker frame recognition, and dual register module interrupt synchronization mechanism can reduce the voice wake-up response latency while ensuring the accuracy of the voice wake-up response, thereby effectively improving the user experience.

[0014] In one possible embodiment of this application, the voice service system further includes a voice acquisition module and a voice processing module. The method further includes obtaining the first user voice in the following manner: the voice acquisition module acquires the initial user voice after the voice service system is started, and sends the initial user voice to the voice processing module; the voice processing module performs corresponding acoustic processing on the initial user voice to obtain the first user voice.

[0015] By adopting the above technical solution, the user voice input link is clearly defined, namely, acquiring the user's voice, performing acoustic processing on the user's voice, and inputting it into the voice wake-up module for subsequent wake-up operations. The accuracy of front-end / back-end point detection is improved through acoustic preprocessing, thereby improving the robustness of the voice service system.

[0016] In one possible implementation of this application, acoustic processing includes audio noise reduction processing and echo cancellation processing.

[0017] In one possible implementation of this application, the target wake-up information may be a wake-up word.

[0018] By adopting the above technical solution, the ambient noise interference can be suppressed through audio noise reduction processing, which can improve the wake-up word recognition rate; echo cancellation processing of audio can prevent the secondary acquisition of audio, such as audio played by the audio playback processing module, from causing false wake-ups, thereby improving system reliability.

[0019] In one possible implementation of this application, the target marker frame is an audio frame composed of hexadecimal integer values ​​and appears as a spiked waveform on an oscilloscope.

[0020] By adopting the above technical solution, a hardware-recognizable marker frame consisting of a hexadecimal integer value is inserted before the wake-up response audio, avoiding software decoding delay. The unique waveform of the marker frame can prevent accidental triggering due to environmental noise.

[0021] In one possible implementation of this application, the voice wake-up module sends the target audio to the audio detection management module, including: the voice wake-up module sends the target audio to the audio detection management module in the form of streaming audio.

[0022] By employing the above technical solution, target audio is streamed to the audio detection and management module, enabling dynamic resource scheduling and avoiding memory peak pressure caused by transmitting large audio blocks. Furthermore, this allows the audio detection and management module to simultaneously receive and detect marker frames, further reducing overall wake-up response latency.

[0023] In one possible implementation of this application, the method further includes: a voice wake-up module performing voice trailing-end point detection processing on the first user's voice; if the first user's voice does not include target wake-up information when a target voice trailing-end point is detected, then the level state corresponding to the second register module is obtained; if the level state corresponding to the second register module is the first state, then the second register module is triggered to interrupt, and the level state corresponding to the second register module is adjusted to the second state; otherwise, the second register module is waited for the second register module to trigger an interrupt before the level state corresponding to the second register module is adjusted to the second state.

[0024] By adopting the above technical solution, when no wake-up word is detected, the level state corresponding to the second register module is adjusted to the second state to prevent residual interrupt signals from causing subsequent malfunctions and to ensure system stability.

[0025] In one possible implementation of this application, the method further includes: if the audio detection management module detects that the level state corresponding to the first register module is the second state, and / or the level state corresponding to the second register module is the second state, then the stored wake-up response audio is cleared, the audio playback processing is stopped, and the level states corresponding to the first register module and the second register module are both set to the second state.

[0026] By employing the above technical solution, reliable triggering of playback commands is achieved through AND gate logic decisions on the level states of the first and second register modules. In abnormal states, audio data is cleared and registers are reset to avoid resource consumption or erroneous playback.

[0027] In one possible implementation of this application, the first state is a register level signal value of 1, and the second state is a register level signal value of 0.

[0028] In one possible embodiment of this application, the method further includes: a voice acquisition module acquiring non-initial user voice after the voice service system is started, and sending the non-initial user voice to a voice processing module; the voice processing module performing corresponding acoustic processing on the non-initial user voice to obtain second user voice, and inputting the second user voice to an audio detection management module; the audio detection management module performing marker frame detection processing on the second user voice, and if it is determined that the second user voice does not include a target marker frame, determining that the second user voice is a voice service instruction voice, and performing corresponding instruction response processing based on the second user voice.

[0029] Using the above technical solution, for user speech (i.e., command speech) without target marker frames, the audio detection and management module can determine that it is command speech for the voice service and initiate corresponding response processing. This process relies on the existing digital signal processing link, without the need for additional complex modules. While reusing the original audio processing architecture, it reduces redundant judgment steps in command recognition and shortens the delay from receiving command speech to executing the response.

[0030] Secondly, embodiments of the present invention disclose a voice service system, which includes a voice wake-up module, a text-to-speech module, an audio detection and management module, a first register module, a second register module, and an audio playback processing module.

[0031] The voice wake-up module is used to perform voice front-end point detection processing on the first user's voice. When the target voice front-end point is detected, the first user's voice is determined to be the voice service wake-up voice. The wake-up response audio is obtained from the text-to-speech module, and audio playback initialization processing is performed. The target marker frame is inserted before the wake-up response audio to obtain the target audio. The target audio is sent to the audio detection management module. The wake-up response audio is pre-generated and stored by the text-to-speech module based on the target response information after the voice service system is started.

[0032] The voice wake-up module is also used to perform voice trailing point detection processing on the first user's voice. If the first user's voice includes target wake-up information, the level state corresponding to the first register module is set to the first state, and the first register module is triggered to interrupt.

[0033] The audio detection management module is used to perform marker frame detection processing on the target audio. If it is determined that the target audio includes the target marker frame, the target audio is determined to be the audio corresponding to the wake-up response. The wake-up response audio included in the target audio is stored, and the level state corresponding to the second register module is set to the first state, and the second register module is triggered to interrupt.

[0034] The audio detection management module is also used to obtain the level state corresponding to the first register module when an interrupt triggered by the first register module is detected, and to obtain the level state corresponding to the second register module when an interrupt triggered by the second register module is detected. If the level state corresponding to the first register module is the first state and the level state corresponding to the second register module is the first state, the stored wake-up response audio is sent to the audio playback processing module, and the level states corresponding to the first register module and the second register module are both set to the second state.

[0035] The audio playback processing module is used to play the wake-up response audio.

[0036] In one possible embodiment of this application, the voice service system further includes a voice acquisition module and a voice processing module. The voice acquisition module is used to acquire the initial user voice after the voice service system is started, and send the initial user voice to the voice processing module. The voice processing module is used to perform corresponding acoustic processing on the initial user voice to obtain the first user voice.

[0037] Thirdly, embodiments of this application provide an electronic device, including: a processor and a memory communicatively connected to the processor; the memory stores a computer program; the processor executes the computer program stored in the memory to enable the electronic device to implement the aforementioned voice interaction method.

[0038] It is understood that the beneficial effects of the second and third aspects mentioned above can also be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0039] To more clearly illustrate the technical solution of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0040] Figure 1 This is a schematic diagram of a voice service system provided in an embodiment of the present invention;

[0041] Figure 2 This is a flowchart illustrating a voice interaction method provided in an embodiment of the present invention;

[0042] Figure 3 This is another structural schematic diagram of the voice service system provided in the embodiments of the present invention;

[0043] Figure 4 This is a schematic diagram of a hardware architecture corresponding to the voice service system provided in an embodiment of the present invention;

[0044] Figure 5 This is a schematic diagram of a GPIO connection provided in an embodiment of the present invention;

[0045] Figure 6 This is a flowchart illustrating a voice interaction method implemented on the Android platform according to an embodiment of the present invention.

[0046] Figure 7 This is a schematic diagram of a target marker frame provided in an embodiment of the present invention;

[0047] Figure 8 This is a schematic diagram of the register level signal transition process provided in an embodiment of the present invention;

[0048] Figure 9This is a schematic diagram comparing the time slices of traditional voice wake-up and the voice wake-up solution provided in this embodiment of the invention;

[0049] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0050] Currently, traditional voice service systems that implement voice wake-up functionality mainly include a voice acquisition module, a voice processing module, a voice wake-up module, a TTS (Text-to-Speech) module, and an audio playback processing module. The link in this system to implement the voice wake-up response is a serialized process. Specifically, the voice acquisition module acquires the user's voice and sends it to the voice processing module. The voice processing module performs a series of acoustic processing steps, such as noise reduction, to generate the target voice, which is then sent to the voice wake-up module for wake-up activation detection. If the detection is successful, the voice wake-up module triggers the TTS module to synthesize a wake-up response audio, which is then played back by the audio playback processing module.

[0051] Taking an in-vehicle voice service system as an example, the voice acquisition module may include a microphone device and a digital signal processor (DSP) chip. The microphone device is used to record the user's voice, and the DSP chip is used to convert the user's voice acquired by the microphone device from an audio signal into a digital signal. The audio playback processing module may include a DSP chip, an audio amplifier (AMP), and a speaker. The DSP chip is used to convert the audio to be played from an audio signal into a digital signal, the AMP is used to amplify the audio, and the speaker is used to play the audio.

[0052] The process of implementing voice wake-up functionality in an in-vehicle voice service system is as follows: In a traditional in-vehicle environment, the user speaks a customized voice wake-up phrase, such as "Hello, XX," into the microphone device that continuously records audio in the vehicle. The vehicle's DSP chip converts the user's voice from an audio signal into a digital signal and sends it to the in-vehicle audio system. The signal then enters the acoustic processing module corresponding to the voice service for noise reduction and other audio processing. The clean audio is output and sent to the voice wake-up module. The voice wake-up module performs wake-up activation detection, checking for the presence of a wake-up phrase in the audio. If a wake-up phrase is found, the voice service is notified to perform logical processing, triggering the corresponding TTS module to synthesize a response audio. This then enters the audio system's playback process, sequentially sending the sound to the DSP chip and AMP for amplification, and finally to the speaker for playback. The user will hear an audio message, "Hey," from the speaker; this audio is the wake-up response audio.

[0053] The entire voice wake-up response process described above is a serialized flow, involving numerous modules. Each module requires a certain amount of time to perform its corresponding processing within the entire processing chain, resulting in a lengthy response time and a response delay. The larger time slices of wake-up response delay include the latency of the voice wake-up module performing wake-up activation detection, the latency of the voice acquisition module acquiring the user's voice, the latency of the audio playback processing module playing audio, and the latency of the TTS module synthesizing the wake-up response audio. Each unnecessary delay increases the overall wake-up response latency, and these delays are superimposed in the serial link, leading to a significant voice wake-up response delay. Therefore, improving the voice wake-up response latency is urgently needed.

[0054] Based on the above problems, this application proposes a voice service system and a voice interaction method.

[0055] like Figure 1 As shown, the voice service system includes a voice wake-up module, a text-to-speech module, an audio detection and management module, a first register module, a second register module, and an audio playback processing module. Each module handles voice wake-up during voice interaction, primarily performing the following processes.

[0056] The voice wake-up module is used to detect the leading and trailing points of the user's voice. When a target voice leading point is detected, it retrieves the pre-cached wake-up response audio from the text-to-speech module and performs initial playback preparation. It also inserts a marker frame into the cached wake-up response audio to obtain the target audio and sends the target audio to the audio detection and management module to prepare for fast playback.

[0057] The text-to-speech module is used to pre-synthesize wake-up response audio based on information such as the current speaker after the voice service system starts, and cache it in the corresponding memory.

[0058] The first register module and the second register module are used to implement level status indication, and each includes a corresponding register.

[0059] The audio detection and management module detects whether the target audio contains a marker frame. If a marker frame is present, it is considered a pre-buffered wake-up response audio. The corresponding level state of the second register module is set to the first state, and the audio is buffered. Simultaneously, the level states of the first and second register modules are monitored. When both are in the first state, the buffered wake-up response audio is sent to the audio playback processing module; otherwise, the buffer is cleared. Furthermore, the audio detection and management module can be implemented using an Advanced Digital Signal Processor (ADSP), and therefore can also be called an ADSP module.

[0060] The audio playback processing module is used to drive downstream hardware to complete playback based on the received wake-up response audio. It may include the aforementioned DSP chip, AMP, and speaker, etc., which are devices for implementing audio playback.

[0061] In one implementation of this application, the process of implementing voice interaction based on the voice service system includes: when the voice service system starts, the text-to-speech module pre-generates and stores wake-up response audio. When the voice wake-up module detects the user's voice beginning, it acquires the pre-stored wake-up response audio, performs audio playback initialization processing, inserts a target marker frame into the wake-up response audio to form target audio, and transmits the target audio to the audio detection management module. After the audio detection management module detects the target marker frame, it stores the wake-up response audio, sets the level state corresponding to the second register module to the first state, and triggers an interrupt; when the voice wake-up module detects the voice ending and confirms that it contains target wake-up information such as a wake-up word, it sets the state corresponding to the first register module to the first state and triggers an interrupt. When the audio detection management module detects the interruption of both register modules and both of their level states are in the first state, it sends the wake-up response audio to the voice playback processing module for output. This scheme reduces response latency by pre-generating wake-up response audio, setting target marker frame recognition, and using a dual register module interrupt synchronization mechanism. Compared with the traditional serial processing method that requires waiting for the complete voice wake-up to end before generating response audio, the response speed is significantly improved, thus significantly improving the user experience.

[0062] This application provides a voice interaction method for use in a voice service system. For example... Figure 2 As shown, the voice interaction method based on this voice service system includes the following steps.

[0063] In step S100, the voice wake-up module performs voice front-end point detection processing on the first user's voice. If the target voice front-end point is detected, the first user's voice is determined to be the voice service wake-up voice. The wake-up response audio is obtained from the text-to-speech module, and audio playback initialization processing is performed. A target marker frame is inserted before the wake-up response audio to obtain the target audio. The target audio is sent to the audio detection management module. The wake-up response audio is pre-generated and stored by the text-to-speech module based on the target response information after the voice service system is started.

[0064] The target marker frame can be, for example, an audio frame consisting of the hexadecimal integer value 0x7FF0, and it will appear as a spiked waveform on an oscilloscope; that is, the target marker frame can be a spiked audio frame. Of course, the target marker frame can also be a specific digital signal embedded in the first frame of the audio, and its format can be set as needed.

[0065] The target response information is, for example, the predefined text response content of the voice service system, such as the wake-up reply "Hey".

[0066] The voice wake-up module sends the target audio, such as in streaming audio or other forms, to the audio detection and management module.

[0067] In the voice interaction method provided in this application, a hardware-recognizable marker frame, consisting of a hexadecimal integer value, is inserted before the wake-up response audio. This avoids software decoding delays, and the unique waveform of the marker frame prevents false triggering due to environmental noise. By streaming the target audio to the audio detection management module, dynamic resource scheduling is achieved, avoiding memory peak pressure from transmitting large audio blocks. Furthermore, the audio detection management module can simultaneously receive and detect the marker frame, further reducing the overall wake-up response latency.

[0068] In step S200, the voice wake-up module performs voice trailing point detection processing on the first user's voice. If the first user's voice includes target wake-up information, the module sets the level state corresponding to the first register module to the first state and triggers an interrupt in the first register module.

[0069] Among them, the target wake-up information is a specific voice keyword predefined by the voice service system to trigger the voice service, such as phrases like "Hello XX" or "Little X".

[0070] Set the level state corresponding to the first register module to the first state, where the first state is a register level signal value of 1, and the second state is a register level signal value of 0.

[0071] In step S300, the audio detection management module performs marker frame detection processing on the target audio. If it is determined that the target audio includes a target marker frame, the target audio is determined to be the audio corresponding to the wake-up response. The wake-up response audio included in the target audio is stored, and the level state corresponding to the second register module is set to the first state, and the second register module is triggered to interrupt.

[0072] In step S400, when the audio detection management module detects an interrupt triggered by the first register module, it obtains the level state corresponding to the first register module. When it detects an interrupt triggered by the second register module, it obtains the level state corresponding to the second register module. If the level state corresponding to both the first and second register modules is the first state, the stored wake-up response audio is sent to the audio playback processing module, and the level states corresponding to both the first and second register modules are set to the second state.

[0073] In step S500, the audio playback processing module performs playback processing on the wake-up response audio.

[0074] In the voice interaction method provided in this application, the wake-up response audio is pre-generated and stored by the text-to-speech module after the voice service system starts. The voice wake-up module performs voice front-end point detection processing on the first user's voice. If the target voice front-end point is detected, the pre-generated wake-up response audio can be directly obtained from the text-to-speech module, avoiding the need to generate the wake-up response audio after the voice wake-up module has completed the detection, thus reducing the computational overhead of generating the wake-up response audio. Therefore, using the pre-generated wake-up response audio significantly reduces the wake-up response time, thereby reducing the wake-up response latency and enabling more timely wake-up responses, thereby improving the user experience.

[0075] Furthermore, a target marker frame is inserted before the wake-up response audio to obtain the target audio. This target audio is then sent to the audio detection management module for marker frame detection, enabling the module to accurately identify and verify the wake-up response audio and prevent misplay or incorrect responses. Simultaneously, the voice wake-up module detects the beginning and end points of the user's voice, ensuring that only complete user voice commands are processed, reducing false triggering or false wake-ups caused by factors such as background noise interference.

[0076] Furthermore, an event-driven asynchronous processing mechanism is implemented using the dual-register interrupt mechanism corresponding to the first register module and the second register module. The wake-up response audio is played only when both conditions are met simultaneously: the voice wake-up information detection is complete and the voice tag frame audio is ready. This ensures that all steps of the operation are executed completely, avoids errors caused by partial execution, and reduces false triggering or false wake-up.

[0077] Furthermore, such as Figure 3 As shown, in one implementation of this application, the voice service system further includes a voice acquisition module and a voice processing module. The voice interaction method further includes obtaining the aforementioned first user voice in the following manner: the voice acquisition module acquires the initial user voice after the voice service system is started, and sends the initial user voice to the voice processing module; the voice processing module performs corresponding acoustic processing on the initial user voice to obtain the first user voice.

[0078] In one possible implementation of this application, acoustic processing includes audio noise reduction processing and echo cancellation processing. Of course, acoustic processing may also include methods for converting user speech acquired from the microphone into a clearer, more easily recognizable effective signal by the voice wake-up module.

[0079] The voice interaction method provided in this application clearly defines the user voice input chain, namely, acquiring user voice, performing acoustic processing on user voice, and inputting it to the voice wake-up module for subsequent wake-up operations. Acoustic preprocessing improves the accuracy of front-end / back-end detection, thereby enhancing the robustness of the voice service system. Audio noise reduction processing suppresses environmental noise interference, improving the wake-up word recognition rate; echo cancellation processing prevents false wake-ups caused by secondary acquisition of audio, such as audio played by the audio playback processing module, thus improving system reliability.

[0080] In the implementation of this application, the voice interaction method further includes: a voice wake-up module performs voice trailing point detection processing on the first user's voice; if the first user's voice does not include the target wake-up information when the target voice trailing point is detected, the module obtains the level state corresponding to the second register module; if the level state corresponding to the second register module is the first state, the module triggers an interrupt and adjusts the level state corresponding to the second register module to the second state; otherwise, the module waits for the second register module to trigger an interrupt before adjusting the level state corresponding to the second register module to the second state.

[0081] In the implementation of this application, the voice interaction method further includes: if the audio detection management module detects that the level state corresponding to the first register module is the second state, and / or the level state corresponding to the second register module is the second state, then the stored wake-up response audio is cleared, the audio playback processing is stopped, and the level states corresponding to the first register module and the second register module are both set to the second state.

[0082] In the voice interaction method provided in this application, when no wake-up word is detected, the level state corresponding to the second register module is adjusted to the second state to prevent residual interrupt signals from causing subsequent malfunctions and to ensure system stability. Reliable triggering of playback commands is achieved through an AND gate logic decision between the level states of the first and second register modules. In abnormal states, audio data is cleared and registers are reset to avoid resource consumption or incorrect playback.

[0083] In the implementation of this application, the voice interaction method further includes: a voice acquisition module acquiring non-initial user voice after the voice service system is started, and sending the non-initial user voice to a voice processing module; the voice processing module performing corresponding acoustic processing on the non-initial user voice to obtain second user voice, and inputting the second user voice to an audio detection management module; the audio detection management module performing marker frame detection processing on the second user voice, and if it is determined that the second user voice does not include a target marker frame, determining that the second user voice is a voice service instruction voice, and performing corresponding instruction response processing based on the second user voice.

[0084] In the voice interaction method provided in this application, non-initial user voice can be, for example, other interactive command voices following the user's wake-up voice. For user voices without target marker frames (i.e., command voices), the audio detection and management module can determine that it is a command voice for the voice service and initiate corresponding response processing. This process relies on the existing digital signal processing link, without the need for additional complex modules. By reusing the original audio processing architecture, it reduces redundant judgment steps in command recognition and shortens the delay from receiving command voices to executing responses.

[0085] Furthermore, the following description uses a voice service system running on an in-vehicle Android platform as an example to further illustrate the voice service system and method provided in the embodiments of this application.

[0086] like Figure 4 As shown, the voice service system corresponding to the in-vehicle Android platform includes a System-on-Chip (SOC) chip, such as the 8155A. The SOC chip includes a voice service application (SpeechService), a system firmware layer (Framework, FW), a hardware abstraction layer (HAL), an audio driver, an operating system, an advanced digital signal processor (ADSP), a first register, and a second register. Both the first and second registers include general-purpose input / output (GPIO) pins. The pins in the first register can be simply referred to as GPIO(WT), therefore the first register can be simply referred to as the GPIO(WT) register, and the pins in the second register can be simply referred to as GPIO(WTT), therefore the second register can be simply referred to as the GPIO(WTT) register.

[0087] Furthermore, the voice service system also includes an audio processor, an audio amplifier (AMP), and a speaker. The audio processor performs corresponding audio processing, and the audio amplifier amplifies the audio signal. The audio processor can be, for example, a digital signal processor (DSP, such as a SHARC DSP).

[0088] Furthermore, the hardware connectivity layer may include a GPIO Hardware Abstraction Layer (GPIO HAL) and an Audio Hardware Abstraction Layer (Audio HAL). The GPIO HAL is a software layer used to access the GPIO hardware, enabling control of GPIO(WT) and GPIO(WTT). The Audio HAL is a software layer used to access the audio hardware.

[0089] Audio drivers can include Linux audio drivers (Advanced Linux Sound Architecture, ALSA) and system-on-chip drivers (Audio System on Chip, ASoC).

[0090] The operating system can be the QNX Real-time Operating System (QNX).

[0091] The audio amplifier connects to the digital signal processor (DSP) via a Time Division Multiplexing (TDM) interface to transmit and process audio signals. The audio processor also interacts with the advanced digital signal processor (ADC) within the 8155A SOC via the TDM interface. Inside the 8155A SOC, GPIO (WTT) connects to the ADC, enabling specific input / output control and wake-up functions. The ADC communicates with the operating system via a Fast Remote Procedure Call (Fast RPC) channel, facilitating efficient data transmission and command exchange between different modules within the system. The operating system connects to the audio driver to jointly perform audio-related processing and control. Furthermore, the voice service application communicates with the system firmware layer and then with GPIO (WT) and the audio driver through the hardware connection layer, enabling collaborative operation between voice service functions and system hardware.

[0092] Furthermore, the voice service application may include the aforementioned voice processing module, voice wake-up module, and text-to-speech module to implement the corresponding functions.

[0093] Furthermore, the Android platform voice service system also includes a microphone device as a voice acquisition module to acquire user voice. Alternatively, the Android platform voice service system may also include a microphone device and a DSP chip as voice acquisition modules to acquire user voice and convert it from an audio signal to a digital signal for subsequent processing.

[0094] Of course, the Android platform's voice service system also includes other related components such as buses that enable voice interaction.

[0095] Furthermore, such as Figure 5 As shown, in the aforementioned voice service system based on the in-vehicle Android platform, the bus (Inter-Integrated Circuit master, I2C master) is connected and communicates with the ADSP, GPIO(WT), and GPIO(WTT) via the serial clock line (SCL) and serial data line (SDA), respectively. The GPIO HAL is connected to GPIO(WT) and GPIO(WTT) to control and manage the GPIO.

[0096] Based on the aforementioned Android platform voice service system, this application also provides a voice interaction method.

[0097] like Figure 6 As shown, in one embodiment of this application, the voice interaction method includes the following steps. The voice service system is used to implement the corresponding voice service; therefore, starting the voice service system can also be understood as starting the voice service.

[0098] Step S301: After the voice service starts, the TTS module obtains the current speaker information, begins to pre-synthesize the corresponding wake-up TTS audio, i.e., the wake-up response audio, and stores the wake-up TTS audio in the memory of the TTS module in audio data in the form of, for example, byte arrays.

[0099] Furthermore, after the voice service is started, the voice wake-up process is initiated, the recording service and acoustic processing service are opened, the recording is sent to the voice processing module for corresponding acoustic processing, and the acoustically processed voice is sent to the voice wake-up module of the voice service for wake-up detection.

[0100] Step S302: The voice wake-up module of the voice service performs a vad start detection. The voice wake-up module detects a vad start signal in the input user voice, retrieves the pre-synthesized wake-up TTS audio from the memory corresponding to the TTS module, initializes the audio playback processing module of the voice service, and begins preparation for playback.

[0101] Step S303: Insert a spiked audio frame before the pre-buffered wake-up TTS audio to obtain the target audio, and send the target audio to the ADSP in a streaming manner.

[0102] Furthermore, the peak audio frame value for waking up the TTS audio is a hexadecimal integer value: for example, 0x7FF0, such as... Figure 7 As shown, its signal waveform on the oscilloscope is a purple "peaked" audio signal.

[0103] The target audio is written to the ADSP via the Framework and HAL. In the HAL, the level status of GPIO (WTT) and GPIO (WT) is monitored.

[0104] Step S304: The voice wake-up module of the voice service performs backend detection. If a voice wake-up active event is detected, such as... Figure 8 As shown, the GPIO(WT) level is pulled high via the address corresponding to GPIO(WT). If no wake-up word is found after vad end, an inactive signal is sent. The non-wake-up signal sends an inactive notification to obtain the GPIO(WTT) level status. If it is high, an interrupt is triggered, pulling the GPIO(WTT) level low; otherwise, it waits for the GPIO(WTT) level to be high before sending the GPIO(WTT) signal low again.

[0105] Furthermore, the voice wake-up module detects the end of speech via the vad end and whether the user's speech triggers a wake-up word. If triggered, a wake-up notification is generated. Upon receiving the notification, the module can manipulate the address of GPIO (WT) to pull it high, generating an interrupt to notify other modules. If it is not a wake-up, an unative event is sent. First, the level status of GPIO (WTT) is obtained. If it is high, an interrupt is triggered, and the GPIO (WTT) level is pulled low. Otherwise, the module waits for the GPIO (WTT) level to go high before pulling the GPIO (WTT) signal low. (This step of waiting for the GPIO (WTT) level to go high is to prevent the ADSP from being blocked because the wake-up failure occurs before the "peak" audio frame is sent to the ADSP, causing the ADSP to be blocked because it cannot wait for the WT wake-up GPIO level to go high.)

[0106] Step S305: The ADSP determines whether the input audio contains spiked audio frames.

[0107] Step S306: If the ADSP determines that there is no spiked audio frame, it considers it to be non-wake-up audio and performs corresponding processing for non-wake-up audio.

[0108] Furthermore, non-wake-up audio can be user-command audio (such as "turn up the temperature") or audio from a spiked audio frame that is lost or corrupted during transmission. Appropriate processing for non-wake-up audio can involve executing the corresponding command or outputting the corresponding audio, etc.

[0109] Step S307: If a spiked audio frame is detected, it indicates that the audio is a pre-wake-up TTS audio. The GPIO (WTT) pin is pulled high by manipulating its address. The received streaming target audio is then buffered and not transmitted to the Sharcdsp.

[0110] Furthermore, the ADSP continuously receives the target audio sent by the voice wake-up module. Once it detects a peaked audio frame in the target audio, it considers it to be the corresponding voice wake-up audio and buffers subsequent non-peaked audio frame data. Simultaneously, it pulls the GPIO (WTT) pin high by manipulating its address. It also monitors the interrupt level of the GPIO (WT) pin and buffers the continuously written streaming target audio, preventing it from being transmitted to the Sharc DSP. If there is no peaked audio frame, it is considered non-wake-up audio and requires no special processing; the corresponding non-wake-up audio processing is performed.

[0111] Step S308: The ADSP detects the interrupt signal of GPIO(WT), obtains the level state of GPIO(WT), and simultaneously detects the interrupt signal of GPIO(WTT), obtains the level state of GPIO(WTT).

[0112] Furthermore, the ADSP detects the interrupt signal of GPIO(WT) and obtains the level state of GPIO(WT). At the same time, it detects the interrupt of GPIO(WTT) and obtains the level state of GPIO(WTT). The next operation is determined by the level states of the two GPIOs: a high level is 1 and a low level is 0.

[0113] Step S309: If GPIO(WT) or GPIO(WTT) is low, clear the audio buffer in the ADSP. Pull GPIO(WT) and GPIO(WTT) low to reset.

[0114] Step S310: If GPIO(WT) and GPIO(WTT) are high, send the pre-wake-up TTS audio buffered in the ADSP to the audio playback processing module. Pull GPIO(WT) and GPIO(WTT) low to reset.

[0115] Furthermore, if the AND value of the level signals of GPIO(WT) and GPIO(WTT) is 0, i.e., GPIO(WT)&GPIO(WTT) = 0, then the buffered audio data in the ADSP is cleared, and the audio playback action is stopped, i.e., audio playback processing is stopped, and the levels of GPIO(WT) and GPIO(WTT) are pulled low. If the AND value of the level signals of GPIO(WT) and GPIO(WTT) is 1, i.e., GPIO(WT)&GPIO(WTT) = 1, then the buffered wake-up TTS audio data is sent to the voice playback module.

[0116] Step S311: The audio playback processing module plays the wake-up TTS audio.

[0117] Furthermore, the audio playback processing module processes and amplifies the audio using the aforementioned Sharc DSP and AMP before sending it to the speaker for playback. That is, the input audio is processed by the Sharc DSP, then amplified by the AMP, and finally input to the speaker to produce sound.

[0118] The voice interaction method provided in this application triggers the high and low levels of two GPIO pins in the chip's registers through different wake-up events. By pre-synthesizing and buffering TTS audio, and inserting a special marker frame before the first frame of the TTS audio, the original serialized voice wake-up response processing flow is transformed into a parallel one, while reducing processing links to improve voice wake-up response latency. Moreover, it does not require major changes to the original audio architecture, greatly reducing development costs.

[0119] In summary, in the voice interaction method provided in this application, such as Figure 9As shown, the traditional voice wake-up response process first requires front-end and back-end point detection to confirm the voice input, then triggers the wake-up response, and finally synthesizes the wake-up TTS audio in real time and sends it sequentially to the ADSP, SHARC DSP, and AMP for playback. This entire process involves eight steps. The optimized process in this solution eliminates the real-time synthesis stage, using TTS pre-buffering to eliminate the time consumed by real-time wake-up response audio synthesis. The process involves confirming the target audio using GPIO level status, retrieving the buffered wake-up TTS audio when confirmed, and sending it sequentially to the SHARC DSP, ADSP, and AMP for playback, requiring six steps. Therefore, this solution eliminates the time consumed by real-time wake-up response audio synthesis through TTS pre-buffering, triggers hardware-level parallel transmission through marker frames, resulting in a shorter overall response time than the traditional serial process, and avoids false triggers through AND logic on the register's dual pin levels, ensuring playback only when wake-up is successful and the audio is the wake-up audio. This voice interaction method significantly reduces wake-up response time, enabling more timely wake-up responses and improving the user experience.

[0120] This application provides an electronic device, such as... Figure 10 As shown, it includes: a memory 123 for storing a computer program, the computer program including program instructions; a processor 122 for executing the program instructions; and a transceiver 121. This enables the electronic device to perform the voice interaction method as described in the above embodiments.

[0121] Processor 122 executes computer execution instructions stored in memory, causing processor 122 to perform the voice interaction method in the above embodiments. Processor 122 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital data processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0122] The memory 123 is connected to the processor 122 via the system bus and completes communication between them. The memory 123 is used to store computer program instructions.

[0123] Transceiver 121 can be used to obtain the task to be run and its configuration information.

[0124] Furthermore, electronic devices can be vehicles, mobile phones, watches, and other electronic devices.

[0125] It should be noted that, in addition to the specific embodiments described above, those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Although the description of the present invention is presented in conjunction with preferred embodiments, this does not mean that the features of the invention are limited to these embodiments. On the contrary, the purpose of describing the invention in conjunction with the embodiments is to cover other options or modifications that may be derived based on the claims of the present invention. To provide a deep understanding of the invention, many specific details are included in the above description, and the invention may also be implemented without using these details. Furthermore, to avoid confusion or obscuring the focus of the invention, some specific details will be omitted in the description. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0126] It should be noted that in this specification, similar reference numerals and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0127] In the description of this embodiment, it should be noted that the terms "upper", "lower", "inner", "bottom", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship that the product of the invention is usually placed in during use. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting the present invention.

[0128] The terms “first”, “second”, etc., are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0129] In the description of this embodiment, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set up," "connected," and "linked" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this embodiment based on the specific circumstances.

[0130] While the present invention has been illustrated and described with reference to certain preferred embodiments, those skilled in the art should understand that the above description is a further detailed explanation of the invention in conjunction with specific embodiments, and should not be construed as limiting the specific implementation of the invention to these descriptions. Various changes in form and detail can be made by those skilled in the art, including several simple deductions or substitutions, without departing from the spirit and scope of the invention.

Claims

1. A voice interaction method, characterized in that, Applied to a voice service system, the voice service system includes a voice wake-up module, a text-to-speech module, an audio detection and management module, a first register module, a second register module, and an audio playback processing module, the method includes: The voice wake-up module performs voice front-end point detection processing on the first user's voice. If a target voice front-end point is detected, the first user's voice is determined to be the voice service wake-up voice. The wake-up response audio is obtained from the text-to-speech module, and audio playback initialization processing is performed. A target marker frame is inserted before the wake-up response audio to obtain the target audio. The target audio is sent to the audio detection management module. The wake-up response audio is pre-generated and stored by the text-to-speech module based on the target response information after the voice service system is started. The voice wake-up module performs voice trailing point detection processing on the first user's voice. If the first user's voice includes target wake-up information, the module sets the level state corresponding to the first register module to the first state and triggers an interrupt in the first register module. The audio detection management module performs marker frame detection processing on the target audio. If it is determined that the target audio includes the target marker frame, it determines that the target audio is the audio corresponding to the wake-up response, stores the wake-up response audio included in the target audio, sets the level state corresponding to the second register module to the first state, and triggers an interrupt for the second register module. When the audio detection management module detects an interrupt triggered by the first register module, it obtains the level state corresponding to the first register module. When it detects an interrupt triggered by the second register module, it obtains the level state corresponding to the second register module. If the level state corresponding to the first register module is the first state and the level state corresponding to the second register module is the first state, the stored wake-up response audio is sent to the audio playback processing module, and the level states corresponding to the first register module and the second register module are both set to the second state. The audio playback processing module performs playback processing on the wake-up response audio.

2. The method according to claim 1, characterized in that, The voice service system further includes a voice acquisition module and a voice processing module, and the method further includes obtaining the first user's voice through the following means: The voice acquisition module acquires the initial user voice after the voice service system is started, and sends the initial user voice to the voice processing module; The speech processing module performs corresponding acoustic processing on the initial user speech to obtain the first user speech.

3. The method according to claim 2, characterized in that, The acoustic processing includes audio noise reduction and echo cancellation.

4. The method according to claim 3, characterized in that, The target marker frame is an audio frame composed of hexadecimal integer values, and the target marker frame appears as a spiked waveform on an oscilloscope.

5. The method according to claim 4, characterized in that, The voice wake-up module sends the target audio to the audio detection and management module, including: The voice wake-up module sends the target audio to the audio detection and management module in the form of streaming audio.

6. The method according to claim 5, characterized in that, The method further includes: The voice wake-up module performs voice trailing point detection processing on the first user's voice. If the first user's voice does not include the target wake-up information, the module obtains the level state corresponding to the second register module. If the level state corresponding to the second register module is the first state, the module triggers an interrupt and adjusts the level state corresponding to the second register module to the second state. Otherwise, the module waits for the second register module to trigger an interrupt before adjusting the level state corresponding to the second register module to the second state.

7. The method according to claim 6, characterized in that, The method further includes: If the audio detection and management module detects that the level state corresponding to the first register module is the second state, and / or the level state corresponding to the second register module is the second state, then it will clear the stored wake-up response audio, stop the audio playback process, and set the level states corresponding to the first register module and the second register module to the second state.

8. The method according to claim 7, characterized in that, The first state is when the register level signal value is 1, and the second state is when the register level signal value is 0.

9. The method according to any one of claims 2-8, characterized in that, The method further includes: The voice acquisition module acquires non-initial user voice after the voice service system is started, and sends the non-initial user voice to the voice processing module; The speech processing module performs corresponding acoustic processing on the non-initial user speech to obtain the second user speech, and inputs the second user speech into the audio detection and management module. The audio detection and management module performs marker frame detection processing on the second user's voice. If it is determined that the second user's voice does not include the target marker frame, the second user's voice is determined to be a voice service instruction voice, and corresponding instruction response processing is performed based on the second user's voice.

10. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer programs; The processor executes the computer program stored in the memory to enable the electronic device to implement the voice interaction method as described in any one of claims 1-9.

Images