Voice interaction method and electronic equipment

Abstract

The invention provides a voice interaction method and electronic equipment, and the method comprises the steps: obtaining a pre-generated wake-up response audio when a voice wake-up module detects a target voice front end point for a first user voice, inserting a target mark frame in front of the wake-up response audio to form a target audio, and transmitting the target audio to an audio detection management module; audio playing initialization processing is carried out; after the audio detection management module detects the target mark frame, the level state of the second register module is set, and interruption is triggered; when the voice wake-up module detects a target voice rear end point and confirms that target wake-up information is included, the level state of the first register module is set, and interruption is triggered; and when the audio detection management module detects that the two register modules are interrupted and the level state is the first state, the audio detection management module sends a wake-up response audio to the audio playing processing module for playing. In this way, the wake-up response delay is reduced through wake-up response audio pre-generation, target mark frame recognition and a dual-register module interrupt synchronization mechanism.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of voice interaction, and in particular to a voice interaction method and an electronic device. BACKGROUND

[0002] Voice service has been a standard configuration of electronic devices such as vehicles, mobile phones, watches, etc. The service is mainly based on functions such as voice wake-up, voice recognition, text-to-speech (TTS), natural language understanding, large language models, etc. The implementation of the voice wake-up function mainly includes multiple processing links such as user voice acquisition, acoustic processing such as noise reduction on the user voice, wake-up detection based on the user voice, generation and playing of a wake-up response audio, etc. Since each processing link needs to consume a certain amount of time, the entire voice wake-up response process takes a long time, and there is a voice wake-up response delay. The voice wake-up response delay performance indicator is an important indicator for measuring whether the voice service is excellent. Therefore, how to reduce the voice wake-up response delay is a problem that the current voice interaction technical field is exploring. SUMMARY

[0003] The purpose of the present application is to solve the problem of how to reduce the voice wake-up response delay in the voice interaction process. To solve the technical problem, the present application provides a voice interaction method, a voice service system and an electronic device, which can reduce the voice wake-up response delay.

[0004] To solve the above technical problem, in a first aspect, an embodiment of the present application discloses a voice interaction method applied to a voice service system, the voice service system comprising a voice wake-up module, a text-to-speech module, an audio detection management module, a first register module, a second register module and an audio playing processing module.

[0005] The method comprises: the voice wake-up module performing a voice front-end point detection processing on a first user voice, in the case of detecting a target voice front-end point, determining that the first user voice is a voice service wake-up voice, obtaining a wake-up response audio from the text-to-speech module, and performing an audio playing initialization processing, and inserting a target marker frame in front of the wake-up response audio to obtain a target audio, and sending the target audio to the audio detection management module, the wake-up response audio being pre-generated and stored by the text-to-speech module according to target response information after the voice service system is started.

[0006] The voice wake-up module performs a voice back-end point detection processing on the first user voice, in the case of detecting a target voice back-end point, if it is detected that the first user voice includes target wake-up information, setting a level state corresponding to the first register module to a first state, and causing the first register module to trigger an interruption.

[0007] The audio detection management module performs marker frame detection processing on the target audio, determines that the target audio is audio corresponding to the wake-up response in a case where it is determined that the target audio includes a target marker frame, stores the wake-up response audio included in the target audio, sets the level state corresponding to the second register module to the first state, and causes the second register module to trigger an interrupt.

[0008] The audio detection management module acquires the level state corresponding to the first register module in a case where it is detected that the first register module triggers an interrupt, acquires the level state corresponding to the second register module in a case where it is detected that the second register module triggers an interrupt, and sends the stored wake-up response audio to the audio playback processing module and sets the level states corresponding to the first register module and the second register module to the second state if the level state corresponding to the first register module is the first state and the level state corresponding to the second register module is the first state.

[0009] The audio playback processing module performs playback processing on the wake-up response audio.

[0010] According to the above technical solution, the wake-up response audio is generated in advance by the text-to-speech module after the voice service system is started and is stored, the voice wake-up module performs voice front-end point detection processing on the first user voice, and in a case where a target voice front-end point is detected, the pre-generated wake-up response audio can be directly acquired from the text-to-speech module, avoiding the generation of the wake-up response audio after the voice wake-up module completes detection, and reducing the computational overhead of generating the wake-up response audio. Therefore, the pre-generated wake-up response audio significantly reduces the wake-up response time, thereby reducing the wake-up response delay and enabling more timely wake-up response, thereby improving the user experience.

[0011] Further, a target marker frame is inserted before the wake-up response audio to obtain target audio, and the target audio is sent to the audio detection management module for marker frame detection, so that the audio detection management module can accurately identify and verify the wake-up response audio, preventing false playback or false response. At the same time, the voice wake-up module detects the front-end point and the back-end point of the user voice, which can ensure that only complete user voice commands are processed, reducing false triggering or false wake-up caused by factors such as background noise interference.

[0012] Further, the double-register interrupt mechanism corresponding to the first register module and the second register module is used to implement event-driven asynchronous processing, and the wake-up response audio is played only when both the voice wake-up information detection is completed and the voice marker frame audio is ready, ensuring that all steps of the operation are completely executed, avoiding errors caused by partial execution, and reducing false triggering or false wake-up.

[0013] In summary, based on the voice interaction method provided in the embodiments of the present application, through the pre-generation of the wake-up response audio, the target marker frame identification, and the double register module interrupt synchronization mechanism, the accuracy of the voice wake-up response can be ensured while reducing the delay of the voice wake-up response, thereby effectively improving the user experience.

[0014] In a possible implementation of the present application, the voice service system further comprises a voice acquisition module and a voice processing module, and the method further comprises: obtaining the first user voice by: the voice acquisition module acquiring initial user voice after the voice service system is started, and sending the initial user voice to the voice processing module; and the voice processing module performing corresponding acoustic processing on the initial user voice to obtain the first user voice.

[0015] By using the above technical solution, the user voice input link is determined, that is, the user voice is acquired, acoustic processing is performed on the user voice, and the user voice is input to the voice wake-up module for subsequent wake-up operation, thereby improving the accuracy of the front-end point / double-end point detection through acoustic preprocessing, and improving the robustness of the voice service system.

[0016] In a possible implementation of the present application, the acoustic processing comprises audio noise reduction processing and echo cancellation processing.

[0017] In a possible implementation of the present application, the target wake-up information can be a wake-up word.

[0018] By using the above technical solution, the wake-up word recognition rate can be improved by suppressing environmental noise interference through audio noise reduction processing, and the system reliability can be improved by preventing the acquired user voice from including audio played by the audio playback processing module and being re-collected to cause false wake-up through echo cancellation processing.

[0019] In a possible implementation of the present application, the target marker frame is an audio frame composed of a 16-bit integer value, and presents a sharp roof waveform in an oscilloscope.

[0020] By using the above technical solution, the marker frame composed of a 16-bit integer value is inserted in front of the wake-up response audio, which is a hardware recognizable marker frame, thereby avoiding software decoding delay, and the unique waveform of the marker frame can avoid environmental noise false triggering.

[0021] In a possible implementation of the present application, the voice wake-up module sends the target audio to the audio detection management module in the form of streaming audio.

[0022] The technical scheme is adopted, the target audio is transmitted to the audio detection management module through streaming, resource dynamic scheduling is realized, and the memory peak pressure of large audio block transmission is avoided. Moreover, the audio detection management module can receive and detect the marker frame, and the delay of the overall wake-up response is further reduced.

[0023] In one possible implementation of the present application, the method further includes: the voice wake-up module performs voice back-end point detection processing on the first user voice, and in the case where the target voice back-end point is detected, if it is detected that the first user voice does not include the target wake-up information, the level state corresponding to the second register module is obtained, if the level state corresponding to the second register module is the first state, the second register module is triggered to generate an interrupt, and the level state corresponding to the second register module is adjusted to the second state, otherwise, after the second register module triggers the interrupt, the level state corresponding to the second register module is adjusted to the second state.

[0024] The technical scheme is adopted, the level state corresponding to the second register module is adjusted to the second state when the wake-up word is not detected, the residual interrupt signal is prevented from causing subsequent misoperation, and the system stability is ensured.

[0025] In one possible implementation of the present application, the method further includes: if the audio detection management module detects that the level state corresponding to the first register module is the second state and / or the level state corresponding to the second register module is the second state, the stored wake-up response audio is emptied, the audio playing processing is stopped, and the level states corresponding to the first register module and the second register module are both set to the second state.

[0026] The technical scheme is adopted, the level state corresponding to the first register module and the level state corresponding to the second register module are logically decided by the AND gate, the reliable triggering of the playing instruction is realized, the audio data is emptied and the register is reset in the abnormal state, and the resource occupation or the error playing is avoided.

[0027] In one possible implementation of the present application, the first state is that the register level signal value is 1, and the second state is that the register level signal value is 0.

[0028] In one possible implementation of the present application, the method further includes: the voice acquisition module acquires a non-initial user voice after the voice service system is started, and sends the non-initial user voice to the voice processing module; the voice processing module performs corresponding acoustic processing on the non-initial user voice to obtain a second user voice, and inputs the second user voice to the audio detection management module; the audio detection management module performs marker frame detection processing on the second user voice, and in the case where it is determined that the second user voice does not include the target marker frame, the second user voice is determined to be a voice service instruction voice, and corresponding instruction response processing is performed according to the second user voice.

[0029] With the technical solution, for the user voice (i.e. instruction voice) without the target marker frame, the audio detection management module can determine it as the instruction voice of the voice service and start the corresponding response processing. The process relies on the existing digital signal processing link, without adding complex modules, and reduces the redundant judgment steps of instruction recognition and shortens the delay from receiving the instruction voice to executing the response on the basis of multiplexing the original audio processing architecture.

[0030] In a second aspect, embodiments of the present application disclose a voice service system, which comprises a voice wake-up module, a text-to-speech module, an audio detection management module, a first register module, a second register module and an audio playback processing module.

[0031] The voice wake-up module is configured to perform voice front-end point detection on the first user voice, determine the first user voice as a voice service wake-up voice in a case where a target voice front-end point is detected, obtain a wake-up response audio from the text-to-speech module, and perform audio playback initialization processing, and insert a target marker frame before the wake-up response audio to obtain a target audio, and send the target audio to the audio detection management module. The wake-up response audio is pre-generated and stored by the text-to-speech module according to target response information after the voice service system is started.

[0032] The voice wake-up module is further configured to perform voice back-end point detection on the first user voice, and in a case where a target voice back-end point is detected, set a level state corresponding to the first register module to a first state if the first user voice is detected to include target wake-up information, and make the first register module trigger an interrupt.

[0033] The audio detection management module is configured to perform marker frame detection on the target audio, and in a case where it is determined that the target audio includes the target marker frame, determine that the target audio is audio corresponding to the wake-up response, store the wake-up response audio included in the target audio, set a level state corresponding to the second register module to the first state, and make the second register module trigger an interrupt.

[0034] The audio detection management module is further configured to, in a case where an interrupt triggered by the first register module is detected, obtain a level state corresponding to the first register module, and in a case where an interrupt triggered by the second register module is detected, obtain a level state corresponding to the second register module, and if the level state corresponding to the first register module is the first state and the level state corresponding to the second register module is the first state, send the stored wake-up response audio to the audio playback processing module, and set the level states corresponding to the first register module and the second register module to a second state.

[0035] The audio playback processing module is configured to perform playback processing on the wake-up response audio.

[0036] In one implementation of the present application, the voice service system further comprises a voice acquisition module and a voice processing module. The voice acquisition module is configured to acquire an initial user voice after the voice service system is started, and send the initial user voice to the voice processing module. The voice processing module is configured to perform corresponding acoustic processing on the initial user voice to obtain a first user voice.

[0037] In a third aspect, the embodiments of the present application provide an electronic device, comprising: a processor, and a memory connected with the processor in communication; the memory stores a computer program; and the processor executes the computer program stored in the memory, so that the electronic device implements the voice interaction method described above.

[0038] It can be understood that the beneficial effects of the second aspect and the third aspect described above can also be referred to the related description in the first aspect described above, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the present application, the drawings used in the embodiments will be briefly introduced as follows.

[0040] Figure 1 is a structural schematic diagram of a voice service system provided by an embodiment of the present application;

[0041] Figure 2 is a flowchart of a voice interaction method provided by an embodiment of the present application;

[0042] Figure 3 is another structural schematic diagram of a voice service system provided by an embodiment of the present application;

[0043] Figure 4 is a hardware architecture schematic diagram corresponding to the voice service system provided by an embodiment of the present application;

[0044] Figure 5 is a connection schematic diagram of a GPIO provided by an embodiment of the present application;

[0045] Figure 6 is a flowchart of a voice interaction method implemented based on an Android platform provided by an embodiment of the present application;

[0046] Figure 7 is a schematic diagram of a target marker frame provided by an embodiment of the present application;

[0047] Figure 8 is a schematic diagram of a register level signal transition process provided by an embodiment of the present application;

[0048] Figure 9Figure 1 is a time slice comparison diagram of traditional voice wake-up and voice wake-up provided by an embodiment of the present application;

[0049] Figure 10 Figure 1 is a time slice comparison diagram of traditional voice wake-up and voice wake-up provided by an embodiment of the present application. DETAILED DESCRIPTION

[0050] Currently, a traditional voice service system implementing a voice wake-up function mainly includes a voice acquisition module, a voice processing module, a voice wake-up module, a TTS module, and an audio playback processing module. The link for the voice service system to implement a voice wake-up response is a serialized process link. That is, the voice acquisition module acquires user voice and sends it to the voice processing module, the voice processing module performs a series of acoustic processing such as noise reduction on the user voice to generate target voice, and then sends the target voice to the voice wake-up module for wake-up activation detection. In the case of successful detection, the voice wake-up module triggers the TTS module to synthesize wake-up response audio, and plays the wake-up response audio through the audio playback processing module.

[0051] Taking a vehicle-mounted voice service system as an example, the voice acquisition module can include a microphone device and a digital signal processor (DSP) chip. The microphone device is used to record user voice, and the DSP chip is used to convert the user voice acquired by the microphone device from a sound signal into a digital signal. The audio playback processing module can include a DSP chip, an audio amplifier (AMP), and a loudspeaker. The DSP chip is used to convert the audio to be played from a sound signal into a digital signal, the AMP is used to amplify the audio, and the loudspeaker is used to play the audio.

[0052] The process of the vehicle-mounted voice service system implementing a voice wake-up function is, for example: in a traditional vehicle-mounted environment, a user speaks a customized voice wake-up word, such as "Hello, XX", through a microphone device that continuously records sound on the vehicle. The DSP chip of the vehicle converts the user voice from a sound signal into a digital signal, sends it to the vehicle-mounted audio system, and enters the acoustic processing module corresponding to the voice service for a series of audio processing such as noise reduction, taking the clean audio as output, and sending it to the voice wake-up module corresponding to the voice service. The voice wake-up module performs wake-up activation detection to detect whether the audio has a wake-up word. If there is a wake-up word, the voice service is notified to perform logical processing, and then the TTS module corresponding to the voice service is triggered to synthesize response audio, and enters the playback process of the audio system, sends the sound to the DSP chip and the AMP in turn to amplify the audio, and finally sends it to the loudspeaker for playback. The user will hear an audio broadcast "Here" from the loudspeaker, which is the wake-up response audio.

[0053] The whole process of the voice wake-up response is a serialized process, and there are many modules participating in the voice wake-up response process. Each module needs to consume a certain amount of time in the whole processing link, which leads to a long time consumption of the whole voice wake-up response process and a response delay. The large time slice of the wake-up response delay includes the delay of the voice wake-up module in wake-up activation detection, the delay of the voice acquisition module in acquiring user voice, the delay of the audio playing processing module in playing audio, and the delay of the TTS module in synthesizing wake-up response audio, etc. Each unnecessary delay increases the whole wake-up response delay, and these delays are superimposed in the serial link, resulting in a large voice wake-up response delay. Therefore, how to improve the voice wake-up response delay is imminent.

[0054] Based on the above problems, the embodiments of the present application propose a voice service system and a voice interaction method.

[0055] As shown in Figure 1 The voice service system includes a voice wake-up module, a text-to-speech module, an audio detection management module, a first register module, a second register module, and an audio playing processing module. Each module is mainly used for the following processing in the voice interaction process.

[0056] The voice wake-up module is used for front-end point and back-end point detection of user voice, and when a target voice front-end point is detected, the pre-cached wake-up response audio is obtained from the text-to-speech module and initialized for playing preparation. The target audio is obtained by inserting a marker frame into the cached wake-up response audio, and the target audio is sent to the audio detection management module for pre-preparation for fast playing.

[0057] The text-to-speech module is used for pre-synthesizing wake-up response audio according to, for example, current speaker information after the voice service system is started, and caching in the corresponding memory.

[0058] The first register module and the second register module are used for implementing level state indication, and each includes a corresponding register.

[0059] The audio detection management module is used for detecting whether the target audio contains a marker frame. If there is a marker frame, it is considered to be a pre-cached wake-up response audio, and the level state of the second register module is set to the first state and the audio is cached. At the same time, the level states of the first register module and the second register module are monitored. When both are in the first state, the cached wake-up response audio is sent to the audio playing processing module, otherwise the cache is emptied. In addition, the audio detection management module can be implemented by an advanced digital signal processor (ADSP), so it can also be called an ADSP module.

[0060] The audio playing processing module is used to drive downstream hardware to complete playing according to the received wake-up response audio, and can include the aforementioned DSP chip, AMP, loudspeaker and other devices for realizing audio playing.

[0061] In an implementation manner of the present application, the process of realizing voice interaction based on the voice service system includes: the text-to-speech module pre-generates and stores the wake-up response audio when the voice service system starts. When the voice wake-up module detects the front end point of the user voice, the pre-stored wake-up response audio is acquired, and audio playing initialization processing is performed. The wake-up response audio is inserted into a target marker frame to form a target audio, and the target audio is transmitted to the audio detection management module. After the audio detection management module detects the target marker frame, the wake-up response audio is stored, the level state corresponding to the second register module is set to the first state and an interrupt is triggered; when the voice wake-up module detects the back end point of the voice and confirms that the target wake-up information such as the wake-up word is contained, the state corresponding to the first register module is set to the first state and an interrupt is triggered. When the audio detection management module detects the interrupts of the two register modules and the level states of the two register modules are both the first state, the wake-up response audio is sent to the voice playing processing module for output. The scheme reduces the response delay by pre-generating the wake-up response audio, setting the target marker frame identification and the double register module interrupt synchronization mechanism. Compared with the traditional serial processing mode of waiting for the complete voice wake-up to end before generating the response audio, the response speed is significantly improved, and thus the user experience is significantly improved.

[0062] The implementation manner of the present application provides a voice interaction method applied to a voice service system. As shown in Figure 2 The voice interaction method based on the voice service system includes the following steps.

[0063] In step S100, the voice wake-up module performs voice front end point detection processing on the first user voice, determines that the first user voice is a voice service wake-up voice in the case of detecting a target voice front end point, acquires the wake-up response audio from the text-to-speech module, performs audio playing initialization processing, inserts a target marker frame in front of the wake-up response audio to obtain a target audio, and sends the target audio to the audio detection management module. The wake-up response audio is pre-generated and stored by the text-to-speech module after the voice service system starts according to target response information.

[0064] The target marker frame is, for example, an audio frame composed of a 16-bit integer value 0x7FF0, and presents a sharp peak waveform in an oscilloscope, that is, the target marker frame can be a sharp peak audio frame. Of course, the target marker frame can be a specific digital signal embedded in the first frame of the audio, and the format thereof can be set as needed.

[0065] The target response information is, for example, a text response content predefined by the voice service system, such as a wake-up reply "I'm here".

[0066] The voice wake-up module sends the target audio, for example, in the form of streaming audio or other forms, to the audio detection management module.

[0067] The voice interaction method provided in the present application inserts a hardware-recognizable marker frame formed by a 16-bit integer value before the wake-up response audio, avoids software decoding delay, and the unique waveform of the marker frame can avoid environmental noise false triggering. The target audio is transmitted to the audio detection management module in a streaming manner, resource dynamic scheduling is achieved, and memory peak pressure caused by large audio block transmission is avoided. Moreover, the audio detection management module can receive and detect the marker frame at the same time, further reducing the overall wake-up response delay.

[0068] In step S200, the voice wake-up module performs voice back-end point detection processing on the first user voice. When the target voice back-end point is detected, if the first user voice is detected to include the target wake-up information, the level state corresponding to the first register module is set to the first state, and the first register module is triggered to interrupt.

[0069] The target wake-up information is a specific voice keyword predefined by the voice service system and used to trigger the voice service, for example, "Hello XX" and "Xiao X student" phrases.

[0070] The level state corresponding to the first register module is set to the first state, wherein the first state is a register level signal value of 1, and the second state is a register level signal value of 0.

[0071] In step S300, the audio detection management module performs marker frame detection processing on the target audio. When it is determined that the target audio includes the target marker frame, it is determined that the target audio is the audio corresponding to the wake-up response, the wake-up response audio included in the target audio is stored, the level state corresponding to the second register module is set to the first state, and the second register module is triggered to interrupt.

[0072] In step S400, the audio detection management module detects the interrupt triggered by the first register module, acquires the level state corresponding to the first register module, detects the interrupt triggered by the second register module, acquires the level state corresponding to the second register module, and if the level state corresponding to the first register module is the first state and the level state corresponding to the second register module is the first state, the stored wake-up response audio is sent to the audio playback processing module, and the level states corresponding to the first register module and the second register module are both set to the second state.

[0073] In step S500, the audio playback processing module performs playback processing on the wake-up response audio.

[0074] In the voice interaction method provided in this application, the wake-up response audio is pre-generated and stored by the text-to-speech module after the voice service system starts. The voice wake-up module performs voice front-end point detection processing on the first user's voice. If the target voice front-end point is detected, the pre-generated wake-up response audio can be directly obtained from the text-to-speech module, avoiding the need to generate the wake-up response audio after the voice wake-up module has completed the detection, thus reducing the computational overhead of generating the wake-up response audio. Therefore, using the pre-generated wake-up response audio significantly reduces the wake-up response time, thereby reducing the wake-up response latency and enabling more timely wake-up responses, thereby improving the user experience.

[0075] Furthermore, a target marker frame is inserted before the wake-up response audio to obtain the target audio. This target audio is then sent to the audio detection management module for marker frame detection, enabling the module to accurately identify and verify the wake-up response audio and prevent misplay or incorrect responses. Simultaneously, the voice wake-up module detects the beginning and end points of the user's voice, ensuring that only complete user voice commands are processed, reducing false triggering or false wake-ups caused by factors such as background noise interference.

[0076] Furthermore, an event-driven asynchronous processing mechanism is implemented using the dual-register interrupt mechanism corresponding to the first register module and the second register module. The wake-up response audio is played only when both conditions are met simultaneously: the voice wake-up information detection is complete and the voice tag frame audio is ready. This ensures that all steps of the operation are executed completely, avoids errors caused by partial execution, and reduces false triggering or false wake-up.

[0077] Furthermore, such as Figure 3 As shown, in one implementation of this application, the voice service system further includes a voice acquisition module and a voice processing module. The voice interaction method further includes obtaining the aforementioned first user voice in the following manner: the voice acquisition module acquires the initial user voice after the voice service system is started, and sends the initial user voice to the voice processing module; the voice processing module performs corresponding acoustic processing on the initial user voice to obtain the first user voice.

[0078] In one possible implementation of this application, acoustic processing includes audio noise reduction processing and echo cancellation processing. Of course, acoustic processing may also include methods for converting user speech acquired from the microphone into a clearer, more easily recognizable effective signal by the voice wake-up module.

[0079] The voice interaction method provided in the application clearly links the user voice input, that is, acquires the user voice, performs acoustic processing on the user voice, inputs the user voice to a voice wake-up module for subsequent wake-up operation, improves the accuracy of front-end point / detection by acoustic pre-processing, thereby improving the robustness of the voice service system. By performing audio noise reduction processing to suppress environmental noise interference, the wake-up word recognition rate can be improved. Echo cancellation processing on the audio can prevent the acquired user voice from including audio played by the audio playback processing module and being re-collected to cause false wake-up, thereby improving system reliability.

[0080] In the implementation manner of the application, the voice interaction method further includes: the voice wake-up module performs voice back-end point detection processing on the first user voice, in the case of detecting a target voice back-end point, if the first user voice is detected not to include target wake-up information, the level state corresponding to the second register module is acquired, if the level state corresponding to the second register module is the first state, the second register module is triggered to generate an interrupt, and the level state corresponding to the second register module is adjusted to the second state, otherwise, after the second register module is triggered to generate an interrupt, the level state corresponding to the second register module is adjusted to the second state.

[0081] In the implementation manner of the application, the voice interaction method further includes: if the audio detection management module detects that the level state corresponding to the first register module is the second state and / or the level state corresponding to the second register module is the second state, the stored wake-up response audio is cleared, the audio playback processing is stopped, and the level states corresponding to the first register module and the second register module are both set to the second state.

[0082] In the voice interaction method provided in the application, the level state corresponding to the second register module is adjusted to the second state when no wake-up word is detected, so as to prevent residual interrupt signals from causing subsequent false actions and guarantee system stability. The level states of the first register module and the second register module are used for "AND gate" logical decision to reliably trigger the playback instruction. In the abnormal state, the audio data is cleared and the register is reset to avoid resource occupation or false playback.

[0083] In the implementation manner of the application, the voice interaction method further includes: the voice acquisition module acquires non-initial user voice after the voice service system is started, and sends the non-initial user voice to the voice processing module; the voice processing module performs corresponding acoustic processing on the non-initial user voice to obtain second user voice, and inputs the second user voice to the audio detection management module; the audio detection management module performs marker frame detection processing on the second user voice, in the case of determining that the second user voice does not include a target marker frame, determines that the second user voice is a voice service instruction voice, and performs corresponding instruction response processing according to the second user voice.

[0084] In the voice interaction method provided in the application, the non-initial user voice may be, for example, other interaction instruction voice after the user wake-up voice. For the user voice without the target marker frame (i.e., the instruction voice), the audio detection management module can determine that it is the instruction voice of the voice service and start the corresponding response processing. The process relies on the existing digital signal processing link, without the need for additional complex modules, and reduces the redundant judgment steps of instruction recognition and shortens the delay from receiving the instruction voice to executing the response on the basis of multiplexing the original audio processing architecture.

[0085] Further, the voice service system and method provided in the embodiments of the application are further described below taking the voice service system running on the vehicle-mounted Android platform as an example.

[0086] As shown in Figure 4 The voice service system corresponding to the vehicle-mounted Android platform includes a system on chip (SOC) chip, and the model of the SOC chip may be, for example, 8155A. The SOC chip includes a speech service application (Speech Service), a system firmware layer (Framework, FW), a hardware abstraction layer (Hardware Abstraction Layer, HAL), an audio driver, an operating system, an advanced digital signal processor (Advanced Digital Signal Processor, ADSP), a first register, and a second register. The first register and the second register both include general-purpose input / output (General-purpose input / output, GPIO) pins, the pins included in the first register may be referred to as GPIO(WT), and therefore the first register may be referred to as the GPIO(WT) register. The pins included in the second register may be referred to as GPIO(WTT), and therefore the second register may be referred to as the GPIO(WTT) register.

[0087] Further, the voice service system further includes an audio processor, an audio amplifier (Audio Amplifier, AMP), and a loudspeaker. The audio processor can be used to perform corresponding audio processing on the audio, and the audio amplifier can be used to amplify the audio signal. The audio processor may be, for example, a digital signal processor (Digital Signal Processor, DSP, such as SHARC DSP).

[0088] Further, the hardware connection layer can include a GPIO hardware abstraction layer (General Purpose Input / Output Hardware Abstraction Layer, GPIO HAL), an audio hardware abstraction layer (Audio Hardware Abstraction Layer, Audio HAL). The GPIO HAL is used for accessing the software layer of the GPIO hardware, and can implement control of the GPIO (WT) and the GPIO (WTT). In addition, the Audio HAL is a software layer for accessing the audio hardware.

[0089] The audio driver can include a Linux audio driver (Advanced Linux Sound Architecture, ALSA), a system-on-chip driver (Audio System on Chip, ASoC).

[0090] The operating system can be a QNX real-time operating system (QNX Real-time Operating System, QNX).

[0091] The audio amplifier is connected to the digital signal processor through a time division multiplexing (Time Division Multiplexing, TDM) interface to realize transmission and processing of audio signals. The audio processor also interacts with the advanced digital signal processor in the 8155A SOC through the TDM interface. Inside the 8155A SOC, the GPIO (WTT) is connected to the advanced digital signal processor to realize specific input and output control and wake-up functions. The advanced digital signal processor communicates with the operating system through a fast remote procedure call (Fast Remote Procedure Call, Fast RPC) channel, facilitating efficient data transmission and instruction interaction between different modules in the system. The operating system is connected to the audio driver to jointly complete audio-related processing and control. In addition, the voice service application communicates with the system firmware layer, and then communicates with the GPIO (WT) and the audio driver through the hardware connection layer, respectively, to realize the cooperative work of the voice service related functions and the system hardware.

[0092] Further, the voice service application can include the aforementioned voice processing module, voice wake-up module, and text-to-speech module for realizing corresponding functions.

[0093] Further, the Android platform voice service system also includes a microphone device as a voice acquisition module for acquiring user voice. Of course, the Android platform voice service system can also include a microphone device and a DSP chip as a voice acquisition module for acquiring user voice and converting the user voice from a sound signal to a digital signal for subsequent processing.

[0094] Of course, the Android platform voice service system also includes other components related to the implementation of voice interaction, such as buses and the like.

[0095] Further, as shown in Figure 5 the above voice service system based on the vehicle-mounted Android platform, the bus (Inter-Integrated Circuit master, I2C master) is connected and communicates with the ADSP, GPIO(WT) and GPIO(WTT) through the serial clock line (Serial Clock Line, SCL) and the serial data line (Serial Data Line, SDA), respectively. The GPIO HAL is connected to the GPIO(WT) and the GPIO(WTT) to realize the control and management of the GPIO.

[0096] Based on the above Android platform voice service system, the present application also provides a voice interaction method.

[0097] As shown in Figure 6 in an embodiment of the present application, the voice interaction method includes the following steps. Wherein the voice service system is used to realize the corresponding voice service, therefore, the voice service system is started, which can also be understood as the voice service is started.

[0098] Step S301: After the voice service is started, the TTS module acquires the current speaker information and starts to pre-synthesize the corresponding wake-up TTS audio, i.e. the wake-up response audio, and stores the wake-up TTS audio in the form of audio data such as byte array into the memory corresponding to the TTS module.

[0099] Further, after the voice service is started, the voice wake-up work is started, the recording service and the acoustic processing service are opened, the recording is sent to the voice processing module for corresponding acoustic processing, and the acoustic processing is sent to the voice wake-up module of the voice service for wake-up detection.

[0100] Step S302: The voice wake-up module of the voice service performs front-end point (vad start) detection. The voice wake-up module detects a signal indicating that the front-end point starts in the input user voice, finds the pre-synthesized wake-up TTS audio from the memory corresponding to the TTS module, initializes the audio playing processing module of the voice service, and starts to prepare playing.

[0101] Step S303: A sharp peak audio frame is inserted before the pre-cached wake-up TTS audio to obtain target audio, and the target audio is sent to the ADSP in a streaming form.

[0102] Further, the sharp peak audio frame value of the wake-up TTS audio is a 16 hexadecimal integer value: for example, 0x7FF0, as shown in the signal waveform purple "sharp peak" audio signal of an oscilloscope. Figure 7

[0103] The target audio passes through the Framework, the HAL, and the GPIO (WTT) and the GPIO (WT) level states are monitored, and then written into the ADSP.

[0104] Step S304: The voice wake-up module of the voice service performs back-end point (vad end) detection. If a voice wake-up active event (wakeup active) is detected, as shown in FIG. 6, the level of the GPIO (WT) is pulled high through the address corresponding to the GPIO (WT). Figure 8 If no wake-up word is found after the vad end, an unactive signal is sent, and the unactive notification is sent out. The level state of the GPIO (WTT) is acquired, if it is a high level, an interrupt is triggered, the level of the GPIO (WTT) is pulled low, otherwise, after the level interrupt of the GPIO (WTT) is pulled high, the GPIO (WTT) signal is pulled low.

[0105] Further, the voice wake-up module detects the end of speaking through the vad end, and detects whether the user speaking triggers a wake-up word. If yes, a wake-up notification is triggered. Once the notification is received, the address of the GPIO (WT) can be operated to pull the level high, an interrupt is generated, and other modules are notified. If no, an unactive event is sent. The level state of the GPIO (WTT) is acquired, if it is a high level, an interrupt is triggered, the level of the GPIO (WTT) is pulled low, otherwise, after the level interrupt of the GPIO (WTT) is pulled high, the GPIO (WTT) signal is pulled low. (This step waits for the GPIO (WTT) to be pulled high to prevent the problem that the ADSP is blocked because it cannot wait for the wake-up GPIO (WT) to be pulled high due to the wake-up failure being earlier than the "sharp peak" audio frame of the audio being sent into the ADSP.)

[0106] ​Step S305: ADSP judges whether the input audio contains the tip-top audio frame.

[0107] Step S306: ADSP judges that there is no tip-top audio frame, and considers it as non-wakeup audio, and carries out corresponding processing for non-wakeup audio.

[0108] Further, the non-wakeup audio can be user instruction audio (such as "turn up the temperature") or audio in which the tip-top audio frame is lost or damaged in transmission. The corresponding processing for non-wakeup audio can be executing corresponding instructions, or outputting corresponding audio, etc.

[0109] Step S307: If it is judged that there is a tip-top audio frame, it indicates that the audio is pre-wakeup TTS audio, the address of GPIO (WTT) is operated to pull up the level of the pin of GPIO (WTT). And the received streaming target audio is buffered and not transmitted to SharcDsp.

[0110] Further, ADSP directly receives the target audio sent by the voice wake-up module, and once it detects that the target audio contains a tip-top audio frame, it considers it as voice wake-up corresponding audio, and buffers the subsequent non-tip-top audio frame audio data. At the same time, the address of GPIO (WTT) is operated to pull up the level of the pin of GPIO (WTT). The interrupt level state signal of GPIO (WT) is listened to, and the streaming target audio constantly written is buffered and not transmitted to SharcDsp. If there is no tip-top audio frame, it is considered as non-wakeup audio, and no special processing is needed, and corresponding processing for non-wakeup audio is carried out.

[0111] Step S308: ADSP detects the interrupt signal of GPIO (WT), and acquires the level state of GPIO (WT). At the same time, the interrupt signal of GPIO (WTT) is detected, and the level state of GPIO (WTT) is acquired.

[0112] Further, ADSP detects the interrupt signal of GPIO (WT), and acquires the level state of GPIO (WT). At the same time, the interrupt of GPIO (WTT) is detected, and the level state of GPIO (WTT) is acquired. The next operation is determined by the level states of the two GPIOs, high level is 1, and low level is 0.

[0113] Step S309: If the level of GPIO (WT) or GPIO (WTT) is low, the audio buffered in ADSP is emptied. The levels of GPIO (WT) and GPIO (WTT) are pulled down, and the system is restored.

[0114] Step S310: If GPIO(WT) and the level of GPIO(WTT) is high, the pre-wakeup TTS audio buffered in the ADSP is sent to the audio playing processing module. The level of GPIO(WT) and GPIO(WTT) is pulled low and reset.

[0115] Further, if the AND value of the level signal state of GPIO(WT) and GPIO(WTT) is 0, i.e. GPIO(WT)&GPIO(WTT)=0, the audio data buffered in the ADSP is emptied, the playing action of the audio is stopped, i.e. the audio playing processing is stopped, and the level of GPIO(WT) and GPIO(WTT) is pulled low. If the AND value of the level signal state of GPIO(WT) and GPIO(WTT) is 1, i.e. GPIO(WT)&GPIO(WTT)=1, the buffered wakeup TTS audio data is sent to the voice playing module.

[0116] Step S311: The audio playing processing module plays the wakeup TTS audio.

[0117] Further, the audio playing processing module sends the audio into the loudspeaker to play sound after the audio is processed and amplified by the aforementioned Sharc DSP and AMP. That is, the input audio is processed by the sharc DSP, then enters the AMP to amplify the audio source signal, and finally is input into the loudspeaker to emit sound.

[0118] In the voice interaction method provided in the application, the level of the two GPIO pins of the register on the chip is triggered by different wakeup events, the TTS audio is synthesized and buffered in advance, and a special marker frame is inserted before the first frame of the TTS audio, so that the original voice wakeup response serialization processing flow is changed into parallelization, and the processing link is reduced to improve the voice wakeup response delay. Moreover, no great change is needed for the original audio architecture, and the development cost is greatly reduced.

[0119] In summary, in the voice interaction method provided in the application, as Figure 9As shown, the traditional voice wake-up response process first needs to pass through the front end point and the rear end point to confirm the voice input, then trigger the wake-up response, and then synthesize the wake-up TTS audio in real time and send it to the ADSP, SHARC DSP and AMP in turn for wake-up TTS audio playback. The whole process contains 8 steps. The optimized process of the present scheme cancels the real-time synthesis stage, and eliminates the time consumption of real-time synthesis of wake-up response audio by pre-caching wake-up TTS audio. Whether it is the target audio is confirmed by the GPIO level state, and when it is confirmed as the target audio, the cached wake-up TTS audio is obtained, and then sent to the SHARC DSP, ADSP and AMP for wake-up TTS audio playback, which needs 6 steps. Therefore, the present scheme eliminates the time consumption of real-time synthesis of wake-up response audio by pre-caching wake-up response audio, triggers hardware-level parallel transmission through marker frames, and shortens the overall response time compared with the traditional serial process; the double-pin level "and" logic of the register avoids false triggering and ensures that the audio will only be played when the wake-up is successful and the audio is the wake-up audio. The above-mentioned voice interaction method significantly reduces the wake-up response time, can respond more timely, and thus improves the user experience.

[0120] The implementation manner of the present application provides an electronic device, such as Figure 10 As shown, comprising: a memory 123 for storing a computer program, the computer program comprising program instructions; a processor 122 for executing program instructions; a transceiver 121. So that the electronic device executes the voice interaction method in the above-mentioned embodiments.

[0121] The processor 122 executes the computer execution instructions stored in the memory, so that the processor 122 executes the voice interaction method in the above-mentioned embodiments. The processor 122 can be a general-purpose processor, including a central processing unit CPU, a network processor NP, etc.; it can also be a digital data processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0122] The memory 123 is connected with the processor 122 through a system bus and completes mutual communication, and the memory 123 is used for storing computer program instructions.

[0123] The transceiver 121 can be used to obtain a to-be-run task and configuration information of the to-be-run task.

[0124] Further, the electronic device can be a vehicle, a mobile phone, a watch and the like.

[0125] It is to be understood that the above description is intended to be illustrative and not restrictive. Many other embodiments will be apparent to those of skill in the art upon reading and understanding the above description. Although the application has been described with reference to the preferred embodiment, it is to be understood that various other adaptations and changes can be made within the scope of the application. For example, equivalent

[0126] It should be noted that in this specification and the appended claims, the notation "like" and "similar" refers to similar items in the following drawings, and therefore, once an item is defined in one drawing, it need not be further defined and explained in the subsequent drawings.

[0127] In the description of the present embodiments, it should be noted that the terms "upper", "lower", "inner", "bottom", etc. indicate the positional or location relationship based on the positional or location relationship shown in the drawings, or the positional or location relationship when the product of the application is usually placed, and are only for the convenience of describing the application and simplifying the description, and therefore cannot be understood as indicating or implying that the indicated device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the application.

[0128] The terms "first", "second", etc. are only used for differentiation in description, and cannot be understood as indicating or implying relative importance.

[0129] In the description of the present embodiments, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set", "connected", "connected" should be understood broadly, for example, it can be fixedly connected, or detachably connected, or integrally connected; it can be mechanically connected, or electrically connected; it can be directly connected, or indirectly connected through an intermediate medium, or the internal communication of two elements. For those skilled in the art, the specific meaning of the above terms in the present embodiments can be understood according to the specific circumstances.

[0130] Although the application has been illustrated and described with reference to certain preferred embodiments thereof, it should be understood by those skilled in the art that the above description is further detailed description of the application in combination with the specific embodiments, and cannot be considered as limiting the specific implementation of the application. Those skilled in the art can make various changes in form and details, including making several simple deductions or substitutions, without departing from the spirit and scope of the application.

Claims

1. A voice interaction method, characterized in that, The method is applied to a voice service system, and the voice service system comprises a voice wake-up module, a text-to-speech module, an audio detection management module, a first register module, a second register module and an audio playing processing module, and the method comprises the following steps: The voice wake-up module performs a voice front end point detection processing on a first user voice, in a case where a target voice front end point is detected, determines that the first user voice is a voice service wake-up voice, acquires a wake-up response audio from the text-to-speech module, and performs an audio playing initialization processing, and inserts a target marker frame before the wake-up response audio to obtain a target audio, and sends the target audio to the audio detection management module, wherein the wake-up response audio is generated and stored in advance by the text-to-speech module according to target response information after the voice service system is started; The voice wake-up module performs a voice back end point detection processing on the first user voice, in a case where a target voice back end point is detected, if it is detected that the first user voice comprises target wake-up information, sets a level state corresponding to the first register module to a first state, and makes the first register module trigger an interruption; The audio detection management module performs a marker frame detection processing on the target audio, in a case where it is determined that the target audio comprises the target marker frame, determines that the target audio is an audio corresponding to a wake-up response, stores the wake-up response audio comprised by the target audio, and sets a level state corresponding to the second register module to a first state, and makes the second register module trigger an interruption; The audio detection management module acquires a level state corresponding to the first register module in a case where it is detected that the first register module triggers an interruption, acquires a level state corresponding to the second register module in a case where it is detected that the second register module triggers an interruption, if the level state corresponding to the first register module is the first state and the level state corresponding to the second register module is the first state, sends the stored wake-up response audio to the audio playing processing module, and sets the level states corresponding to the first register module and the second register module to a second state; The audio playing processing module performs a playing processing on the wake-up response audio.

2. The method of claim 1, wherein, The voice service system further comprises a voice acquisition module and a voice processing module, and the method further comprises the following steps of obtaining the first user voice by the following ways: The voice acquisition module acquires an initial user voice after the voice service system is started, and sends the initial user voice to the voice processing module; The voice processing module performs a corresponding acoustic processing on the initial user voice to obtain the first user voice.

3. The method of claim 2, wherein, The acoustic processing comprises an audio noise reduction processing and an echo cancellation processing.

4. The method of claim 3, wherein, The target marker frame is an audio frame composed of a 16 hexadecimal integer value, and the target marker frame presents a sharp roof waveform in an oscilloscope.

5. The method of claim 4, wherein, The voice wake-up module sends the target audio to the audio detection management module, comprising the following steps: The voice wake-up module sends the target audio to the audio detection management module in a streaming audio form.

6. The method of claim 5, wherein, The method further includes: The voice wake-up module performs voice backend endpoint detection on the first user voice, and in a case where a target voice backend endpoint is detected, if it is detected that the first user voice does not include target wake-up information, the voice wake-up module acquires a level state corresponding to the second register module, if the level state corresponding to the second register module is a first state, the voice wake-up module causes the second register module to trigger an interrupt, and adjusts the level state corresponding to the second register module to a second state, otherwise, the voice wake-up module waits until the second register module triggers an interrupt, and then adjusts the level state corresponding to the second register module to the second state.

7. The method of claim 6, wherein, The method further includes: The audio detection management module, if it is detected that the level state corresponding to the first register module is the second state, and / or the level state corresponding to the second register module is the second state, clears the stored wake-up response audio, stops audio playing processing, and sets the level states corresponding to the first register module and the second register module to the second state.

8. The method of claim 7, wherein, The first state is a register level signal value of 1, and the second state is a register level signal value of 0.

9. The method according to any one of claims 2 to 8, characterized in that, The method further includes: The voice acquisition module acquires a non-initial user voice after the voice service system is started, and sends the non-initial user voice to the voice processing module; The voice processing module performs corresponding acoustic processing on the non-initial user voice to obtain a second user voice, and inputs the second user voice to the audio detection management module; The audio detection management module performs marker frame detection processing on the second user voice, and in a case where it is determined that the second user voice does not include the target marker frame, determines that the second user voice is a voice service instruction voice, and performs corresponding instruction response processing according to the second user voice.

10. An electronic device, comprising: comprise: a processor, and a memory connected to the processor in communication; the memory stores a computer program; the processor executes the computer program stored in the memory, so that the electronic device implements the voice interaction method according to any one of claims 1-9.

Images