Voice interaction method and device, equipment and storage medium

Abstract

The invention provides a voice interaction method, device and equipment and a storage medium, and the voice interaction method comprises the steps: collecting first audio data, carrying out the feature analysis of the first audio data, determining the audio type of the first audio data, and waking up a voice interaction system when the audio type comprises a voice feature; the voice interaction system is prevented from being awakened by mistake due to noise interference, and resources are saved; when the voice interaction system is in the wake-up state, the wake-up state is fed back through multiple modes, the second audio data are collected, the second audio data are recognized through the voice interaction system, interaction is achieved, and the multi-mode feedback enables a user to know that the voice interaction system is awakened, so that voice interaction is facilitated.

Description

Technical Field

[0001] The present application relates to the field of systems, and in particular to a voice interaction method, apparatus, device and storage medium. Background Art

[0002] With the development of computer technology, voice interaction systems are widely used in various fields. Speakers can use voice interaction systems to perform corresponding operations. At present, in order to ensure the efficiency of interaction, the entire voice interaction system needs to be in a working state, or the voice interaction system will enter the working state after hearing any sound, such as when there is background noise, in order to receive voice in real time for analysis, resulting in a waste of resources. Summary of the invention

[0003] The embodiments of the present application provide a voice interaction method, apparatus, device, and storage medium to solve at least one problem existing in the related art. The technical solution is as follows:

[0004] In a first aspect, an embodiment of the present application provides a method for voice interaction, including:

[0005] collecting first audio data;

[0006] Performing feature analysis on the first audio data to determine an audio type of the first audio data;

[0007] When the audio type includes a voice feature, waking up the voice interaction system;

[0008] When the voice interaction system is in an awake state, the awake state is fed back through multimodality, and second audio data is collected. The second audio data is recognized by the voice interaction system to achieve interaction.

[0009] In one implementation, performing feature analysis on the first audio data to determine the audio type of the first audio data includes:

[0010] Performing frequency characteristic analysis on the first audio data to determine the fluctuation amplitude of the spectrum within a specified time threshold, and determining the audio type including speech features when the fluctuation amplitude is greater than a preset threshold;

[0011] or,

[0012] Perform voice activation detection on the first audio data to detect whether there are voice features, and obtain the audio type of the first audio data.

[0013] In one implementation, collecting the second audio data includes:

[0014] extracting environmental noise from the first audio data by using a filtering algorithm, and analyzing the frequency spectrum and decibel level of the environmental noise to determine the noise level;

[0015] When the noise level is within a specified range, reducing the input sensitivity of the audio device;

[0016] When the noise level exceeds a specified range, reducing the input sensitivity of the audio device and enhancing the filtering parameters of the audio device;

[0017] The second audio data is collected through the audio device.

[0018] In one embodiment, when the voice interaction system is in an awake state, feeding back the awake state through multimodal means includes:

[0019] When the voice interaction system is in the wake-up state, adjusting the color or indication mode of the indicator light of the voice interaction system to provide visual feedback, or displaying wake-up state information on the screen of the voice interaction system to provide visual feedback;

[0020] When the voice interaction system is in the awake state, a prompt sound is emitted to provide auditory feedback;

[0021] When the voice interaction system is in the awakened state, when the visual feedback and the auditory feedback fail, the tactile feedback is performed through vibration prompts.

[0022] In one embodiment, when the voice interaction system is in an awake state, feeding back the awake state through multimodal means further comprises:

[0023] When providing visual feedback, adjusting the brightness of the indicator light according to the noise level;

[0024] When performing auditory feedback, the volume of the prompt sound is adjusted according to the noise level.

[0025] In one implementation, the recognizing the second audio data by the voice interaction system to achieve interaction includes:

[0026] Identify the second audio data and determine the audio content through the machine learning model of the voice interaction system;

[0027] In response to the audio content, corresponding operation instructions are generated to achieve interaction.

[0028] In a second aspect, an embodiment of the present application provides a voice interaction device, including:

[0029] A collection module, used for collecting first audio data;

[0030] a determination module, configured to perform feature analysis on the first audio data to determine an audio type of the first audio data;

[0031] A wake-up module, used to wake up the voice interaction system when the audio type includes a voice feature;

[0032] The interaction module is used to feedback the awakening state through multimodality when the voice interaction system is in the awakening state, collect the second audio data, and recognize the second audio data through the voice interaction system to achieve interaction.

[0033] In one embodiment, the interaction module includes a first unit, a second unit, and a third unit;

[0034] The first unit is used to adjust the color or indication mode of the indicator light of the voice interaction system to provide visual feedback when the voice interaction system is in the wake-up state, or to display the wake-up state information on the screen of the voice interaction system to provide visual feedback;

[0035] The second unit is used to emit a prompt sound and provide auditory feedback when the voice interaction system is in a wake-up state;

[0036] The third unit is used to provide tactile feedback through vibration prompts when the voice interaction system is in the awakened state and the visual feedback and the auditory feedback fail.

[0037] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory, wherein the memory stores instructions, and the instructions are loaded and executed by the processor to implement a method in any one of the above-mentioned embodiments.

[0038] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed, the method in any one of the above-mentioned embodiments is implemented.

[0039] The beneficial effects of the above technical solution include at least:

[0040] By collecting first audio data, performing feature analysis on the first audio data, determining the audio type of the first audio data, and waking up the voice interaction system when the audio type includes voice features, it is beneficial to avoid waking up the voice interaction system by mistake due to noise interference, and it is beneficial to save resources; when the voice interaction system is in an awakened state, the awakening state is fed back through multimodal feedback, and second audio data is collected, and the second audio data is identified by the voice interaction system to achieve interaction. The multimodal feedback enables the user to understand that the voice interaction system has been awakened, which is convenient for voice interaction.

[0041] The above summary is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present application will be readily apparent by reference to the accompanying drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments disclosed in the present application and should not be regarded as limiting the scope of the present application.

[0043] Figure 1 This is a schematic diagram of the steps of the voice interaction method according to an embodiment of the present application;

[0044] Figure 2 This is a structural block diagram of a voice interaction device according to an embodiment of the present application;

[0045] Figure 3 This is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0046] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the spirit or scope of the present application. Therefore, the drawings and descriptions are considered to be exemplary and non-restrictive in nature.

[0047] Reference Figure 1 , a flow chart of a voice interaction method according to an embodiment of the present application is shown, and the voice interaction method may at least include steps S100-S400:

[0048] S100: Collect first audio data.

[0049] S200: Perform feature analysis on first audio data to determine an audio type of the first audio data.

[0050] S300: When the audio type includes voice features, wake up the voice interaction system.

[0051] S400: When the voice interaction system is in an awake state, the awake state is fed back through multimodal feedback, and second audio data is collected, and the second audio data is recognized by the voice interaction system to achieve interaction.

[0052] It should be noted that the voice interaction system of the present application includes an audio device M1, an M5S module (referred to as M5S) and a terminal. The terminal may be, for example, an electronic whiteboard all-in-one machine, a computer, etc., which has a display screen. M1 is a front-end audio device for audio acquisition, and M5S is a back-end data processing module. M5S is mainly used for data analysis and processing, and realizes functions such as status update and decision-making. For example, through multi-modal feedback wake-up status, identify the second audio data, realize interaction, etc.; at the same time, M5S can control the sensitivity, sampling rate and other parameters of M1, so that M1 can adapt to changes under different environmental noise levels. The wake-up voice interaction system in the embodiment of the present application mainly refers to waking up M5S.

[0053] The technical solution of the embodiment of the present application collects first audio data, performs feature analysis on the first audio data, determines the audio type of the first audio data, and when the audio type includes voice features, wakes up the voice interaction system, which is beneficial to avoid erroneous wake-up of the voice interaction system due to noise interference, and is beneficial to saving resources; when the voice interaction system is in an awakened state, the awakening state is fed back through multimodal feedback, and second audio data is collected, and the second audio data is recognized by the voice interaction system to achieve interaction, and the multimodal feedback enables the user to understand that the voice interaction system has been awakened, thereby facilitating voice interaction.

[0054] In an embodiment of the present application, after the first audio data and the second audio data are collected, preprocessing can be performed, which can be performed by the M1 or M5S module. The preprocessing may include noise filtering and signal enhancement; basic noise filtering and audio signal enhancement are performed by M1, which can reduce redundancy in audio data transmission and ensure transmission efficiency.

[0055] In one embodiment, when the voice interaction system is in a low power consumption standby mode, the first audio data is collected by M1, and then the first audio data is feature analyzed to determine the audio type of the first audio data. In addition, the user can set the voice interaction system to a paused state to temporarily turn off the audio recognition function.

[0056] In one embodiment, step S200 includes step S210 or S220:

[0057] S210: Perform frequency feature analysis on the first audio data to determine the fluctuation amplitude of the spectrum within a specified time threshold, and when the fluctuation amplitude is greater than a preset threshold, determine the audio type including speech features.

[0058] Optionally, the time threshold and fluctuation amplitude can be set by the user, for example, in advance through a GUI interface on the screen or a voice command; since the fluctuation amplitude of the spectrum of environmental noise is usually different from that of human voice, the first audio data can be subjected to frequency characteristic analysis to determine the fluctuation amplitude of the spectrum within the specified time threshold. When the fluctuation amplitude is greater than a preset threshold, it can be determined that the current first audio data includes speech features of human voice, for example, the speech features of one or more people, and the audio type including the speech features is determined, which may include environmental noise (background noise) at the same time as the speech features.

[0059] S220: Perform voice activation detection on the first audio data to detect whether there is a voice feature, and obtain the audio type of the first audio data.

[0060] In one embodiment, voice activation detection (VAD) technology can be used to perform voice activation detection on the first audio data to detect whether there are voice features, thereby determining the audio type of the first audio data. If so, the audio type including the voice features is determined, and the voice features may also include ambient noise (background noise). If not, the audio type is ambient noise (background noise).

[0061] In one embodiment, when it is determined that the audio type includes voice features, it means that the user is speaking and voice interaction is required. At this time, a signal (which can be any signal or the first audio data) is sent to M5S to wake up M5S, which is equivalent to waking up the voice interaction system. It should be noted that when sending the signal, it can be compressed by a data compression algorithm to improve the transmission efficiency and synchronize the states of M1 and M5S.

[0062] In an embodiment of the present application, when the voice interaction system is in an awake state, the speaker's second audio data and environmental changes are dynamically monitored in real time to improve recognition accuracy, reduce delays, and adapt to changing usage scenarios.

[0063] In one implementation, when the voice interaction system is in an awake state in step S400, the awake state is fed back through multimodal feedback, which may include steps S410-S430:

[0064] S410: When the voice interaction system is in the wake-up state, adjust the color or indication mode of the indicator light of the voice interaction system to provide visual feedback, or display the wake-up state information on the screen of the voice interaction system to provide visual feedback.

[0065] Optionally, when the voice interaction system is in the wake-up state, if the voice interaction system has an indicator light, visual feedback can be provided by changing the color or indication mode of the indicator light (for example, constantly on, flashing, etc.), so that the user can understand through the indicator light that the voice interaction system has been awakened and can continue to speak; or, the wake-up status information is displayed on the screen of the voice interaction system for visual feedback, and the user can understand by watching the screen that the voice interaction system has been awakened and can continue to speak.

[0066] In one implementation, the brightness or flashing frequency of the indicator light may be adjusted according to the noise level. The higher the noise level, the higher the brightness or the faster the indicator light flashes, reminding the user to increase the volume appropriately.

[0067] S420: When the voice interaction system is in the awake state, a prompt sound is emitted to provide auditory feedback.

[0068] Optionally, when the voice interaction system is in the awake state, a prompt tone may be emitted to provide auditory feedback, such as "awakened", "please speak", etc., so that the user knows that the voice interaction system has been awakened.

[0069] In one implementation, when providing auditory feedback, the volume of the prompt sound is adjusted according to the noise level. For example, the higher the noise level, the higher the volume of the prompt sound, thereby ensuring that the user can hear the prompt sound clearly.

[0070] S430: When the voice interaction system is in the awakened state, when the visual feedback and the auditory feedback fail, a tactile feedback is provided through a vibration prompt.

[0071] Optionally, when the voice interaction system is in the awake state, when visual feedback and auditory feedback fail, for example, when there are no conditions for playing sound or indicator light, or when no further audio is collected within a specified time, a vibration prompt is used and tactile feedback is provided to let the user know that the voice interaction system has been awakened.

[0072] In one implementation, collecting the second audio data in step S400 may include steps S440-S460:

[0073] S440: extracting environmental noise from the first audio data by using a filtering algorithm, and analyzing the frequency spectrum and decibel level of the environmental noise to determine a noise level.

[0074] Optionally, the second audio data can be continuously collected by M1, and then the second audio data can be transmitted to M5S for further processing. In the embodiment of the present application, M5S extracts environmental noise from the first audio data through a filtering algorithm, such as (such as mechanical sound, wind noise), etc., and can classify and quantify it, and then analyzes the spectrum and decibel level of the extracted environmental noise to determine the noise level of the current environmental noise.

[0075] S450: When the noise level is within a specified range, the input sensitivity of the audio device is reduced; when the noise level exceeds the specified range, the input sensitivity of the audio device is reduced, and the filtering parameters of the audio device are enhanced.

[0076] Optionally, the specified range can be set based on user needs, for example, a range of 40-60dB. If the noise level is less than the specified range, it means that the noise level is low and suitable for high-sensitivity audio input; if the noise level is within the specified range, it means that the noise level is medium, and the audio input sensitivity is appropriately reduced to avoid misrecognition; when the noise level exceeds the specified range, it means that the noise level is high, and the input sensitivity of the audio device is reduced to avoid affecting subsequent voice recognition, and the filtering parameters of the audio device are enhanced to enhance the noise filtering effect of the audio device.

[0077] For example, the frequency domain filter response function H(f) of the audio device can be adjusted. The size of the frequency domain filter response function represents the gain or attenuation of different frequency components. Y(t)=X(t)×H(f), where Y(t) is the processed audio signal and X(t) is the audio signal before processing.

[0078] S460: Collect second audio data through an audio device.

[0079] Then, the second audio data is collected by continuing to adjust the audio device with the above parameters to ensure the audio quality of the second audio data, so as to improve the accuracy of subsequent recognition.

[0080] In one implementation, in step S400, the second audio data is recognized by the voice interaction system, and the interaction may include steps S470-S480:

[0081] S470. Identify the second audio data and determine the audio content through the machine learning model of the voice interaction system.

[0082] Optionally, audio recognition of the second audio data is performed through a machine learning model set by M5S in the voice interaction system, and the second audio data is converted into text to determine the audio content.

[0083] In one implementation, when identifying the second audio data, language features such as sound intensity, frequency, intonation, and voice pattern may also be simultaneously analyzed, and further noise filtering and signal enhancement processing may be performed.

[0084] S480: Generate corresponding operation instructions in response to the audio content to achieve interaction.

[0085] Optionally, the M5S responds to the audio content to generate an operation instruction corresponding to the audio content, such as controlling the terminal system to execute a corresponding function to achieve voice interaction. Wherein, when the audio content involves multiple operations, a corresponding operation instruction sequence is generated in sequence according to the appearance time in the second audio data to achieve interaction.

[0086] Reference Figure 2 , shows a structural block diagram of a voice interaction device according to an embodiment of the present application, the device may include:

[0087] A collection module, used for collecting first audio data;

[0088] A determination module, configured to perform feature analysis on the first audio data to determine an audio type of the first audio data;

[0089] A wake-up module, used to wake up the voice interaction system when the audio type includes voice features;

[0090] The interaction module is used to feedback the awakening state through multimodal feedback when the voice interaction system is in the awakening state, collect the second audio data, and recognize the second audio data through the voice interaction system to realize interaction.

[0091] Among them, the acquisition module and the determination module can be sub-modules of the audio device M1, and the wake-up module and the interaction module can be sub-modules of the M5S module.

[0092] In one embodiment, the interaction module includes a first unit, a second unit, and a third unit;

[0093] The first unit is used to adjust the color or indication mode of the indicator light of the voice interaction system when the voice interaction system is in the wake-up state to provide visual feedback, or to display the wake-up state information on the screen of the voice interaction system to provide visual feedback;

[0094] The second unit is used to emit a prompt tone and provide auditory feedback when the voice interaction system is in the awake state;

[0095] The third unit is used to provide tactile feedback through vibration prompts when the voice interaction system is in an awake state and the visual feedback and the auditory feedback fail.

[0096] The functions of each module in each device in the embodiments of the present application can be found in the corresponding description in the above method, and will not be repeated here.

[0097] Reference Figure 3 , shows a structural block diagram of an electronic device according to an embodiment of the present application, the electronic device includes: a memory 310 and a processor 320, the memory 310 stores instructions that can be run on the processor 320, and the processor 320 loads and executes the instructions to implement the voice interaction method in the above embodiment. The number of the memory 310 and the processor 320 can be one or more.

[0098] In one embodiment, the electronic device also includes a communication interface 330 for communicating with external devices and performing data exchange transmission. If the memory 310, the processor 320 and the communication interface 330 are implemented independently, the memory 310, the processor 320 and the communication interface 330 can be interconnected through a bus and complete mutual communication. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0099] Optionally, in a specific implementation, if the memory 310, the processor 320 and the communication interface 330 are integrated on a chip, the memory 310, the processor 320 and the communication interface 330 can communicate with each other through an internal interface.

[0100] An embodiment of the present application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the voice interaction method provided in the above embodiment.

[0101] An embodiment of the present application also provides a chip, which includes a processor for calling and executing instructions stored in the memory from the memory, so that a communication device equipped with the chip executes the method provided by the embodiment of the present application.

[0102] An embodiment of the present application also provides a chip, including: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method provided in the embodiment of the application.

[0103] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor supporting the advanced RISC machines (ARM) architecture.

[0104] Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory, and may also include a non-volatile random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available. For example, static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct rambus RAM (DR RAM).

[0105] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0106] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the described specific features, structures, materials or characteristics may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0107] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0108] Any process or method description in the flow chart or otherwise described herein can be understood to represent a module, fragment or portion of a code including one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.

[0109] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be embodied in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, apparatus or device and execute instructions), or used in combination with these instruction execution systems, apparatuses or devices.

[0110] It should be understood that the various parts of the present application can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium, and when the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0111] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0112] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A voice interaction method, characterized in that: include: collecting first audio data; Performing feature analysis on the first audio data to determine an audio type of the first audio data; When the audio type includes a voice feature, waking up the voice interaction system; When the voice interaction system is in an awake state, the awake state is fed back through multimodality, and second audio data is collected. The second audio data is recognized by the voice interaction system to achieve interaction.

2. The voice interaction method according to claim 1, characterized in that: The performing feature analysis on the first audio data to determine the audio type of the first audio data includes: Performing frequency characteristic analysis on the first audio data to determine the fluctuation amplitude of the spectrum within a specified time threshold, and determining the audio type including speech features when the fluctuation amplitude is greater than a preset threshold; or, Perform voice activation detection on the first audio data to detect whether there are voice features, and obtain the audio type of the first audio data.

3. The voice interaction method according to claim 2, characterized in that: The collecting of the second audio data comprises: extracting environmental noise from the first audio data by using a filtering algorithm, and analyzing the frequency spectrum and decibel level of the environmental noise to determine the noise level; When the noise level is within a specified range, reducing the input sensitivity of the audio device; When the noise level exceeds a specified range, reducing the input sensitivity of the audio device and enhancing the filtering parameters of the audio device; The second audio data is collected through the audio device.

4. The voice interaction method according to claim 3, characterized in that: When the voice interaction system is in an awake state, feeding back the awake state through multimodal means includes: When the voice interaction system is in the wake-up state, adjusting the color or indication mode of the indicator light of the voice interaction system to provide visual feedback, or displaying wake-up state information on the screen of the voice interaction system to provide visual feedback; When the voice interaction system is in the awake state, a prompt sound is emitted to provide auditory feedback; When the voice interaction system is in the awakened state, when the visual feedback and the auditory feedback fail, the tactile feedback is performed through vibration prompts.

5. The voice interaction method according to claim 4, characterized in that: When the voice interaction system is in the awake state, feeding back the awake state through multimodal means further comprises: When providing visual feedback, adjusting the brightness of the indicator light according to the noise level; When performing auditory feedback, the volume of the prompt sound is adjusted according to the noise level.

6. The voice interaction method according to claim 2 or 3, characterized in that: The identifying the second audio data by the voice interaction system to achieve interaction includes: Identify the second audio data and determine the audio content through the machine learning model of the voice interaction system; In response to the audio content, corresponding operation instructions are generated to achieve interaction.

7. A voice interaction device, characterized in that: include: A collection module, used for collecting first audio data; a determination module, configured to perform feature analysis on the first audio data to determine an audio type of the first audio data; A wake-up module, used to wake up the voice interaction system when the audio type includes a voice feature; The interaction module is used to feedback the awakening state through multimodality when the voice interaction system is in the awakening state, collect the second audio data, and recognize the second audio data through the voice interaction system to achieve interaction.

8. The voice interaction device according to claim 7, characterized in that: The interaction module includes a first unit, a second unit and a third unit; The first unit is used to adjust the color or indication mode of the indicator light of the voice interaction system to provide visual feedback when the voice interaction system is in the wake-up state, or to display the wake-up state information on the screen of the voice interaction system to provide visual feedback; The second unit is used to emit a prompt sound and provide auditory feedback when the voice interaction system is in a wake-up state; The third unit is used to provide tactile feedback through vibration prompts when the voice interaction system is in the awakened state and the visual feedback and the auditory feedback fail.

9. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores instructions, and the instructions are loaded and executed by the processor to implement the method according to any one of claims 1 to 6.

10. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed, the method according to any one of claims 1 to 6 is implemented.

Images