Wake-up method of intelligent interaction device, electronic device, and storage medium

Abstract

Embodiments of the present application provide a wake-up method of an intelligent interaction device, an electronic device and a storage medium. The method comprises: in response to detecting a wake-up trigger signal of the intelligent interaction device, acquiring a first ambient sound and a second ambient sound of an environment in which the intelligent interaction device is located; determining acoustic energy of the wake-up trigger signal and a plurality of candidate positioning areas based on the first ambient sound and the second ambient sound; determining whether the wake-up trigger signal is valid based on the acoustic energy and the plurality of candidate positioning areas; and in the case that the wake-up trigger signal is valid, controlling the intelligent interaction device to enter a wake-up state from a standby state. The present application solves the technical problem of poor accuracy of the wake-up detection of the intelligent interaction device in a complex sound field in the related art.

Description

Technical Field

[0001] This application relates to the field of human-computer interaction, and more specifically, to a method for waking up an intelligent interactive device, an electronic device, and a storage medium. Background Technology

[0002] With the widespread application of intelligent companion robots, desktop intelligent agents, and various intelligent interactive devices with microphone arrays in homes, offices, and public service areas, voice interaction has become the main entry point for human-computer interaction. The reliability and smoothness of the wake-up mechanism of intelligent interactive devices directly determine the user experience. However, current wake-up mechanisms have poor accuracy in detecting wake-up in complex sound fields.

[0003] There is currently no good solution to the above problems. Summary of the Invention

[0004] This application provides a wake-up method, electronic device, and storage medium for a smart interactive device, to at least solve the technical problem of poor accuracy in wake-up detection of smart interactive devices under complex sound fields in related technologies.

[0005] According to one aspect of the embodiments of this application, a wake-up method for a smart interactive device is provided, comprising: responding to the detection of a wake-up trigger signal of the smart interactive device; acquiring a first ambient sound and a second ambient sound of the environment in which the smart interactive device is located, wherein the smart interactive device is in a standby state, the first ambient sound is used to characterize the ambient sound before the wake-up trigger signal is detected, and the second ambient sound is used to characterize the ambient sound after the wake-up trigger signal is detected; determining the acoustic energy of the wake-up trigger signal and a plurality of candidate positioning regions based on the first ambient sound and the second ambient sound; determining whether the wake-up trigger signal is valid based on the acoustic energy and the plurality of candidate positioning regions; and controlling the smart interactive device to enter a wake-up state from the standby state if the wake-up trigger signal is valid, wherein the wake-up state is used for voice interaction.

[0006] Furthermore, based on the first ambient sound and the second ambient sound, the acoustic energy of the wake-up trigger signal and multiple candidate positioning regions are determined, including: determining the environment type based on the first ambient sound; determining the acoustic energy based on the environment type, the first ambient sound, and the second ambient sound; and determining multiple candidate positioning regions based on the first ambient sound and the second ambient sound.

[0007] Furthermore, based on the first ambient sound, the environment type is determined, including: extracting features from the first ambient sound to obtain its acoustic features; and determining the environment type based on the acoustic features.

[0008] Furthermore, based on acoustic characteristics, the environment type is determined, including: based on acoustic characteristics, determining the signal-to-noise ratio and attenuation of the first ambient sound; based on the signal-to-noise ratio and attenuation, determining the sound source distance and reverberation intensity of the first ambient sound; and based on the sound source distance and reverberation intensity, determining the environment type.

[0009] Further, based on the environment type, the first ambient sound, and the second ambient sound, the acoustic energy is determined, including: determining a first interception window based on the environment type; intercepting the first ambient sound and the second ambient sound based on the first interception window to obtain a first reference audio; and determining the acoustic energy based on the average energy of the first reference audio.

[0010] Furthermore, based on the first ambient sound and the second ambient sound, multiple candidate positioning regions are determined, including: determining a second interception window based on the time when the wake-up trigger signal is detected; intercepting the first ambient sound and the second ambient sound based on the second interception window to obtain a second reference audio; and determining multiple candidate positioning regions based on the second reference audio.

[0011] Furthermore, based on the second reference audio, multiple candidate positioning regions are determined, including: inputting the second reference audio into a direction analysis model, processing the second reference audio using the direction analysis model to obtain positioning probability distributions in multiple directions; selecting multiple candidate positioning directions from the multiple directions based on the positioning probability distributions; and determining multiple candidate positioning regions based on the multiple candidate positioning directions.

[0012] Furthermore, based on acoustic energy and multiple candidate positioning regions, it is determined whether the wake-up trigger signal is valid, including: determining whether the acoustic energy is greater than a preset energy; if the acoustic energy is greater than the preset energy, performing time delay compensation on the initial audio signals of multiple candidate positioning regions to obtain multiple enhanced audio signals corresponding to multiple candidate positioning regions, and determining whether the wake-up trigger signal is valid based on the multiple enhanced audio signals.

[0013] Furthermore, time delay compensation is performed on the initial audio signals of multiple candidate positioning regions to obtain multiple enhanced audio signals corresponding to the multiple candidate positioning regions, including: determining the search range and search step size of any candidate positioning region; determining multiple search directions within any candidate positioning region based on the search range and search step size; and performing time delay compensation on the initial audio signals of multiple candidate positioning regions based on the multiple search directions to obtain multiple enhanced audio signals.

[0014] Furthermore, based on multiple search directions, time delay compensation is performed on the initial audio signals of multiple candidate positioning regions to obtain multiple enhanced audio signals. This includes: acquiring the geometric position of the microphone array of the intelligent interactive device, as well as the sound source direction angle, sound velocity, and sampling rate of the initial audio signal; determining the time delay of the initial audio signal in multiple search directions based on the geometric position, sound source direction angle, sound velocity, and sampling rate; compensating the initial audio signal in multiple search directions based on the time delay to obtain multiple compensated audio signals; and fusing the multiple compensated audio signals to obtain the enhanced audio signal corresponding to any candidate positioning region.

[0015] Furthermore, based on multiple enhanced audio signals, the validity of the wake-up trigger signal is determined, including: based on multiple enhanced audio signals, determining multiple secondary wake-up confidence levels corresponding to multiple candidate positioning regions; and determining that the wake-up trigger signal is valid if any secondary wake-up confidence level is greater than a preset confidence level.

[0016] Furthermore, before controlling the intelligent interactive device to enter the wake-up state from the standby state, the method also includes: determining the audio parameters corresponding to multiple candidate positioning areas based on the second ambient sound; and determining the target positioning area from the multiple candidate positioning areas based on multiple enhanced audio signals, multiple secondary wake-up confidence levels, and audio parameters.

[0017] Furthermore, the audio parameters corresponding to each candidate positioning region include the audio activity and audio temporal continuity of that candidate positioning region. Based on multiple enhanced audio signals, multiple secondary wake-up confidence levels, and audio parameters, a target positioning region is determined from multiple candidate positioning regions, including: determining the weights of the audio activity, secondary wake-up confidence level, audio temporal continuity, and enhanced audio signals for any candidate positioning region; based on the weights, performing weighted fusion of the audio activity, secondary wake-up confidence level, audio temporal continuity, and enhanced audio signals to obtain a confidence index value for any candidate positioning region; based on the multiple confidence index values ​​corresponding to multiple candidate positioning regions, a target positioning region is determined from multiple candidate positioning regions, wherein the target confidence index value corresponding to the target positioning region is greater than the confidence index values ​​corresponding to the candidate positioning regions other than the target positioning region.

[0018] Furthermore, after controlling the intelligent interactive device to enter the wake-up state from the standby state, the method also includes: collecting the interaction parameters of the intelligent interactive device during the voice interaction process; determining whether the interaction parameters meet the preset conditions; and controlling the intelligent interactive device to enter the standby state from the wake-up state if the interaction parameters do not meet the preset conditions.

[0019] Furthermore, the interaction parameters include the voice energy of the interactive voice, the effective voice duration, and the prompt tone protection period; determining whether the interaction parameters meet the preset conditions includes: determining that the interaction parameters meet the preset conditions when the voice energy is greater than the preset energy, the effective voice duration is greater than the preset time, and the time when the interactive voice is emitted is not within the prompt tone protection period.

[0020] Furthermore, controlling the intelligent interactive device to enter the wake-up state from the standby state includes: constructing feedback prompt information based on the target positioning area, wherein the feedback prompt information is used to prompt the intelligent interactive device to enter the wake-up state; and outputting feedback prompt information based on various output devices on the intelligent interactive device.

[0021] According to another aspect of the embodiments of this application, a wake-up device for a smart interactive device is also provided, comprising: an acquisition module, configured to acquire a first ambient sound and a second ambient sound of the environment in which the smart interactive device is located in response to the detection of a wake-up trigger signal of the smart interactive device, wherein the smart interactive device is in a standby state, the first ambient sound is used to characterize the ambient sound before the wake-up trigger signal is detected, and the second ambient sound is used to characterize the ambient sound after the wake-up trigger signal is detected; a determination module, configured to determine the acoustic energy of the wake-up trigger signal and a plurality of candidate positioning regions based on the first ambient sound and the second ambient sound; a verification module, configured to determine whether the wake-up trigger signal is valid based on the acoustic energy and the plurality of candidate positioning regions; and a control module, configured to control the smart interactive device to enter a wake-up state from a standby state when the wake-up trigger signal is valid, wherein the wake-up state is used for voice interaction.

[0022] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of this application when it runs.

[0023] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of this application.

[0024] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods of various embodiments of this application.

[0025] According to another aspect of the embodiments of this application, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods in various embodiments of this application.

[0026] According to another aspect of the embodiments of this application, a computer program is also provided, which, when executed by a processor, implements the methods of the various embodiments of this application.

[0027] In this embodiment, a method is adopted to detect the wake-up trigger signal of the smart interactive device, acquire the first ambient sound and the second ambient sound of the environment in which the smart interactive device is located; determine the acoustic energy of the wake-up trigger signal and multiple candidate positioning regions based on the first ambient sound and the second ambient sound; determine whether the wake-up trigger signal is valid based on the acoustic energy and the multiple candidate positioning regions; and control the smart interactive device to enter the wake-up state when the wake-up trigger signal is valid. By synchronously determining the acoustic energy of the wake-up trigger signal and multiple candidate positioning regions through the first ambient sound and the second ambient sound, the acoustic energy is used for initial screening to filter weak sound or noise interference in the far field. The potential direction of the wake-up trigger signal in a multi-sound source environment is retained through multiple candidate positioning regions, avoiding misjudgment in complex reflection or reverberation scenarios by single direction estimation. Finally, the acoustic energy and candidate positioning regions are combined to jointly determine whether the wake-up trigger signal is valid. The smart interactive device is controlled to enter the wake-up state only after the wake-up trigger signal is confirmed to be valid. This achieves the purpose of effectively distinguishing the real wake-up trigger signal from complex background noise or echo interference, thereby solving the technical problem of poor accuracy of wake-up detection of smart interactive devices in complex sound fields in related technologies. Attached Figure Description

[0028] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0029] Figure 1 This is a flowchart of a wake-up method for an intelligent interactive device according to an embodiment of this application;

[0030] Figure 2 This is a schematic diagram of an optional feedback prompt information output process according to an embodiment of this application;

[0031] Figure 3 This is a schematic diagram of an optional wake-up system decision process according to an embodiment of this application;

[0032] Figure 4 This is a schematic diagram of an optional wake-up detection logic according to an embodiment of this application;

[0033] Figure 5 This is a schematic diagram of an optional target positioning area determination process according to an embodiment of this application;

[0034] Figure 6This is a schematic diagram of a wake-up device for a smart interactive device according to an embodiment of this application. Detailed Implementation

[0035] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0036] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0037] According to an embodiment of this application, an embodiment of a wake-up method for a smart interactive device is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0038] This embodiment provides a method for waking up a smart interactive device. Figure 1 This is a flowchart of a wake-up method for a smart interactive device according to an embodiment of this application, such as... Figure 1 As shown, the process includes the following steps:

[0039] Step S102: In response to the detection of the wake-up trigger signal of the smart interactive device, acquire the first ambient sound and the second ambient sound of the environment in which the smart interactive device is located. The smart interactive device is in standby mode. The first ambient sound is used to characterize the ambient sound before the wake-up trigger signal is detected, and the second ambient sound is used to characterize the ambient sound after the wake-up trigger signal is detected.

[0040] The aforementioned intelligent interactive device refers to a hardware terminal equipped with a multi-channel microphone array, capable of collecting environmental acoustic information and performing human-computer voice interaction functions. It can be used in home service robots, desktop companion robots, or various other devices. The core functions of this intelligent interactive device may include, but are not limited to, listening for wake words in standby mode, confirming sound source orientation, and managing session status.

[0041] The aforementioned wake-up trigger signal can refer to a logical event or data marker identified by the intelligent interactive device in the audio stream of a reference channel when the matching degree of a preset wake-up word exceeds a threshold. This reference channel can be a fixed channel pre-specified in a multi-channel microphone array or selected from multiple input channels based on channel validity and signal-to-noise ratio, used to perform the wake-up trigger signal detection. This wake-up trigger signal signifies that the intelligent interactive device has transitioned from simple background listening to a processing flow requiring further validity screening and direction confirmation, but it does not directly cause the device to immediately enter an activated state.

[0042] The aforementioned first ambient sound can refer to a segment of multi-channel audio data recorded by the intelligent interactive device in its cached historical audio, spanning a time period prior to the detection of the wake-up trigger signal. This first ambient sound can be used to trace the environmental acoustic context before the wake-up trigger signal was issued, thereby providing historical reference for subsequent construction of candidate direction regions and evaluation of the effectiveness of the wake-up trigger signal.

[0043] The aforementioned second ambient sound can refer to a segment of multi-channel audio data recorded by the intelligent interactive device in its cached historical audio, spanning a time period after the detection of the wake-up trigger signal. This second ambient sound, together with the first ambient sound, can constitute a continuous environmental acoustic context, which can then be used to construct candidate direction regions and evaluate the effectiveness of the wake-up trigger signal.

[0044] The aforementioned standby state refers to the low-power listening mode in which the intelligent interactive device operates when it has not established a valid session connection with the user. In standby state, the intelligent interactive device can continuously collect ambient sound through the microphone array and save it to the cache to perform the task of detecting wake-up trigger signals. Only after receiving a definite wake-up trigger signal and passing subsequent multi-stage joint confirmation will it switch to the active state.

[0045] In one alternative embodiment, considering the acquisition of multi-channel acoustic information of the environment in which the smart interactive device is located before and after the detection time of the wake-up trigger signal, a complete acoustic time window containing historical context and real-time response can be constructed, thereby providing a data foundation with temporal continuity and spatial richness for subsequent evaluation of the effectiveness of the wake-up trigger signal and construction of candidate positioning directions.

[0046] Based on this, the wake-up system of intelligent interactive devices (hereinafter referred to as the wake-up system) can utilize the multi-channel microphone array in the intelligent interactive device to continuously sample the environment in which the intelligent interactive device is located, and can store the collected audio data into a circular buffer to construct a historical time window. Upon detecting a wake-up trigger signal, the wake-up system can trigger a data acquisition mechanism to extract multi-channel audio data within a preset time range before the detection of the wake-up trigger signal from the circular buffer, as the first ambient sound. The wake-up system can also extract multi-channel audio data within a preset time range after the detection of the wake-up trigger signal, as the second ambient sound. By combining the first and second ambient sounds, the wake-up system ensures that subsequent steps can make progressive collaborative decisions based on the complete acoustic context.

[0047] For example, when a smart interactive device is in standby mode, the wake-up system can collect ambient audio data in real time through a microphone array and write it to a circular buffer. Once the wake-up system detects a wake-up word matching a preset rule on a reference channel, it can trigger the wake-up response process. The wake-up system can immediately retrieve multi-channel audio data from the circular buffer within a preset time window before the wake-up trigger time as the first ambient sound, and can continue to collect multi-channel audio data within the same preset time window after the wake-up trigger time as the second ambient sound.

[0048] In another optional embodiment, when the smart interactive device is in standby mode, the wake-up system can continuously acquire ambient audio streams through a microphone array and analyze these ambient audio streams using a joint sound source localization and wake-up detection model. When the joint sound source localization and wake-up detection model recognizes a wake-up intention, the wake-up system can extract a first ambient sound segment of a preset duration before the wake-up trigger time and a second ambient sound segment of a preset duration after the wake-up trigger time from the audio stream cache, based on the timestamp of the wake-up trigger time.

[0049] In another optional embodiment, when the smart interactive device is in standby mode, the wake-up system can perform real-time frame-segmentation processing of multi-channel audio using a microphone array, and can utilize a deep learning-based wake word detector to detect wake words in the frame-segmented multi-channel audio. If the wake word detector outputs a prediction result with a confidence level exceeding a threshold, the wake-up system can use the wake-up trigger time as a reference to obtain a first ambient sound within a preset window before the wake-up trigger and a second ambient sound within a preset window after the wake-up trigger from the audio frame sequence.

[0050] Step S104: Based on the first ambient sound and the second ambient sound, determine the acoustic energy of the wake-up trigger signal and multiple candidate positioning areas.

[0051] The aforementioned acoustic energy can refer to the average audio intensity index of the wake-up trigger signal determined based on the first and second ambient sounds during the wake-up trigger signal detection process. This acoustic energy can be expressed as an RMS (Root Mean Square) value or a dBFS (Decibels relative to Full Scale) value, used to determine the authenticity and validity of the wake-up trigger signal through a pre-set threshold, thereby filtering out weak far-field sounds, occasional noise, and echo residue.

[0052] The aforementioned candidate localization region can refer to a set of directional fan-shaped regions formed after spatial proximity clustering and temporal continuous activity analysis, based on the angular probability distribution or angle output by the sound source localization model, using the first and second ambient sounds. This candidate localization region can be used to replace a single-directional output, thereby preserving multiple possible interaction directions in multi-sound-source or complex reflection environments, facilitating subsequent evaluation of the effectiveness of the wake-up trigger signal.

[0053] In an alternative embodiment, considering that an acoustic context containing spatial distribution information can be constructed by utilizing a first ambient sound and a second ambient sound, after a single wake-up trigger signal is detected, the acoustic energy of the wake-up trigger signal can be quantified and multiple candidate localization regions can be constructed to achieve initial screening of the validity of the wake-up trigger signal and refined analysis of its spatial source, thereby improving the robustness of wake-up decision-making and the accuracy of direction confirmation under complex sound fields.

[0054] Based on this, the wake-up system can determine the acoustic energy of the wake-up trigger signal based on the first and second ambient sounds, and compare it with a preset energy threshold to determine whether the wake-up trigger signal meets the interaction validity conditions. If the acoustic energy meets the preset energy threshold, the wake-up system can input the first and second ambient sounds into the sound source localization model for processing to obtain an angular probability distribution within the range of 0 to 360 degrees. The wake-up system can then perform cluster analysis on high-confidence candidate angles based on this angular probability distribution, generating multiple candidate localization regions with spatial coverage, thus completing a coherent processing flow from acoustic energy validity verification to spatial candidate region construction.

[0055] For example, after acquiring the first and second ambient sounds, the wake-up system can determine the RMS or dBFS of the first and second ambient sounds within a preset time window as the acoustic energy of the wake-up trigger signal. The wake-up system can also input the first and second ambient sounds into a convolutional-time-frequency attention structure sound source localization network for inference, obtaining a probability distribution on an angle grid from 0 to 360 degrees or a list of candidate angles containing angle confidence. Based on this, the wake-up system can select original angles from the candidate angle list whose confidence is higher than a preset threshold, and cluster them according to the spatial proximity of the angles to form multiple fan-shaped candidate localization regions.

[0056] In another alternative embodiment, the wake-up system can utilize a deep learning-based speech activity detection model to segment the first and second ambient sounds, extract feature segments containing wake-up keywords, and determine the acoustic energy of the wake-up trigger signal by determining the RMS of the feature segments on a reference channel. Furthermore, the wake-up system can use the first and second ambient sounds as multi-channel inputs, connected to a beamforming spatial spectrum analysis module based on time delay difference estimation, to determine the spatial spectral power in each direction. Peak directions exceeding a preset threshold are selected as the original sound source directions, and based on the distribution density of these original sound source directions on the time axis, a clustering algorithm is used to merge spatially close and temporally continuous sound source directions into multiple candidate localization regions.

[0057] In another alternative embodiment, the wake-up system can also, based on a first ambient sound and a second ambient sound, utilize an adaptive beamforming algorithm to scan the directions of potential sound sources in a 360-degree omnidirectional range with a fixed step size, perform beam enhancement for each scanning direction, and determine the energy of the enhanced output signal. The wake-up system can use the energy peaks in all scanning directions as the acoustic energy distribution of the wake-up trigger signal. Furthermore, the wake-up system can identify multiple local maxima points with energy significantly higher than the background noise level based on the energy distribution map, and can define the angles corresponding to these maxima points and their neighborhood ranges as multiple candidate localization regions.

[0058] Step S106: Based on acoustic energy and multiple candidate localization regions, determine whether the wake-up trigger signal is valid.

[0059] In one alternative embodiment, considering that multi-channel acoustic information is susceptible to interference from weak far-field sound, environmental noise and echoes in complex home sound fields, a multi-dimensional verification mechanism from signal strength to spatial direction can be constructed by combining acoustic energy statistics with joint analysis of candidate positioning regions. This can effectively extract the real user interaction intent and suppress interference from non-target sound sources.

[0060] Based on this, the wake-up system can compare acoustic energy with a preset fixed threshold or a dynamic threshold adaptively determined based on ambient noise. If the acoustic energy meets the interaction validity criteria, the wake-up system can determine that the wake-up trigger signal is qualified for a preliminary response. Furthermore, the wake-up system can perform local fine-search and beam enhancement processing on each candidate positioning region to facilitate secondary wake-up confirmation of the wake-up trigger signal, thereby completing a multi-dimensional collaborative confirmation of the wake-up trigger signal's validity.

[0061] For example, the wake-up system can compare acoustic energy with a preset fixed energy threshold or an energy threshold adaptively calculated based on ambient noise. If the acoustic energy is below the threshold, the wake-up trigger signal is deemed invalid. If the acoustic energy is above the threshold, the wake-up system can further check whether the current moment is within the cooldown period after the end of the previous valid wake-up event. If the current moment is within the cooldown period, it is determined to be a duplicate trigger and therefore invalid. If the current moment is not within the cooldown period, the wake-up trigger signal is initially deemed valid, and the system proceeds to the subsequent direction confirmation process. In the direction confirmation process, the wake-up system can analyze the audio signal corresponding to each candidate positioning area to determine whether a wake-up keyword exists. Only when a wake-up keyword exists in the audio signal corresponding to any candidate positioning area is the wake-up trigger signal deemed valid, thus enabling secondary wake-up confirmation of the wake-up trigger signal.

[0062] In another optional embodiment, the wake-up system can statistically analyze the temporal continuity and activity of each candidate positioning region within a historical time window, and normalize this temporal continuity and activity. Then, it can combine this with a pre-set fixed energy threshold to determine whether the acoustic energy meets the basic activation conditions. If the acoustic energy meets the basic activation conditions, and the weighted combination of the temporal continuity and activity of any candidate positioning region exceeds a preset joint threshold, the wake-up trigger signal can be deemed valid.

[0063] Step S108: When the wake-up trigger signal is valid, control the smart interactive device to enter the wake-up state from the standby state, wherein the wake-up state is used for voice interaction.

[0064] The aforementioned wake-up state refers to the working state that a smart interactive device switches into from standby mode after detecting a valid wake-up trigger signal, used to initiate and maintain subsequent voice interaction processes. In the wake-up state, the smart interactive device is no longer in idle listening mode, but performs voice interaction with the user.

[0065] In one alternative embodiment, considering that the wake-up system has sufficiently filtered out weak far-field sounds, environmental noise, and interference signals from repeated triggering during the determination of the validity of the wake-up trigger signal, the validity determination result is more accurate.

[0066] Based on this, once the wake-up trigger signal is confirmed to be valid, the wake-up system can execute a state transition instruction to switch the smart interactive device from standby to wake-up state. Furthermore, during the state transition process, the wake-up system can also use a timer to suppress wake-up trigger signals that reappear within a short period, thereby preventing the smart interactive device from responding multiple times consecutively in a short time, which could lead to processing overload or confusing feedback.

[0067] In this embodiment, a method is adopted to detect the wake-up trigger signal of the smart interactive device, acquire the first ambient sound and the second ambient sound of the environment in which the smart interactive device is located; determine the acoustic energy of the wake-up trigger signal and multiple candidate positioning regions based on the first ambient sound and the second ambient sound; determine whether the wake-up trigger signal is valid based on the acoustic energy and the multiple candidate positioning regions; and control the smart interactive device to enter the wake-up state when the wake-up trigger signal is valid. By synchronously determining the acoustic energy of the wake-up trigger signal and multiple candidate positioning regions through the first ambient sound and the second ambient sound, the acoustic energy is used for initial screening to filter weak sound or noise interference in the far field. The potential direction of the wake-up trigger signal in a multi-sound source environment is retained through multiple candidate positioning regions, avoiding misjudgment in complex reflection or reverberation scenarios by single direction estimation. Finally, the acoustic energy and candidate positioning regions are combined to jointly determine whether the wake-up trigger signal is valid. The smart interactive device is controlled to enter the wake-up state only after the wake-up trigger signal is confirmed to be valid. This achieves the purpose of effectively distinguishing the real wake-up trigger signal from complex background noise or echo interference, thereby solving the technical problem of poor accuracy of wake-up detection of smart interactive devices in complex sound fields in related technologies.

[0068] Furthermore, based on the first ambient sound and the second ambient sound, the acoustic energy of the wake-up trigger signal and multiple candidate positioning regions are determined, including: determining the environment type based on the first ambient sound; determining the acoustic energy based on the environment type, the first ambient sound, and the second ambient sound; and determining multiple candidate positioning regions based on the first ambient sound and the second ambient sound.

[0069] The aforementioned environment type can refer to the acoustic scene category determined by analyzing sounds such as silence, background noise, multiple voices, and television playback sounds in the first ambient sound. This environment type can be used to characterize the sound source composition, noise level, or reverberation characteristics in the environment, so as to adaptively calibrate the acoustic energy judgment threshold according to the environment type.

[0070] In one alternative embodiment, considering that the ambient sound collected by the multi-channel microphone array in the complex home sound field contains rich time-frequency features and spatial distribution information, by analyzing the first ambient sound and the second ambient sound, the acoustic energy level of the wake-up trigger signal can be accurately quantified, and multiple candidate positioning regions characterizing the location of the sound source of the wake-up trigger signal can be constructed.

[0071] Based on this, the wake-up system can extract the spectral characteristics, noise floor distribution, and reverberation properties of the first ambient sound, and use a preset environmental classification model or statistical laws to classify the current sound field into one or more combinations of static background noise, dynamic multi-source noise, or high reverberation noise, thereby clearly identifying the environmental type of the current smart interactive device. On this basis, the wake-up system can select audio data within the time window of the wake-up trigger signal from the first and second ambient sounds, according to the energy determination strategy corresponding to this environmental type, to determine the acoustic energy of the wake-up trigger signal.

[0072] Furthermore, the wake-up system can input the first and second ambient sounds into a multi-channel sound source localization network. This network can extract the amplitude spectrum, phase difference, and GCC-PHAT (Generalized Cross-Correlation with Phase Transform) features of multiple channels through short-time Fourier transform. It can also fuse spatiotemporal features using convolution and time-frequency attention mechanisms to generate an angular probability distribution from 0 to 360 degrees. The wake-up system can then perform spatial proximity clustering on angles with high probability distributions to merge consecutive or nearby angle points into fan-shaped regions, thereby forming candidate localization regions containing multiple potential sound source directions.

[0073] Furthermore, based on the first ambient sound, the environment type is determined, including: extracting features from the first ambient sound to obtain its acoustic features; and determining the environment type based on the acoustic features.

[0074] The aforementioned acoustic features can refer to the numerical representations of the sound physical properties or semantic information of the first ambient sound extracted after preprocessing and transforming it. These acoustic features may take the form of, but are not limited to, amplitude and phase spectra obtained from STFT (Short-Time Fourier Transform), GCC-PHAT features based on cross-correlation functions, sound intensity vectors characterizing the spatial distribution of sound sources, time delay difference features characterizing the time difference between channels, phase difference features characterizing the frequency correlation between channels, and high-dimensional temporal or time-frequency attention features extracted through deep learning models.

[0075] In one alternative embodiment, considering the challenges faced by multi-channel acoustic information in complex home sound fields, such as background noise interference, multi-source aliasing, and echo reflection, by determining the environment type based on the first ambient sound, the sound field characteristics can be quantified using the statistical regularity of acoustic features, thereby providing a basis for determining the acoustic energy of the subsequent wake-up trigger signal.

[0076] Based on this, the wake-up system can extract features from the acquired ambient sound to obtain acoustic features reflecting the physical properties of the sound field in the current environment. The system can then input these extracted acoustic features into a pre-trained environment classification model or rule-based judgment module to determine the distribution probability or distance metric of the acoustic features in multiple predefined environment category spaces, thereby matching more similar environment types. This process enables the mapping from raw acoustic signals to semantic environmental states, allowing the wake-up system to accurately determine the acoustic energy of the wake-up trigger signal based on the identified environment type.

[0077] Furthermore, based on acoustic characteristics, the environment type is determined, including: based on acoustic characteristics, determining the signal-to-noise ratio and attenuation of the first ambient sound; based on the signal-to-noise ratio and attenuation, determining the sound source distance and reverberation intensity of the first ambient sound; and based on the sound source distance and reverberation intensity, determining the environment type.

[0078] The signal-to-noise ratio (SNR) mentioned above refers to the ratio of speech signal power to background noise power. It can be used to measure the clarity of the first ambient sound. A higher SNR indicates that the first ambient sound is more obvious relative to the noise.

[0079] The aforementioned attenuation can refer to the reduction in sound pressure level caused by factors such as air absorption, geometric diffusion, and obstruction during the propagation of the first ambient sound. It can be used to quantify the degree of sound intensity loss between the sound source and the microphone array of the smart interactive device.

[0080] The aforementioned sound source distance can refer to the straight-line physical distance between the sound-emitting object and the microphone array. This sound source distance reflects how close or far the sound source is relative to the intelligent interactive device.

[0081] The reverberation intensity mentioned above can refer to the energy level or duration of the reverberation component formed after multiple reflections of the first ambient sound within a closed space. This reverberation intensity can be used to characterize the acoustic reflection characteristics of the environment in which the smart interactive device is located. A higher reverberation intensity means a larger space or less sound-absorbing material, resulting in a more significant echo effect.

[0082] In one alternative embodiment, considering that acoustic features can objectively characterize the physical properties of the sound field in which the smart interactive device is located, a reliable environmental context can be provided for accurately determining the acoustic energy of the wake-up trigger signal.

[0083] Based on this, the wake-up system can obtain the signal-to-noise ratio and attenuation parameters corresponding to the first ambient sound, and can calculate the sound source distance and reverberation intensity according to the numerical mapping relationship between the signal-to-noise ratio and the attenuation. On this basis, the wake-up system can match the preset environmental category label according to the combined characteristics of the sound source distance and reverberation intensity to complete the deterministic classification of the environmental type, thereby determining the environmental type of the environment in which the intelligent interactive device is located.

[0084] Further, based on the environment type, the first ambient sound, and the second ambient sound, the acoustic energy is determined, including: determining a first interception window based on the environment type; intercepting the first ambient sound and the second ambient sound based on the first interception window to obtain a first reference audio; and determining the acoustic energy based on the average energy of the first reference audio.

[0085] The aforementioned first extraction window can refer to a time interval determined based on the environment type, used to extract audio data for a fixed period of time from the first ambient sound and the second ambient sound. For example, the first extraction window can be 200ms to 1000ms, preferably 300ms to 500ms, while in far-field or reverberant scenarios, the first extraction window can be extended to about 1 second, thereby obtaining sufficiently stable acoustic context information for energy assessment.

[0086] The aforementioned first reference audio may refer to an audio segment extracted from the first ambient sound and the second ambient sound through the first capture window.

[0087] The aforementioned average energy can refer to the audio signal strength index reflecting the wake-up trigger signal obtained after statistical processing of the first reference audio. The average energy can be represented by RMS, normalized RMS value or dBFS, and can be used to compare with a preset threshold to determine the validity of the wake-up event and filter out invalid triggers.

[0088] In one alternative embodiment, considering that the dynamic adaptation time window is used to accurately extract multi-channel audio before and after the wake-up trigger signal, the energy estimation bias of the fixed window in reverberation environment or different signal-to-noise ratio scenarios can be eliminated, thereby improving the robustness and accuracy of the determined acoustic energy.

[0089] Based on this, the wake-up system can determine the first interception window according to the environment type. Specifically, a shorter first interception window can be set for quiet environments, and a longer first interception window can be set for noisy or reverberant environments to accommodate more stable acoustic features.

[0090] The wake-up system can extract a first reference audio segment that is temporally adjacent to and spatially correlated with the wake-up trigger signal based on a determined first interception window, the first ambient sound before and after the detected wake-up trigger signal and the corresponding second ambient sound in the time domain.

[0091] The wake-up system can perform signal processing on the intercepted first reference audio to determine its root mean square value or logarithmic amplitude, thus obtaining the average energy. Based on this, the wake-up system can map this average energy into an acoustic energy index, which serves as a quantitative basis for judging whether the wake-up trigger signal has interactive validity.

[0092] Furthermore, based on the first ambient sound and the second ambient sound, multiple candidate positioning regions are determined, including: determining a second interception window based on the time when the wake-up trigger signal is detected; intercepting the first ambient sound and the second ambient sound based on the second interception window to obtain a second reference audio; and determining multiple candidate positioning regions based on the second reference audio.

[0093] The aforementioned second capture window can refer to a time interval determined based on the time point at which the wake-up trigger signal is detected, used to extract audio data for a fixed period from the first and second ambient sounds. This second capture window can cover audio segments before and after the time the wake-up trigger signal is detected, thereby providing a multi-channel audio data foundation containing complete acoustic context for subsequent directional localization.

[0094] The aforementioned second reference audio can refer to an audio segment extracted from the first and second ambient sounds based on the second extraction window, encompassing the time before and after the detection of the wake-up trigger signal. This second reference audio can be used as input to a direction analysis model to infer multiple candidate localization regions using a sound source localization algorithm or a deep learning model.

[0095] In one alternative embodiment, considering the timing of the wake-up trigger signal, a second reference audio containing the acoustic context before and after the wake-up trigger signal is constructed. This allows for the acquisition of more complete sound source radiation characteristics and environmental propagation features, thereby providing an acoustic basis with temporal continuity and rich information for the spatial positioning of the wake-up trigger signal. This, in turn, improves the robustness and accuracy of sound source direction estimation in complex reverberation or multi-source interference environments.

[0096] Based on this, the wake-up system can determine a second interception window by extending a preset time length forward and backward according to the specific time when the wake-up trigger signal is detected. This second interception window can be used to cover the acoustic segments before and after the wake-up trigger signal is emitted.

[0097] Based on this, the wake-up system can use the second interception window to synchronously intercept the pre-stored first and second ambient sounds, extracting multi-channel audio data within the corresponding time period as the second reference audio. The wake-up system can input the second reference audio into the sound source localization model, and generate an angle probability distribution within the range of 0 to 360 degrees by calculating the time delay difference, phase difference, or spatiotemporal feature mapping between multiple channels. Furthermore, it can cluster these angles based on spatial proximity and temporal continuity of activity, thereby determining multiple candidate localization regions representing the location of the wake-up trigger signal.

[0098] Furthermore, based on the second reference audio, multiple candidate positioning regions are determined, including: inputting the second reference audio into a direction analysis model, processing the second reference audio using the direction analysis model to obtain positioning probability distributions in multiple directions; selecting multiple candidate positioning directions from the multiple directions based on the positioning probability distributions; and determining multiple candidate positioning regions based on the multiple candidate positioning directions.

[0099] The aforementioned directional analysis model can refer to a model used to process multi-channel historical audio data to estimate the spatial location of a sound source. The specific implementation of this directional analysis model can be a traditional model based on time delay difference, phase difference, or conventional array signal processing algorithms such as GCC-PHAT or SRP-PHAT (Steered Response Power with Phase Transform), or it can be a deep learning model employing a convolutional-time-frequency attention structure. This directional analysis model can identify the location of a sound source by extracting the spatiotemporal features of a second reference audio.

[0100] The aforementioned location probability distribution can refer to the mathematical representation of the probability that a sound source is located on a discrete angular grid or a continuous angular interval within the range of 0 to 360 degrees, output by the directional analysis model after inference processing of the second reference audio. This location probability distribution can be represented as a probability density distribution curve covering the entire angular range, or as a candidate list containing specific angular values ​​and their corresponding confidence scores, thereby quantifying the credibility of each direction as the true sound source direction.

[0101] The aforementioned candidate positioning directions can refer to angle values ​​that are selected from the positioning probability distribution, have high confidence, or are near the peak of the probability distribution. These angles can represent the sound source location initially determined by the wake-up system, and thus can serve as the basic unit for subsequently constructing candidate positioning areas.

[0102] In one alternative embodiment, considering the use of a directional analysis model to extract spatial acoustic features from the second reference audio, and the construction of candidate localization regions by combining probability distribution and spatial clustering strategies, it is possible to systematically mine potential sound source locations from multi-dimensional acoustic information and preserve the spatial diversity in a multi-sound source environment.

[0103] Based on this, the wake-up system can input the second reference audio into a pre-trained direction analysis model. Utilizing the time-frequency local feature extraction layer, inter-channel relationship fusion layer, and temporal attention layer within this model, the system can encode the second reference audio's features and map its spatial correlation. Furthermore, through an angle output layer, the fused spatiotemporal features can be converted into a location probability distribution covering all angles. The wake-up system can then select multiple candidate location directions from this distribution, choosing those with probability peaks higher than a preset threshold or ranking relatively high. Based on this, the wake-up system can perform circular clustering based on the spatial proximity and temporal continuity of these candidate location directions, merging spatially adjacent and continuously occurring candidate location directions into multiple candidate location regions.

[0104] Furthermore, based on acoustic energy and multiple candidate positioning regions, it is determined whether the wake-up trigger signal is valid, including: determining whether the acoustic energy is greater than a preset energy; if the acoustic energy is greater than the preset energy, performing time delay compensation on the initial audio signals of multiple candidate positioning regions to obtain multiple enhanced audio signals corresponding to multiple candidate positioning regions, and determining whether the wake-up trigger signal is valid based on the multiple enhanced audio signals.

[0105] The aforementioned initial audio signal can refer to audio data retrieved back from the buffer of a multi-channel microphone array, which is temporally adjacent to the wake-up trigger signal and has not undergone beamforming enhancement processing for any spatial direction. This initial audio signal may include the wake-up trigger signal and retain the original time delay and phase information of the sound source in space, serving as the basic input for subsequent directional analysis and signal enhancement.

[0106] The aforementioned enhanced audio signal can refer to the audio signal in the initial audio signal corresponding to the center direction of any candidate positioning region, which is compensated for with integer or fractional delay based on the geometric time delay difference in the center direction, and the audio signal obtained by the delay superposition can enhance the energy of the initial audio signal in the center direction and suppress the interference of the initial audio signal in directions other than the center direction.

[0107] In one alternative embodiment, considering the use of acoustic energy for initial effectiveness screening, the intensity characteristics of the wake-up trigger signal can be quantitatively evaluated, thereby eliminating weak far-field sound and occasional noise interference. Furthermore, time delay compensation based on the initial audio signal of the candidate positioning region allows for the use of spatial filtering techniques to suppress interference from other regions on the initial audio signal, facilitating secondary verification of the wake-up trigger signal.

[0108] Based on this, the wake-up system can compare the acoustic energy with the preset energy. If the acoustic energy is greater than the preset energy, the wake-up system can determine that the wake-up trigger signal has interactive validity and proceed to the next process.

[0109] The wake-up system can construct a corresponding beamformer for each candidate positioning region. Based on the azimuth parameters of the candidate positioning region and the geometric position of the microphone array, it determines the relative time delay of each audio acquisition channel of the microphone array. The wake-up system can then perform integer or fractional delay compensation on the audio signals of each audio acquisition channel according to this relative time delay, and then sum them to obtain the enhanced audio signal corresponding to the candidate positioning region.

[0110] The wake-up system can input each enhanced audio signal into the wake word recognition detector for secondary detection, and can combine the confidence score output by the secondary detection with the energy characteristics of the enhanced audio signal to jointly score the candidate localization region, thereby determining whether the wake-up trigger signal is effective.

[0111] Furthermore, time delay compensation is performed on the initial audio signals of multiple candidate positioning regions to obtain multiple enhanced audio signals corresponding to the multiple candidate positioning regions, including: determining the search range and search step size of any candidate positioning region; determining multiple search directions within any candidate positioning region based on the search range and search step size; and performing time delay compensation on the initial audio signals of multiple candidate positioning regions based on the multiple search directions to obtain multiple enhanced audio signals.

[0112] The search range mentioned above can refer to the angular interval extending to the left and right sides from the center direction of the candidate location area, which can be used to define the directional boundaries of the local fine search. For example, the search range can be 10 to 30 degrees to the left and right of the center direction.

[0113] The search step size mentioned above can refer to the angular interval between two adjacent search directions within a set search range, and can be used to determine the fineness of the directional search. For example, the search step size can be 2 to 5 degrees.

[0114] The aforementioned search direction can refer to a specific angular point within the candidate localization area, determined based on the search range and search step size, used for beamforming and time delay compensation. This search direction can represent the direction of potential sound sources to be verified within the candidate localization area.

[0115] In one alternative embodiment, considering that multiple search directions are used to compensate for the time delay of the initial audio signal of the candidate localization area, a high spatial resolution acoustic evidence chain can be constructed. This allows for precise matching of sound source directions by refining the spatial search granularity in complex reverberation and multi-source interference environments, thereby significantly improving the confidence of secondary wake-up confirmation.

[0116] Based on this, the wake-up system can determine the search range and search step size of any candidate positioning area. The search range can be a local angle interval near the center direction of the candidate positioning area, and the search step size can determine the density of discrete direction points within the search range, so as to balance computational complexity and positioning accuracy.

[0117] The wake-up system can generate multiple discrete search directions within the candidate localization area based on the search range and search step size. These search directions can form a spatial sampling grid for the candidate area. Based on these multiple search directions, the wake-up system can determine the time delay of each audio acquisition channel in the microphone array relative to that search direction.

[0118] The wake-up system can perform corresponding time delay compensation operations on the initial audio signals corresponding to each audio acquisition channel based on the time delay, to obtain multiple compensated audio signals corresponding to each search direction. Based on this, the wake-up system can superimpose or weightedly fuse these multiple compensated audio signals to obtain the enhanced audio signal for the candidate positioning region. Using the above steps, the wake-up system can determine multiple enhanced audio signals corresponding to multiple candidate positioning regions.

[0119] Furthermore, based on multiple search directions, time delay compensation is performed on the initial audio signals of multiple candidate positioning regions to obtain multiple enhanced audio signals. This includes: acquiring the geometric position of the microphone array of the intelligent interactive device, as well as the sound source direction angle, sound velocity, and sampling rate of the initial audio signal; determining the time delay of the initial audio signal in multiple search directions based on the geometric position, sound source direction angle, sound velocity, and sampling rate; compensating the initial audio signal in multiple search directions based on the time delay to obtain multiple compensated audio signals; and fusing the multiple compensated audio signals to obtain the enhanced audio signal corresponding to any candidate positioning region.

[0120] The aforementioned geometric position can refer to the specific coordinate distribution or relative positional relationship of each microphone in the microphone array of a smart interactive device in physical space.

[0121] The aforementioned sound source direction angle can refer to the angle between the expected direction of the audio signal source and the microphone array reference axis, and can be used to define the target direction of beamforming or time delay compensation.

[0122] The aforementioned speed of sound can refer to the propagation speed of the initial audio signal, which is a fundamental physical constant for determining the time difference in the arrival of sound waves at different microphone positions.

[0123] The sampling rate mentioned above can refer to the number of sample points collected per second when the microphone array converts continuous analog audio signals into discrete digital signals.

[0124] The aforementioned delay can refer to the time difference in arrival caused by the difference in path length when the initial audio signal propagates from the sound source to microphones at different positions in the microphone array.

[0125] The aforementioned compensated audio signal can refer to the intermediate processing result where, after time delay adjustment, multi-channel audio signals from different search directions are aligned on the time axis, so that the sound source signals in that direction are superimposed in phase and the non-target signals are canceled out of phase, thereby enhancing the signal-to-noise ratio.

[0126] In one alternative embodiment, considering the time delay compensation of the initial audio signals of multiple candidate positioning regions to construct an enhanced audio signal, it is possible to achieve focused enhancement of audio energy in a fixed direction and suppress noise interference from other directions, thereby improving the accuracy of wake-up trigger signal detection and the reliability of direction confirmation in complex multi-sound-source and reverberant environments.

[0127] Based on this, the wake-up system can obtain the geometric coordinates of each microphone in the microphone array of the smart interactive device, and determine the sound source direction angle, known sound velocity, and audio sampling rate corresponding to the initial audio signal to be processed. Using this geometric position, sound source direction angle, sound velocity, and sampling rate, the wake-up system can determine the specific time delay values ​​of the initial audio signal in multiple search directions. Based on this, the wake-up system can compensate the initial audio signal accordingly in multiple search directions according to the determined time delay values ​​to obtain multiple compensated audio signals. The wake-up system can then perform linear superposition and fusion processing on the multiple compensated audio signals to obtain the enhanced audio signal corresponding to the candidate localization region.

[0128] Furthermore, based on multiple enhanced audio signals, the validity of the wake-up trigger signal is determined, including: based on multiple enhanced audio signals, determining multiple secondary wake-up confidence levels corresponding to multiple candidate positioning regions; and determining that the wake-up trigger signal is valid if any secondary wake-up confidence level is greater than a preset confidence level.

[0129] The aforementioned secondary wake-up confidence score refers to the probability value or matching score obtained by performing secondary wake-up word detection on the enhanced audio signal corresponding to the candidate localization region. This score can be used to quantify the reliability of confirming whether a valid wake-up command issued by the user actually exists in the direction of the sound source.

[0130] In one alternative embodiment, considering that the raw signals collected by multi-channel microphone arrays in complex home sound fields often contain reverberation, background noise and multi-source interference, directly judging the validity of the wake-up trigger signal based on a single frame or single channel data is easily affected by instantaneous noise fluctuations and may lead to misjudgment.

[0131] Based on this, the wake-up system can receive multiple enhanced audio signals, each corresponding to a different spatial angle on a preset orientation grid. The wake-up system can independently input each enhanced audio signal into a wake-up word detection model. This model can extract audio features to determine a matching score, thereby generating a corresponding secondary wake-up confidence score for each candidate localization region. This enables parallel verification of the existence of a valid wake-up trigger signal in each spatial direction.

[0132] The wake-up system compares the calculated secondary wake-up confidence scores with a preset confidence level to identify candidate localization regions where the secondary wake-up confidence score is higher than the preset confidence level. Once the secondary wake-up confidence score of any candidate localization region is detected to be greater than the preset confidence level, the wake-up system can determine that there is a high probability of a wake-up trigger signal in that candidate localization region, thus confirming the validity of the wake-up trigger signal. The above steps, by combining spatial signal enhancement with temporal semantic confirmation, achieve accurate determination of the validity of wake-up trigger signals in complex acoustic environments.

[0133] Furthermore, before controlling the intelligent interactive device to enter the wake-up state from the standby state, the method also includes: determining the audio parameters corresponding to multiple candidate positioning areas based on the second ambient sound; and determining the target positioning area from the multiple candidate positioning areas based on multiple enhanced audio signals, multiple secondary wake-up confidence levels, and audio parameters.

[0134] The aforementioned audio parameters can refer to comprehensive quantitative indicators used to characterize the audio characteristics of candidate localization areas, including but not limited to the activity level of candidate directional areas and the temporal continuity of the candidate localization area within a historical time window.

[0135] The aforementioned target localization area can refer to the final directional sector region that scores highly according to the scoring mechanism among multiple candidate localization regions and possesses sufficient confidence. This target localization area can represent the true location of the sound source that the intelligent interactive device has identified as the user's wake-up trigger signal.

[0136] In one optional embodiment, to ensure the accuracy of interaction between the smart interactive device and the user, the wake-up system also needs to determine the target location area that issued the wake-up trigger signal. Based on this, the wake-up system can extract audio parameters corresponding to multiple candidate location areas from the second ambient sound. The wake-up system can also determine a joint score for each of the multiple candidate location areas based on multiple enhanced audio signals, multiple secondary wake-up confidence levels, and audio parameters corresponding to these multiple candidate location areas, and can determine the target location area from the multiple candidate location areas based on this joint score.

[0137] Furthermore, the audio parameters corresponding to each candidate positioning region include the audio activity and audio temporal continuity of that candidate positioning region. Based on multiple enhanced audio signals, multiple secondary wake-up confidence levels, and audio parameters, a target positioning region is determined from multiple candidate positioning regions, including: determining the weights of the audio activity, secondary wake-up confidence level, audio temporal continuity, and enhanced audio signals for any candidate positioning region; based on the weights, performing weighted fusion of the audio activity, secondary wake-up confidence level, audio temporal continuity, and enhanced audio signals to obtain a confidence index value for any candidate positioning region; based on the multiple confidence index values ​​corresponding to multiple candidate positioning regions, a target positioning region is determined from multiple candidate positioning regions, wherein the target confidence index value corresponding to the target positioning region is greater than the confidence index values ​​corresponding to the candidate positioning regions other than the target positioning region.

[0138] The aforementioned audio activity refers to the frequency or proportion of sound sources detected within a candidate localization region. This audio activity can be quantified by statistically analyzing the proportion of angular candidate points appearing within a continuous time period in that candidate localization region, reflecting the persistence and significance of the sound source within that region.

[0139] The aforementioned audio temporal continuity can refer to the proportion or time span of the candidate localization region repeating in adjacent or consecutive historical time segments. It can be used to assess the stability of the sound source location and eliminate instantaneous misjudgment of direction caused by echoes, noise abrupt changes, or brief interference.

[0140] The aforementioned confidence index value can refer to a single numerical value determined by a weighted fusion algorithm, used to comprehensively evaluate the credibility of a candidate localization region as a target localization region. This value can be fused from normalized audio activity, second wake-up confidence, audio temporal continuity, and enhanced audio signals. The higher the confidence index value, the greater the likelihood that the candidate localization region is the target localization region.

[0141] The aforementioned target confidence index value can refer to the confidence index value with the higher score after weighted fusion calculation among all candidate positioning regions, and this target confidence index value corresponds to the target positioning region.

[0142] In an alternative embodiment, considering that by introducing a weighted fusion mechanism, the acoustic characteristics of candidate positioning regions can be comprehensively evaluated, thereby accurately determining the target positioning region.

[0143] Based on this, the wake-up system can extract the audio activity, secondary wake-up confidence, audio temporal continuity, and weight parameters corresponding to the enhanced audio signal for each candidate localization region, and normalize the audio activity, secondary wake-up confidence, audio temporal continuity, and the intensity of the enhanced audio signal. The wake-up system can then use these weights to perform a linear weighted fusion operation on the normalized audio activity, secondary wake-up confidence, audio temporal continuity, and enhanced audio signal to generate a confidence index value reflecting the probability of the candidate localization region as the target localization region. Based on this, the wake-up system can compare the confidence index values ​​of all candidate localization regions and select the region with the larger confidence index value as the target localization region, thereby achieving a precise mapping from multidimensional acoustic evidence to a single deterministic spatial target.

[0144] For example, the process by which the wake-up system performs a linear weighted fusion operation on the normalized audio activity, secondary wake-up confidence, audio temporal continuity, and the intensity of the enhanced audio signal can be represented by the following formula:

[0145] ;

[0146] In the formula, This indicates the audio activity level of the candidate location region. This indicates the intensity of the enhanced audio signal in the candidate localization region. This indicates the confidence level of the second wake-up in the candidate location region. This indicates the audio temporal continuity of the candidate localization region. , , and They represent , , and The weight, for example, It can be 0.25. It can be 0.25. It can be 0.35. It can be 0.15.

[0147] Furthermore, after controlling the intelligent interactive device to enter the wake-up state from the standby state, the method also includes: collecting the interaction parameters of the intelligent interactive device during the voice interaction process; determining whether the interaction parameters meet the preset conditions; and controlling the intelligent interactive device to enter the standby state from the wake-up state if the interaction parameters do not meet the preset conditions.

[0148] The aforementioned interaction parameters refer to a set of quantitative indicators used to determine the validity of the current session state of the intelligent interactive device during voice interaction. These may include, but are not limited to, the average energy of detected voice activity, the continuous duration of valid voice, and the echo protection period after the local prompt tone is played. If the interaction parameters fail to meet the preset conditions simultaneously, the wake-up system can determine that the user has no further interaction with the intelligent interactive device, thereby triggering the control logic to return from the wake-up state to the standby state.

[0149] In one alternative embodiment, considering the presence of multiple sound source interference, weak far-field sound, and environmental noise fluctuations in complex home sound field environments, directly relying on a single instantaneous detection can easily lead to unstable wake-up state maintenance or misjudgment of the end of interaction, thereby improving the robustness and naturalness of voice interaction control.

[0150] Based on this, the wake-up system can continuously collect acoustic energy, effective voice duration, and local feedback protection period status during voice interaction of the intelligent interactive device as interaction parameters. The wake-up system can compare these interaction parameters with preset session duration requirements, minimum effective voice duration limits, and echo protection period thresholds. If the current acoustic energy reaches the session duration requirements, the effective voice duration reaches the preset minimum limit, and it is confirmed that the current voice activity is not within the echo protection period after the local prompt tone is played, the wake-up system can determine that the interaction parameters meet the preset conditions to maintain the wake-up state.

[0151] Conversely, if no continuous interactive behavior meeting the above conditions is detected within the preset session waiting time, the wake-up system can determine that the interaction parameters do not meet the preset conditions, and then control the smart interactive device to enter the standby state from the wake-up state, thereby realizing the progressive session state confirmation and smooth switching based on historical acoustic context information.

[0152] Furthermore, the interaction parameters include the voice energy of the interactive voice, the effective voice duration, and the prompt tone protection period; determining whether the interaction parameters meet the preset conditions includes: determining that the interaction parameters meet the preset conditions when the voice energy is greater than the preset energy, the effective voice duration is greater than the preset time, and the time when the interactive voice is emitted is not within the prompt tone protection period.

[0153] The aforementioned speech energy refers to the average intensity index of the interactive speech signal within different time windows, which can be expressed as RMS value or dBFS, and is used to quantify the loudness of the user's voice. Under the adaptive threshold mechanism, this speech energy must be higher than a preset energy to eliminate background noise interference.

[0154] The aforementioned effective voice duration refers to the length of time a user continuously emits voice signals that are recognized as valid, i.e., the time interval from the detection of the first valid voice frame to the termination of continuous valid voice. It can be used to distinguish between brief bursts of noise and sustained conversational intent. This effective voice duration must exceed a preset time to confirm that the user has a sustained intention to interact.

[0155] The aforementioned prompt tone protection period can refer to a fixed time period after the smart interactive device plays the feedback audio locally. During this prompt tone protection period, the wake-up system can prevent the collected sound from being identified as a new interactive command, in order to avoid echo interference caused by the sound from its own speaker entering the microphone through the air or structure.

[0156] The aforementioned transmission time can refer to the specific moment when the interactive voice signal is captured by the microphone array, and can be used to determine whether the voice signal falls within the time window of the prompt tone protection period. If the transmission time is within the prompt tone protection period, the voice can be marked as an internal echo rather than valid user input.

[0157] In one optional embodiment, considering that during the conversation continuation phase after wake-up confirmation, a multi-dimensional voice activity maintenance mechanism can be constructed by comprehensively evaluating acoustic parameters in three dimensions: acoustic energy, effective voice duration, and prompt tone protection period. This mechanism can accurately distinguish between the user's intention to initiate continuous interaction and background noise or system feedback, ensuring that the intelligent interactive device maintains the conversation state only when the user has a genuine need for continuous interaction and is not interfered with by local prompt tone echoes.

[0158] Based on this, the wake-up system can compare the acoustic energy of the real-time collected interactive voice with a preset energy, compare the duration of continuous effective voice with a preset time, and verify whether the time of the interactive voice transmission is within the echo protection period after the local prompt tone is played. Only when the acoustic energy is greater than the preset energy, the duration of the effective voice is greater than the preset time, and the time of the interactive voice transmission is not within the prompt tone protection period, can the wake-up system determine that the interaction parameters meet the preset conditions, and thus determine that the current state is a valid session hold state.

[0159] Furthermore, controlling the intelligent interactive device to enter the wake-up state from the standby state includes: constructing feedback prompt information based on the target positioning area, wherein the feedback prompt information is used to prompt the intelligent interactive device to enter the wake-up state; and outputting feedback prompt information based on various output devices on the intelligent interactive device.

[0160] The aforementioned feedback prompts refer to localized prompts issued by the smart interactive device to the user after confirming the user's wake-up intention, indicating that the device has entered a wake-up state or is ready for interaction. These feedback prompts can take various forms, including but not limited to voice prompts, color changes or flashing patterns of device indicator lights, and text or icon displays on the screen. This allows the user to clearly perceive, through multiple dimensions, that the smart interactive device has responded and is in a standby state.

[0161] The aforementioned output devices can refer to hardware components installed on smart interactive devices to present feedback prompts to users in a physical form. These may include, but are not limited to, speakers for playing sound, indicator light arrays or displays for displaying visual signals, and vibration motors for generating tactile feedback. These output devices can work together to provide intuitive and immediate status feedback.

[0162] In one alternative embodiment, considering that providing immediate feedback prompts can confirm to the user that the smart interactive device has successfully received the signal and entered the working state, it can reduce the user's waiting anxiety and clarify the boundaries of the interaction.

[0163] Based on this, the wake-up system can analyze the target positioning area for locating the user's sound source and generate feedback prompts containing visual or auditory instructions based on the target positioning area. On this basis, the wake-up system can control various output device modules configured on the smart interactive device, and according to the content and direction attributes of the feedback prompts, synchronously or sequentially execute actions such as adjusting light orientation, playing sound, or displaying on the screen, thereby completing a closed-loop output from status detection to user perception.

[0164] For ease of understanding, Figure 2 This is a schematic diagram of an optional feedback prompt information output process according to an embodiment of this application, such as... Figure 2 As shown, the wake-up system can construct feedback prompts based on the target location area, and can output the feedback prompts through methods such as voice broadcast, screen display and light signals.

[0165] Figure 3 This is a schematic diagram of an optional wake-up system decision-making process according to an embodiment of this application, such as... Figure 3As shown, the wake-up system may include a multi-channel audio acquisition layer, a wake-up initial screening layer, a direction joint confirmation layer, and a session closed-loop control layer. The wake-up system can utilize the multi-channel microphone array in the multi-channel audio acquisition layer to extract reference channels and store the extracted audio information in a circular buffer.

[0166] In the initial wake-up screening layer, the wake-up system can perform global wake-up detection based on audio information extracted from the reference channel, and can also perform initial energy screening based on historical audio information in the circular buffer. Furthermore, the wake-up system can also perform repetitive trigger suppression in the initial wake-up screening layer.

[0167] In the joint direction confirmation layer, the wake-up system can perform delay compensation based on the audio information filtered by the initial wake-up screening layer, and can perform historical multi-channel audio backtracking based on historical audio information in the circular buffer. After completing the above steps, the wake-up system can generate candidate angles and construct candidate localization regions based on these candidate angles. After completing the construction of candidate localization regions, the wake-up system can perform local direction refinement and audio enhancement within the candidate localization regions to facilitate secondary wake-up confirmation based on the enhanced audio information. After successful secondary wake-up confirmation, the wake-up system can perform joint scoring decision-making to determine the target localization region.

[0168] Within the session closed-loop control layer, the wake-up system can implement local feedback protection and automatically establish session connections. Once a session begins, the intelligent interactive device can engage in conversation with the user. During this process, the wake-up system can maintain the session and proactively terminate it if session conditions are no longer met. After terminating the session, the wake-up system can close the connection and return to standby mode, awaiting the next wake-up attempt.

[0169] Figure 4 This is a schematic diagram of an optional wake-up detection logic according to an embodiment of this application, such as... Figure 4 As shown, the intelligent interactive device is initially in standby mode. At this time, the wake-up system can detect the wake-up trigger signal based on the first ambient sound and the second ambient sound.

[0170] If the wake-up system does not detect a suspected wake-up, the smart interactive device can remain in standby mode. If the wake-up system detects a suspected wake-up, it can determine whether the energy level of the historical time window meets the standard based on the first and second ambient sounds.

[0171] If the energy level within the historical time window is insufficient, the smart interactive device can remain in standby mode. If the energy level within the historical time window is sufficient, the wake-up system can further determine whether the current moment is within the cooldown period. If the current moment is within the cooldown period, the smart interactive device can remain in standby mode. If the current moment is not within the cooldown period, the wake-up system can perform latency compensation and review historical audio. Then, the wake-up system can construct candidate positioning regions and perform local fine-tuning and audio enhancement for each candidate positioning region to facilitate secondary wake-up confirmation.

[0172] The wake-up system can determine whether the secondary wake-up confirmation of multiple candidate positioning areas has passed. If the secondary wake-up confirmation of the current candidate positioning area has failed, the wake-up system can continue to traverse the candidate positioning areas until the traversal is completed or the secondary wake-up confirmation of any candidate positioning area has passed.

[0173] If the secondary wake-up confirmation of the current candidate positioning area is successful, the wake-up system can perform joint scoring and output the target direction. Subsequently, the wake-up system can perform local feedback protection and automatic session continuation. During this process, the wake-up system can detect whether the user continues to interact with the smart interactive device.

[0174] If continued interaction is detected, the wake-up system can determine whether the audio energy supports maintaining the session. If the audio energy supports maintaining the session, the wake-up system can automatically resume the session. If no continued interaction is detected, or the audio energy does not support maintaining the session, the wake-up system can determine whether the current session has met the exit conditions.

[0175] If the current session does not meet the exit conditions, the wake-up system can automatically resume the session. If the current session meets the exit conditions, the wake-up system can actively end the session and reconnect to the terminal, then control the smart interactive device to return to standby.

[0176] Figure 5 This is a schematic diagram illustrating an optional target location area determination process according to an embodiment of this application, such as... Figure 5As shown, the wake-up system can determine multiple angles from the original set of candidate angles (only angles 1, 2, and N are shown in the figure). Subsequently, the wake-up system can construct candidate localization regions based on these angles. Specifically, the wake-up system can aggregate spatial proximity relationships among these angles to obtain multiple initial regions. Then, the wake-up system can perform activity filtering on these initial regions to obtain multiple candidate localization regions. Based on this, the wake-up system can perform joint scoring on these multiple candidate localization regions. For each candidate localization region, the wake-up system can first determine the center direction of the candidate localization region and, based on the search range and search step size, perform local fine-tuning and audio enhancement on the center direction. Subsequently, the wake-up system can perform secondary wake-up confirmation based on the results of local fine-tuning and audio enhancement. If the secondary wake-up confirmation of the candidate localization region passes, the wake-up system can perform joint scoring on the candidate localization region. Finally, the wake-up system can compare the scores of all candidate localization regions and select the candidate localization region with the higher score as the target localization region.

[0177] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0178] According to an embodiment of this application, a wake-up device for a smart interactive device is provided. It should be noted that this device can be used to execute the wake-up method for the aforementioned smart interactive device. The specific implementation process and application scenarios are the same as those in the above embodiment, and will not be repeated here. Figure 6 This is a schematic diagram of a wake-up device for a smart interactive device according to an embodiment of this application, such as... Figure 6 As shown, the device includes:

[0179] The acquisition module 602 is used to respond to the detection of a wake-up trigger signal of the smart interactive device and acquire a first ambient sound and a second ambient sound of the environment in which the smart interactive device is located. The smart interactive device is in a standby state. The first ambient sound is used to characterize the ambient sound before the wake-up trigger signal is detected, and the second ambient sound is used to characterize the ambient sound after the wake-up trigger signal is detected.

[0180] The determination module 604 is used to determine the acoustic energy of the wake-up trigger signal and multiple candidate positioning regions based on the first ambient sound and the second ambient sound.

[0181] The verification module 606 is used to determine whether the wake-up trigger signal is valid based on acoustic energy and multiple candidate positioning regions.

[0182] The control module 608 is used to control the intelligent interactive device to enter the wake-up state from the standby state when the wake-up trigger signal is valid, wherein the wake-up state is used for voice interaction.

[0183] Furthermore, the determination module is also used to: determine the environment type based on the first ambient sound; determine the acoustic energy based on the environment type, the first ambient sound, and the second ambient sound; and determine multiple candidate positioning areas based on the first ambient sound and the second ambient sound.

[0184] Furthermore, the determination module is also used to: extract features from the first ambient sound to obtain the acoustic features of the first ambient sound; and determine the environment type based on the acoustic features.

[0185] Furthermore, the determination module is also used to: determine the signal-to-noise ratio and attenuation of the first ambient sound based on acoustic characteristics; determine the sound source distance and reverberation intensity of the first ambient sound based on the signal-to-noise ratio and attenuation; and determine the environment type based on the sound source distance and reverberation intensity.

[0186] Furthermore, the determining module is also used to: determine a first interception window based on the environment type; intercept the first ambient sound and the second ambient sound based on the first interception window to obtain a first reference audio; and determine the acoustic energy based on the average energy of the first reference audio.

[0187] Furthermore, the determining module is also used to: determine a second interception window based on the time when the wake-up trigger signal is detected; intercept the first ambient sound and the second ambient sound based on the second interception window to obtain a second reference audio; and determine multiple candidate positioning regions based on the second reference audio.

[0188] Furthermore, the determination module is also used to: input the second reference audio into the direction analysis model, process the second reference audio using the direction analysis model to obtain the positioning probability distribution of multiple directions; select multiple candidate positioning directions from the multiple directions based on the positioning probability distribution; and determine multiple candidate positioning areas based on the multiple candidate positioning directions.

[0189] Furthermore, the verification module is also used to: determine whether the acoustic energy is greater than the preset energy; if the acoustic energy is greater than the preset energy, perform time delay compensation on the initial audio signals of multiple candidate positioning areas to obtain multiple enhanced audio signals corresponding to multiple candidate positioning areas, and determine whether the wake-up trigger signal is valid based on the multiple enhanced audio signals.

[0190] Furthermore, the verification module is also used to: determine the search range and search step size of any candidate positioning area; determine multiple search directions within any candidate positioning area based on the search range and search step size; and perform time delay compensation on the initial audio signals of multiple candidate positioning areas based on the multiple search directions to obtain multiple enhanced audio signals.

[0191] Furthermore, the verification module is also used to: obtain the geometric position of the microphone array of the intelligent interactive device, as well as the sound source direction angle, sound velocity, and sampling rate of the initial audio signal; determine the time delay of the initial audio signal in multiple search directions based on the geometric position, sound source direction angle, sound velocity, and sampling rate; compensate the initial audio signal in multiple search directions based on the time delay to obtain multiple compensated audio signals; and fuse the multiple compensated audio signals to obtain the enhanced audio signal corresponding to any candidate positioning region.

[0192] Furthermore, the verification module is also used to: determine multiple secondary wake-up confidence levels corresponding to multiple candidate positioning regions based on multiple enhanced audio signals; and determine that the wake-up trigger signal is valid if any secondary wake-up confidence level is greater than the preset confidence level.

[0193] Furthermore, before controlling the intelligent interactive device to enter the wake-up state from the standby state, the device also includes: a positioning module, used to determine the audio parameters corresponding to multiple candidate positioning areas based on the second ambient sound; and to determine the target positioning area from the multiple candidate positioning areas based on multiple enhanced audio signals, multiple secondary wake-up confidence levels and audio parameters.

[0194] Furthermore, the audio parameters corresponding to each candidate positioning region include the audio activity and audio temporal continuity of that candidate positioning region; the positioning module is also used to: determine the weights of the audio activity, second wake-up confidence, audio temporal continuity, and enhanced audio signal for any candidate positioning region; based on the weights, perform weighted fusion of the audio activity, second wake-up confidence, audio temporal continuity, and enhanced audio signal to obtain the confidence index value of any candidate positioning region; based on the multiple confidence index values ​​corresponding to multiple candidate positioning regions, determine the target positioning region from the multiple candidate positioning regions, wherein the target confidence index value corresponding to the target positioning region is greater than the confidence index value corresponding to the candidate positioning regions other than the target positioning region among the multiple candidate positioning regions.

[0195] Furthermore, after controlling the intelligent interactive device to enter the wake-up state from the standby state, the device also includes: a data acquisition module, used to acquire the interaction parameters of the intelligent interactive device during the voice interaction process; determine whether the interaction parameters meet the preset conditions; and control the intelligent interactive device to enter the standby state from the wake-up state if the interaction parameters do not meet the preset conditions.

[0196] Furthermore, the interaction parameters include the voice energy of the interactive voice, the effective voice duration, and the prompt tone protection period; the acquisition module is also used to: determine that the interaction parameters meet the preset conditions when the voice energy is greater than the preset energy, the effective voice duration is greater than the preset time, and the time of the interactive voice is not within the prompt tone protection period.

[0197] Furthermore, the control module is also used to: construct feedback prompts based on the target positioning area, wherein the feedback prompts are used to prompt the smart interactive device to enter the wake-up state; and output feedback prompts based on the various output devices on the smart interactive device.

[0198] Embodiments of this application also provide an electronic device, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of this application when it runs.

[0199] Embodiments of this application also provide a computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of this application.

[0200] Embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the methods of various embodiments of this application.

[0201] Embodiments of this application also provide a computer program product, including a non-volatile computer-readable storage medium for storing a computer program that, when executed by a processor, implements the methods in various embodiments of this application.

[0202] Embodiments of this application also provide a computer program that, when executed by a processor, implements the methods described in the various embodiments of this application.

[0203] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0204] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0205] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0206] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0207] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0208] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for waking up an intelligent interactive device, characterized in that, include: In response to the detection of a wake-up trigger signal from a smart interactive device, the system acquires a first ambient sound and a second ambient sound of the environment in which the smart interactive device is located, wherein the smart interactive device is in a standby state, the first ambient sound is used to characterize the ambient sound before the wake-up trigger signal is detected, and the second ambient sound is used to characterize the ambient sound after the wake-up trigger signal is detected. Based on the first ambient sound and the second ambient sound, the acoustic energy of the wake-up trigger signal and multiple candidate positioning regions are determined; Based on the acoustic energy and the multiple candidate localization regions, determine whether the wake-up trigger signal is valid; When the wake-up trigger signal is valid, the intelligent interactive device is controlled to enter the wake-up state from the standby state, wherein the wake-up state is used for voice interaction.

2. The method according to claim 1, characterized in that, Based on the first ambient sound and the second ambient sound, the acoustic energy of the wake-up trigger signal and multiple candidate localization regions are determined, including: Based on the first ambient sound, determine the environment type of the environment; The acoustic energy is determined based on the environment type, the first ambient sound, and the second ambient sound. Based on the first ambient sound and the second ambient sound, the plurality of candidate positioning areas are determined.

3. The method according to claim 2, characterized in that, Based on the first ambient sound, the environment type of the environment is determined, including: Feature extraction is performed on the first ambient sound to obtain the acoustic features of the first ambient sound; Based on the acoustic characteristics, the environment type is determined.

4. The method according to claim 3, characterized in that, Determining the environment type based on the acoustic characteristics includes: Based on the acoustic characteristics, the signal-to-noise ratio and attenuation of the first ambient sound are determined; Based on the signal-to-noise ratio and the attenuation, the sound source distance and reverberation intensity of the first ambient sound are determined; The environment type is determined based on the sound source distance and the reverberation intensity.

5. The method according to claim 2, characterized in that, Determining the acoustic energy based on the environment type, the first ambient sound, and the second ambient sound includes: Based on the environment type, determine the first capture window; Based on the first capture window, the first ambient sound and the second ambient sound are captured to obtain the first reference audio; The acoustic energy is determined based on the average energy of the first reference audio.

6. The method according to claim 2, characterized in that, Based on the first ambient sound and the second ambient sound, the plurality of candidate location regions are determined, including: The second interception window is determined based on the time when the wake-up trigger signal is detected; Based on the second capture window, the first ambient sound and the second ambient sound are captured to obtain the second reference audio; Based on the second reference audio, the plurality of candidate positioning regions are determined.

7. The method according to claim 6, characterized in that, Based on the second reference audio, the plurality of candidate localization regions are determined, including: The second reference audio is input into the direction analysis model, and the direction analysis model is used to process the second reference audio to obtain the positioning probability distribution of multiple directions; Based on the positioning probability distribution, multiple candidate positioning directions are selected from the multiple directions; Based on the multiple candidate positioning directions, the multiple candidate positioning regions are determined.

8. The method according to claim 1, characterized in that, Based on the acoustic energy and the multiple candidate localization regions, determining whether the wake-up trigger signal is valid includes: Determine whether the acoustic energy is greater than a preset energy; When the acoustic energy is greater than a preset energy, the initial audio signals of the multiple candidate positioning regions are time-delayed to obtain multiple enhanced audio signals corresponding to the multiple candidate positioning regions, and based on the multiple enhanced audio signals, it is determined whether the wake-up trigger signal is valid.

9. The method according to claim 8, characterized in that, Time delay compensation is applied to the initial audio signals of the multiple candidate positioning regions to obtain multiple enhanced audio signals corresponding to the multiple candidate positioning regions, including: Determine the search range and search step size for any candidate localization region; Based on the search range and the search step size, multiple search directions are determined within any candidate positioning area; Based on the multiple search directions, the initial audio signals of the multiple candidate positioning regions are time-delay compensated to obtain the multiple enhanced audio signals.

10. The method according to claim 9, characterized in that, Based on the multiple search directions, time delay compensation is performed on the initial audio signals of the multiple candidate localization regions to obtain the multiple enhanced audio signals, including: Obtain the geometric position of the microphone array of the intelligent interactive device, as well as the sound source direction angle, sound velocity, and sampling rate of the initial audio signal; Based on the geometric location, the sound source direction angle, the sound speed, and the sampling rate, the time delay of the initial audio signal in the multiple search directions is determined; Based on the time delay, the initial audio signal is compensated in the multiple search directions to obtain multiple compensated audio signals; The multiple compensated audio signals are fused to obtain the enhanced audio signal corresponding to any one of the candidate positioning regions.

11. The method according to claim 8, characterized in that, Based on the multiple enhanced audio signals, determining whether the wake-up trigger signal is valid includes: Based on the multiple enhanced audio signals, multiple secondary wake-up confidence levels corresponding to the multiple candidate positioning regions are determined; If the confidence level of any secondary wake-up is greater than the preset confidence level, the wake-up trigger signal is determined to be valid.

12. The method according to any one of claims 1 to 11, characterized in that, Before controlling the intelligent interactive device to enter the wake-up state from the standby state, the method further includes: Based on the second ambient sound, determine the audio parameters corresponding to the plurality of candidate positioning regions; Based on multiple enhanced audio signals, multiple secondary wake-up confidence levels, and the audio parameters, a target positioning area is determined from the multiple candidate positioning areas.

13. The method according to claim 12, characterized in that, The audio parameters corresponding to each candidate localization region include the audio activity and audio temporal continuity of that candidate localization region; Based on multiple enhanced audio signals, multiple secondary wake-up confidence levels, and the audio parameters, a target positioning region is determined from the multiple candidate positioning regions, including: Determine the audio activity, second wake-up confidence, audio temporal continuity, and weight of the enhanced audio signal for any candidate localization region; Based on the weights, the audio activity, the second wake-up confidence, the audio temporal continuity, and the enhanced audio signal are weighted and fused to obtain the confidence index value of any candidate localization region. Based on multiple confidence index values ​​corresponding to the multiple candidate positioning regions, the target positioning region is determined from the multiple candidate positioning regions, wherein the target confidence index value corresponding to the target positioning region is greater than the confidence index value corresponding to the candidate positioning regions other than the target positioning region among the multiple candidate positioning regions.

14. The method according to any one of claims 1 to 11, characterized in that, After controlling the intelligent interactive device to enter the wake-up state from the standby state, the method further includes: Collect the interaction parameters of the intelligent interactive device during the voice interaction process; Determine whether the interaction parameters meet the preset conditions; If the interaction parameters do not meet the preset conditions, the intelligent interactive device is controlled to enter the standby state from the wake-up state.

15. The method according to claim 14, characterized in that, The interaction parameters include the voice energy of the interactive voice, the effective voice duration, and the prompt tone protection period; determining whether the interaction parameters meet preset conditions includes: If the voice energy is greater than the preset energy, the effective voice duration is greater than the preset time, and the time of the interactive voice transmission is not within the prompt tone protection period, then the interaction parameters are determined to meet the preset conditions.

16. The method according to claim 1, characterized in that, Controlling the intelligent interactive device to enter the wake-up state from the standby state includes: Based on the target positioning area, a feedback prompt message is constructed, wherein the feedback prompt message is used to prompt the intelligent interactive device to enter the wake-up state; The feedback prompt information is output based on the various output devices on the intelligent interactive device.

17. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the method according to any one of claims 1 to 16.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the storage medium is located to perform the method according to any one of claims 1 to 16.

Images