Voice input arousing method, voice interaction device and computer storage medium

Abstract

The application discloses a voice input arousing method, a voice interaction device and a computer storage medium, and relates to the technical field of voice interaction. The method comprises the following steps: acquiring an interface operation event sequence and context features; predicting an intention probability of a user to be performed voice input based on the interface operation event sequence and the context features; and performing a first level or a second level voice input arousing operation according to the intention probability and a preset hierarchical gating. The first level voice input arousing operation comprises at least one of the following operations: executing a resident voice recognition model, loading a voice recognition model, and initializing a voice recognition pipeline. The second level voice input arousing operation comprises the following operations: making the voice recognition pipeline in a ready state and starting a ring buffer, retaining the collected audio through the ring buffer, and performing voice recognition based on the cached audio in the ring buffer through the voice recognition pipeline. The application improves the completeness and recognition accuracy of voice input.

Description

Technical Field

[0001] This application relates to the field of voice interaction technology, and in particular to a voice input activation method, a voice interaction device, and a computer storage medium. Background Technology

[0002] Voice interaction has become an important method of human-computer interaction, widely used in mobile phone input methods, in-vehicle interaction, meeting recording, and wearable devices. With the continuous improvement of voice recognition accuracy, users have placed higher demands on the convenience, response speed, and naturalness of voice input. Especially in high-frequency usage scenarios such as instant messaging, navigation search, and voice memos, the efficiency of voice input activation directly affects the overall user experience.

[0003] Currently, the mainstream voice input activation methods mainly include the following three categories: First, wake-up word activation, where the user needs to say a preset fixed wake-up word, and the system starts the voice recognition process after recognizing the wake-up word; second, press and hold to speak activation, where the user needs to manually press or click a specific button, speak while holding it down, and end the input and trigger recognition after releasing the button; and third, always-on voice activity detection, where the system continuously keeps the microphone on and monitors the audio signal in real time, and automatically starts recognition when voice activity is detected.

[0004] In the existing voice input activation methods mentioned above, the voice recognition pipeline only starts after the user speaks. The system can only trigger the loading and initialization of the recognition pipeline after receiving the user's explicit voice activation action or detecting voice activity. This loading and initialization process inevitably consumes a certain amount of startup time, causing the starting syllable of the user's voice to fall within the system startup window and not be effectively received by the recognition pipeline, thereby reducing the integrity and recognition accuracy of voice input. Summary of the Invention

[0005] The main objective of this application is to provide a voice input activation method, a voice interaction device, and a computer storage medium, aiming to solve the technical problem of how to improve the integrity and recognition accuracy of voice input.

[0006] To achieve the above objectives, this application provides a voice input activation method, which includes the following steps: Obtain the sequence of interface operation events and context features, wherein the context features include at least one of input control type, application scenario and operation timing rhythm; Based on the interface operation event sequence and the context features, predict the probability of the user's intention to perform voice input; Based on the intent probability and the preset hierarchical gating, execute the first level or the second level of voice input activation operation; The first-level voice input activation operation includes at least one operation among executing a resident speech recognition model or loading a speech recognition model and initializing a speech recognition pipeline, and determines whether to upgrade from the first level to the second level or exit the voice input activation operation based on the change in the intent probability. The second-level voice input activation operation includes putting the voice recognition pipeline into a ready state and activating the circular buffer to retain the acquired audio through the circular buffer, and performing voice recognition based on the audio cached in the circular buffer through the voice recognition pipeline.

[0007] In addition, to achieve the above objectives, this application also provides a voice interaction device, including: a microphone, a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the steps of the voice input activation method as described above.

[0008] In addition, to achieve the above objectives, this application also provides a readable storage medium, which is a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement the steps of the voice input activation method as described above.

[0009] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the voice input activation method described above.

[0010] One or more technical solutions proposed in this application have at least the following technical effects: This application embodiment obtains the sequence of interface operation events and contextual features to predict the user's intention to input voice before the user emits a voice signal. Based on this intention probability, a hierarchical gating strategy is used to trigger the corresponding wake-up operation. When the intention probability is high, a second-level voice input wake-up operation is executed, putting the voice recognition pipeline into a fully ready state in advance and simultaneously opening the circular buffer. This means that the entire recognition chain is ready before the user speaks, and the circular buffer can completely retain all audio data from the first frame of the voice's starting syllable. When the intention probability is low, at least one of the first-level preparation operations (such as dwelling or loading the model, initializing the pipeline) can be executed first to complete some resource preparation work in advance. After the intention is clarified, the process is upgraded to the second level. At this time, the prepared basic resources shorten the residual delay required to enter full readiness. Thus, whether directly entering the second level or transitioning through the first level, this solution ensures that the speech recognition pipeline has real-time processing capabilities at the moment the user actually speaks. The circular buffer mechanism can retain the complete audio signal, including the starting syllable, thereby effectively avoiding the syllable loss caused by the cold start process occupying the pronunciation time window in traditional solutions. This allows the speech recognition model to decode based on the full and continuous audio data, rather than guessing based on incomplete signals, thereby improving the integrity of speech input and recognition accuracy. Attached Figure Description

[0011] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0012] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart illustrating the first embodiment of the voice input activation method of this application; Figure 2 This is a schematic diagram of the intent prediction process involved in one embodiment of the voice input invocation method of this application; Figure 3 A schematic diagram comparing the traditional voice input activation method with the voice input activation method of this application; Figure 4 This is a schematic diagram of a graded false wake-up recovery process according to an embodiment of the voice input wake-up method of this application; Figure 5 This is a schematic diagram of the online self-learning model update process involved in one embodiment of the voice input activation method of this application.

[0014] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0015] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] With the improvement of speech recognition accuracy, voice input has been widely used in mobile phone input methods, in-vehicle interaction, meeting recording, and wearable devices. The starting point of voice input is "awakening"—that is, the process from the user's intention to speak to the speech recognition pipeline being ready and starting to receive speech. The current mainstream voice input awakening methods can be roughly divided into three categories: the first is wake word awakening, where the user needs to say a fixed wake word, and the system starts the recognition pipeline accordingly; the second is explicit manual awakening by pressing and holding the button (press and hold the button → speak → release → recognize); and the third is always-on voice activity detection, where the device continuously listens and starts recognition after detecting voice activity. The common feature of these methods is "reactive"—the system must first perform an explicit voice awakening action (such as saying a wake word or pressing the speak button) or produce a sound before it starts occupying the microphone, loading or resident the recognition model, and initializing the recognition pipeline.

[0017] The above-mentioned reactive wake-up usually has the following limitations in actual use: (1) There is a cold start wake-up delay. After the user makes an explicit voice wake-up action (saying the wake word, pressing the button), the recognition pipeline needs a startup time to occupy the microphone, load the model and initialize (forming a startup delay that can be perceived by the user). The user has to pause and wait for the pipeline to be ready, which destroys the natural continuity of "speaking as you want"; (2) The first syllable is easily lost. Since the pipeline is often not ready at the initial moment when the user speaks, the first one or two syllables of the user's mouth are often swallowed (first syllable swallowing), resulting in (3) The cost of always-on listening is high. Although always-on voice activity detection can shorten some of the waiting time, it requires continuous use of microphone and computing power for indiscriminate listening, which brings power consumption burden. Indiscriminate recording raises privacy concerns. Moreover, it still needs to start the complete recognition pipeline after detecting voice activity, making it difficult to completely eliminate cold start delay. (4) The wake-up method itself interrupts natural interaction. The wake-up word needs to be memorized and interrupts spoken language. Pressing and holding to speak requires hand operation. Both of these contradict the original intention of "voice input without hands".

[0018] More importantly, users often leave predictable operational traces on their devices before initiating voice input—such as clicking and focusing on the chat input box, switching to a certain type of application, or performing continuous operations in a specific rhythm. These operational behaviors have a statistically modelable correlation with the imminent initiation of voice input. However, existing publicly available solutions typically only passively initiate preparation after the user makes an explicit voice initiation action or utters a sound, generally failing to utilize the device operation sequence prior to the user's voice input to predict speaking intent and pre-warm-up and audio scrolling buffering. Therefore, there is an urgent need for an intelligent voice input initiation mechanism that can predict speaking intent before the user speaks, securely pre-warm-up and pre-initiation based on hierarchical gating, and preserve the first word with a scrolling buffer.

[0019] To address the aforementioned issues, the main solution of this application is as follows: Obtaining a sequence of interface operation events and contextual features, wherein the contextual features include at least one of input control type, application scenario, and operation timing rhythm; predicting the probability of a user's intent to perform voice input based on the sequence of interface operation events and the contextual features; and executing a first-level or second-level voice input activation operation based on the intent probability and a preset hierarchical gating; wherein the first-level voice input activation operation includes at least one of executing a resident speech recognition model or loading a speech recognition model, or initializing a speech recognition pipeline, and determining whether to upgrade from the first level to the second level or exit the voice input activation operation based on the change in the intent probability; the second-level voice input activation operation includes putting the speech recognition pipeline into a ready state and activating a circular buffer to retain the acquired audio through the circular buffer, and performing speech recognition based on the audio cached in the circular buffer through the speech recognition pipeline.

[0020] This application uses the sequence of interface operation events and contextual features to predict the user's intention to input voice before the user even emits a voice signal. Based on this intention probability, a hierarchical gating strategy is used to trigger the corresponding wake-up operation. When the intention probability is high, a second-level voice input wake-up operation is executed, putting the voice recognition pipeline into a fully ready state in advance and simultaneously activating the circular buffer. This means that the entire recognition chain is ready before the user speaks, and the circular buffer can completely retain all audio data from the first frame of the initial syllable of the voice. When the intention probability is low, at least one of the first-level preparation operations (such as dwelling or loading the model, initializing the pipeline) can be executed first to complete some resource preparation work in advance. After the intention is clarified, the process is upgraded to the second level. At this time, the prepared basic resources shorten the residual delay required to enter full readiness. Thus, whether directly entering the second level or transitioning through the first level, this solution ensures that the speech recognition pipeline has real-time processing capabilities at the moment the user actually speaks. The circular buffer mechanism can retain the complete audio signal, including the starting syllable, thereby effectively avoiding the syllable loss caused by the cold start process occupying the pronunciation time window in traditional solutions. This allows the speech recognition model to decode based on the full and continuous audio data, rather than guessing based on incomplete signals, thereby improving the integrity of speech input and recognition accuracy.

[0021] It should be noted that the execution subject of the various embodiments of the device control method of this application can be a computing service device with data processing and program running functions, such as a personal computer, a server, etc., or a voice interaction device capable of realizing the above functions, such as an AR helmet, a VR helmet, a smart speaker, a vehicle terminal, etc. The various embodiments of the device control method of this application do not impose specific limitations on this.

[0022] Based on this, this application proposes a voice input activation method according to a first embodiment. In this embodiment, refer to... Figure 1 As shown, the voice input activation method includes the following steps S10~S30: Step S10: Obtain the interface operation event sequence and context features, wherein the context features include at least one of the input control type, application scenario and operation timing rhythm; User interface (UI) event sequences can be obtained by listening to various user interaction events distributed by the operating system or application framework layer, such as click events, swipe events, long press events, and focus switching events. These events are arranged in chronological order to form an event sequence, reflecting the user's current interaction trajectory with the UI.

[0023] Contextual features are auxiliary discriminative information extracted from the current application environment and user interaction patterns. Input control types can be determined by the class name, resource identifier, or control attributes of the currently focused view; for example, different types of controls such as search boxes, input boxes, voice input buttons, and text editing areas correspond to different input requirements. Application scenarios can be identified by the package name, Activity name, or page routing information of the currently running foreground application; for example, instant messaging interfaces, navigation search interfaces, and memo editing interfaces. The rhythm of operation sequence can be characterized by statistically analyzing the time intervals, frequency, and regularity of continuous user operations within a certain time window, such as the mean and variance of the time interval sequence between adjacent operations, the frequency of operations per unit time, the time interval between the current operation and the previous operation, and the repetition rate of the same operation type in the most recent operations.

[0024] It should be further explained that contextual features represent a snapshot of the current interactive interface state, describing "what kind of interface environment the user is currently in"; while the sequence of interface operation events constitutes a timeline of the user's behavior over a recent period, describing "what the user has recently done". The two work together to form complete information about the user's current interactive state, providing a relatively comprehensive basis for predicting "what the user is doing now and whether they might want to speak next".

[0025] Step S20: Based on the interface operation event sequence and the context features, predict the probability of the user's intention to perform voice input; Reference Figure 2 As shown, after obtaining the sequence of interface operation events and contextual features, these are input into a pre-built intent prediction model to calculate the probability value, i.e., the intent probability P, of the user's upcoming voice input. This intent prediction model can employ a statistical learning-based classifier or a deep neural network-based classification model. The model's input data can include contextual features such as the sequence of interface operation events, input control type, application scenario, and operation timing.

[0026] Optionally, the model can employ machine learning algorithms such as logistic regression, support vector machines, and gradient boosting decision trees, or deep learning architectures capable of capturing temporal dependencies, such as recurrent neural networks, long short-term memory networks, and Transformers, and be obtained through supervised learning using pre-labeled training samples. The training samples can be derived from the annotation of a large number of real user interaction logs, with the annotation label indicating whether the user actually initiated voice input under specific operation sequences and contextual conditions.

[0027] The intent prediction model outputs a probability value between 0 and 1 based on the feature vector of the current input. This probability value quantifies the likelihood that the user will make a voice input.

[0028] Step S30: Based on the intent probability and the preset hierarchical gating, execute the first level or the second level of voice input activation operation; One or more probability thresholds for hierarchical decision-making are preset, referred to as hierarchical gating. Hierarchical gating includes at least a first threshold and a second threshold, where the second threshold is higher than the first threshold. When the intent probability is lower than the first threshold, it is determined that the user currently has no clear voice input intent, and no activation operation is performed; when the intent probability reaches or exceeds the first threshold but is lower than the second threshold, a first-level voice input activation operation is performed; when the intent probability reaches or exceeds the second threshold, a second-level voice input activation operation is performed.

[0029] The specific values ​​of hierarchical gating can be flexibly configured according to the resource constraints and response requirements of the actual application scenario. For example, in high real-time demand scenarios such as automotive, the threshold value can be appropriately reduced to improve response sensitivity, while in wearable devices that are more sensitive to power consumption, the threshold value can be appropriately increased to reduce unnecessary resource consumption.

[0030] The first-level voice input activation operation includes at least one operation among executing a resident speech recognition model or loading a speech recognition model and initializing a speech recognition pipeline, and determines whether to upgrade from the first level to the second level or exit the voice input activation operation based on the change in the intent probability. In the first level, based on the confidence level of the current intent probability and the current resource usage of the device, at least one preparation operation is selected from the following: residing in the speech recognition model, loading the speech recognition model, or initializing the speech recognition pipeline. It should be noted that residing in the speech recognition model refers to the speech recognition model being permanently resident in memory and in a low-power standby state. This state is usually caused by the model not being unloaded during startup or after the last use. Although it occupies some memory resources, it does not perform computational tasks, thus waking up quickly and requiring no additional loading time. Loading the speech recognition model, on the other hand, refers to the process of reading the model weights from storage media or disk into memory. This process involves input / output operations and is suitable for scenarios with limited memory resources or where on-demand loading is required.

[0031] In addition, initializing the speech recognition pipeline refers to creating, configuring, and connecting the software objects of each software module (such as feature extraction module, acoustic model, language model, decoder, etc.) in the recognition pipeline, so that each module can be loaded.

[0032] After completing at least one of the above preparation operations, continue to monitor changes in the interface operation event sequence and context features, and recalculate the updated intent probability. If the updated intent probability increases and reaches or exceeds the second threshold, an upgrade from the first level to the second level is triggered; if the updated intent probability decreases and falls below the first threshold, exit the voice input wake-up operation and release some occupied resources to avoid unnecessary power consumption.

[0033] The second-level voice input activation operation includes putting the voice recognition pipeline into a ready state and activating the circular buffer to retain the acquired audio through the circular buffer, and performing voice recognition based on the audio cached in the circular buffer through the voice recognition pipeline.

[0034] In the second level, ensure that the speech recognition pipeline has entered the ready state. It should be noted that there is a certain hierarchical relationship between the ready state and the aforementioned initialization state: the initialization state completes the creation of software objects and module configuration, at which point the audio stream has not yet been opened and the decoder has not been warmed up, and it has not yet reached the stage where it can immediately process audio signals; while the ready state is a further advancement based on the completion of initialization, at which point the decoder has completed warm-up (e.g., run idle inference), the audio input channel has been opened, and it is in a standby state for execution of recognition.

[0035] Simultaneously, a circular buffer mechanism is enabled. The circular buffer is a pre-allocated area in memory for continuously written audio data. Audio signals continuously collected by the microphone are written to this buffer area frame by frame. The circular buffer uses a first-in, first-out (FIFO) storage method; when the buffer is full, the latest audio frame overwrites the earliest audio frame, thus preserving audio data within a recent time window. The length of this time window can be pre-configured.

[0036] It is worth emphasizing that during the first-level warm-up operation, model residency or loading and pipeline initialization are performed, but microphone resources are not occupied and recording does not begin, so there is no audio signal available for recognition. After upgrading to the second level, the microphone is officially occupied and the circular buffer is opened, and actual speech recognition is performed based on the audio data stored in the buffer. Since the speech recognition pipeline is already in a ready state before the user actually speaks, when there are audio frames containing the user's starting syllable in the circular buffer, the pipeline can read these audio frames from the circular buffer and perform acoustic feature extraction and decoding processing, without waiting for the user to finish speaking or manually triggering the recognition command.

[0037] The aforementioned operation sequence acquisition, intent prediction, and unconfirmed audio buffering during pre-evoked responses can all be completed locally on the device. Speech recognition after user confirmation can be performed locally or sent to the cloud for processing, allowing for flexible selection based on network conditions and device performance. Simultaneously, the intent prediction model can be updated online based on actual user behavior feedback after the prediction (including whether the user actually spoke within the expected time window and whether the pre-evoked result was accepted), gradually adapting it to individual operating habits and usage patterns. To avoid model degradation due to mislearning, the online updates are limited to adjustments to lightweight classification heads or preference parameters; the pre-trained base model remains unchanged, and abnormal feedback samples are downweighted or not updated. A rollback mechanism for model parameters is also supported.

[0038] As an optional, loosely bound collaboration method, the speech recognition pipeline can be connected to a progressive sentence segmentation construction module once it is ready, so as to present a preview of the recognition results to the user sentence by sentence during the recognition process. In addition, at different stages of voice input activation, including the warm-up stage, pre-activation stage, ready stage, and recognition stage, the context-aware feedback layer can be invoked to output interactive feedback adapted to the environmental state. For example, it can provide concise voice prompts in an in-vehicle environment, or provide vibration and visual signals in wearable devices, thereby enhancing the user's perception of the current recognition status and the progress of system resource preparation.

[0039] Through the coordinated efforts of the above steps, the speech recognition pipeline is ensured to have real-time processing capabilities at the moment the user actually speaks. The circular buffer mechanism can retain audio signals including the starting syllable, thereby effectively avoiding the loss of syllables that may occur due to the cold start process occupying the pronunciation time window in traditional solutions. This allows the speech recognition model to decode based on continuous audio data, thereby improving the integrity and recognition accuracy of speech input.

[0040] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description and will not be repeated hereafter. Based on this, after the step of performing the first-level or second-level voice input activation operation according to the intent probability and the preset hierarchical gating, the method further includes: Step A10: If user input audio is detected after the second-level voice input activation operation is completed, then endpoint detection is performed on the input audio to obtain the endpoint detection result. After the second-level voice input activation operation is completed, the microphone is turned on and the circular buffer continues to retain audio data.

[0041] When the actual input audio signal from the user is detected, endpoint detection processing is performed on the acquired audio stream. Endpoint detection is used to determine the start and end boundaries of the user's speech in a continuous audio signal. It can be performed using a dual-threshold detection method based on short-time energy and zero-crossing rate, or a speech activity detection model based on a deep neural network.

[0042] The endpoint detection result includes at least one preliminary determination of the speech start time, which serves as a reference anchor point for subsequent backtracking processing.

[0043] Step A20: Determine the candidate audio start time based on the endpoint detection results, and define a backtracking window forward from the candidate audio start time in the circular buffer. Based on the endpoint detection results, an audio start candidate moment is determined. This candidate moment represents the speech start boundary position initially determined by the endpoint detection algorithm.

[0044] Considering that the endpoint detection algorithm itself may have a certain detection delay, that is, the actual start time of the user's voice may be earlier than the candidate time reported by the endpoint detection algorithm, a backtracking window is defined in the circular buffer with the audio start candidate time as the endpoint.

[0045] The length of the backtracking window can be pre-configured according to the actual application scenario, for example, set to a value between 200 milliseconds and 500 milliseconds. This window covers a segment of audio data before the candidate moment, providing a time search range for subsequently retrieving potentially missed starting syllables from the buffer.

[0046] Step A30: Determine the backtracking starting point based on the acoustic features of the audio within the backtracking window, wherein the acoustic features include at least one of energy mutation features, acoustic confidence features, and noise level; Energy mutation features are used to detect points in an audio signal where there is a sudden increase in speech energy, transitioning from silence or background noise. In practice, short-time energy is calculated frame-by-frame within the backtracking window, and the mean and variance of the energy are calculated using a sliding window approach. When the energy value of a frame increases by more than a preset threshold relative to the energy value of the previous frame, that frame is considered an energy mutation point. This mutation point often corresponds to the moment the user begins speaking.

[0047] Acoustic confidence features are used to evaluate the confidence score of each audio frame as belonging to speech rather than non-speech. In a specific implementation, a speech / non-speech binary classification neural network model can be pre-trained. Each audio frame within the backtracking window is input into this model, and the posterior probability of each frame belonging to the speech category is obtained as the confidence score. When the confidence score jumps from a value below the first confidence threshold to a value above the second confidence threshold, the position of this jump frame can be used as a reference for the starting position.

[0048] Noise level features are used to evaluate the energy baseline of background noise within the window. In practice, noise samples can be collected within a silent interval before the candidate audio start time, and the mean and standard deviation of their energy can be calculated to establish a noise baseline estimate. When the energy value of a frame exceeds the sum of the baseline and the standard deviation by a preset multiple, the frame is determined to be the speech start frame, and the position of this frame can be used as the backtracking starting point.

[0049] Optionally, the above-mentioned features can be combined to improve the robustness of the judgment. For example, a candidate starting point can be located based on the energy mutation feature and the acoustic confidence feature respectively. When the difference between the frame positions located by the two is less than a preset deviation threshold, the weighted average position of the two is taken as the final backtracking starting point; when the difference is greater than the preset deviation threshold, the candidate position corresponding to the feature whose confidence score jump becomes significant is selected, or the feature type with higher confidence is dynamically selected as the main judgment criterion based on the signal-to-noise ratio of the current environment.

[0050] Step A40: Extract the audio segment from the backtracking starting point to the audio start candidate time from the circular buffer; After determining the backtracking starting point, the audio segments between that backtracking starting point and the previously determined audio start candidate time are extracted from the circular buffer.

[0051] The extracted audio segment represents the portion of the speech signal that actually occurred but was not captured by the endpoint detection algorithm in time before the endpoint detection algorithm reported the starting candidate moment. This segment usually contains the starting syllable or part of the syllable of the user's speech.

[0052] Step A50: The extracted audio segment is merged with the audio collected after the candidate time point to form a complete audio, and the complete audio is input into the speech recognition pipeline for speech recognition.

[0053] The audio segments extracted from the circular buffer are combined with the audio data collected after the candidate audio start time to form a complete speech audio.

[0054] The merged complete audio contains both the initial syllable segment before the candidate moment and the subsequent speech content after the candidate moment, thus restoring the entire speech signal of the user from the beginning to the end of the actual speech.

[0055] Subsequently, the merged, complete audio input is processed in the ready-to-use speech recognition pipeline. Because this complete audio retains the initial acoustic features before the speech start boundary, the speech recognition model can acquire richer pronunciation information, thus obtaining more accurate recognition results during the decoding process.

[0056] Through the coordinated efforts of the above steps, even with delays in endpoint detection, the missing initial speech segments can be retrieved via a backtracking mechanism using a circular buffer mechanism. This effectively compensates for the audio truncation problem that may result from endpoint detection lag, allowing the speech recognition model to decode based on a more complete speech signal, thereby improving the integrity and accuracy of speech input.

[0057] In one possible implementation, the step of performing a first-level or second-level voice input activation operation based on the intent probability and a preset hierarchical gating includes: Step B10: Obtain a preset hierarchical gating, wherein the hierarchical gating includes a first gating threshold and a second gating threshold, and the second gating threshold is greater than the first gating threshold; The second gating threshold is configured to be greater than the first gating threshold. Together, they divide the range of intent probability into three consecutive decision intervals: a low intent interval below the first gating threshold, a medium intent interval between the first and second gating thresholds, and a high intent interval that reaches or exceeds the second gating threshold.

[0058] Step B20: If the intent probability is greater than or equal to the first gating threshold and less than the second gating threshold, then perform the first level of voice input wake-up operation. When the intent probability output by the intent prediction model falls within the range of greater than or equal to the first gating threshold and less than the second gating threshold, it is determined that the user currently has a certain intention to input voice, but the clarity of this intention has not yet reached the level that can directly trigger the complete recognition chain, so the first level of voice input activation operation is triggered.

[0059] In the first level, at least one preparation operation is performed: residing in the speech recognition model, loading the speech recognition model, or initializing the speech recognition pipeline. This level of operation is characterized by performing only software-level resource preparation, without consuming microphone resources or initiating the audio acquisition and recognition process.

[0060] The significance of the above preparation operations is to advance the time-consuming loading and initialization steps in the speech recognition pipeline startup process to before the user actually speaks. This reduces the residual delay required when upgrading to the second level, and ensures that the starting syllable of the user's speech can be captured in time without being lost due to loading time.

[0061] Step B30: If the intent probability is greater than or equal to the second gating threshold, then perform the second level of voice input activation operation.

[0062] When the intent probability output by the intent prediction model reaches or exceeds the second gating threshold, it is determined that the user has a clear and urgent voice input intent, and at this time, the second level of voice input activation operation is triggered.

[0063] In the second level, it is ensured that the speech recognition pipeline is in a ready state and the circular buffer mechanism is activated. Specifically, if the preparation operations of the first level have been performed previously, the time required for the pipeline to switch from the completed initialization state to the fully ready state is extremely short; if the intent probability is high and the pipeline directly enters the second level, the complete pipeline ready process is executed, including model loading, decoder warm-up, and opening of audio input channels. Simultaneously, the circular buffer is activated to continuously retain audio data acquired by the microphone.

[0064] Since the second-level wake-up operation is completed before the user actually speaks, the speech recognition pipeline is already in a state where it can process audio signals in real time when the user starts speaking. This reduces the truncation problem caused by pipeline startup lag, thus providing a complete audio input foundation for subsequent speech recognition.

[0065] In one possible implementation, the method further includes: Step C10: Configure a de-shaking mechanism to suppress frequent level switching caused by fluctuations in the intent probability around the gating threshold; The anti-shake mechanism includes at least one of the following: hysteresis threshold with different thresholds for upgrading and downgrading, minimum dwell time, cooldown time, consecutive hit count, and intent probability smoothing.

[0066] In actual operation, the intent probability output by the intent prediction model may fluctuate slightly around the gating threshold due to brief pauses in user operation, instantaneous changes in interface focus, or environmental factors. Directly switching levels based on these fluctuating probability values ​​could cause frequent oscillations between the first and second levels of voice input activation. This would lead to unnecessary redundant resource requests and releases, increasing power consumption and processing overhead, and could also negatively impact user experience due to instability. Therefore, an anti-shake mechanism is configured to suppress these frequent switching phenomena.

[0067] Anti-shake mechanisms can be implemented using hysteresis thresholds. This involves setting different thresholds for upgrade and downgrade decisions. When the intent probability rises from a low range, it must reach a higher second threshold to upgrade; conversely, when it falls from a high range, it must drop to a lower first threshold to downgrade or exit. Because there is a hysteresis range between the upgrade and downgrade thresholds, fluctuations in probability within this range will not trigger any level switching. Anti-shake mechanisms can also use minimum dwell time. This involves setting a minimum duration requirement for a given level. Once in that level, the user must remain there for a preset duration before switching to another level, preventing immediate state revert due to brief probability fluctuations. Finally, anti-shake mechanisms can use a cooldown period. After a level switch, a preset cooldown window is initiated, during which switching back to the original level is prohibited, thus providing a forced stabilization period for state switching. In addition, a continuous hit count debouncing method can be used. This requires the intent probability to meet a certain level of triggering conditions and be met in multiple consecutive decision periods. Only after the cumulative number of hits reaches a preset threshold will the corresponding level switch be executed, thereby eliminating the interference of occasional single fluctuations on the decision. Intent probability smoothing is also an optional debouncing method. This involves filtering the raw probability values ​​output by the intent prediction model, for example, by using moving averages or exponentially weighted moving averages, to make the probability value change curve smoother. Abnormal jumps in individual periods are effectively filtered out. Classification based on the smoothed probability values ​​can reduce frequent switching caused by instantaneous fluctuations.

[0068] The aforementioned anti-shake mechanisms can be flexibly selected in one or more combinations according to specific application scenarios to achieve a balance between state switching stability and response sensitivity. This ensures timely voice input activation while avoiding resource waste and user experience degradation caused by frequent state oscillations.

[0069] In one possible implementation, after the step of performing a first-level or second-level voice input activation operation based on the intent probability and a preset hierarchical gating, the method further includes: Step D10: If the first level of voice input wake-up operation is executed, but the second level of voice input wake-up operation is not upgraded within the expected time window, then the prediction is determined to be unsuccessful, and the resources occupied by the voice recognition model and the voice recognition pipeline are released or downgraded. After performing the first-level voice input activation operation, an expected time window is set to evaluate the effectiveness of this intent prediction. The length of this expected time window can be pre-configured according to the response requirements and resource constraints of the application scenario, for example, it can be set to the order of several seconds. Its start time is the time point when entering the first level, and its end time is the cutoff time when the user is expected to actually speak.

[0070] If the probability of intent fails to rise to the second gating threshold or trigger the upgrade to the second level within the expected time window, the prediction is deemed to have failed, meaning that the previous prediction of the user's voice input intent has failed to translate into actual voice input behavior.

[0071] At this point, it is necessary to release or downgrade the resources occupied by the first-level operation to avoid unnecessary power consumption and memory overhead. Specifically, if the first-level operation involves loading a speech recognition model, the model can be unloaded from memory to free up storage space; if it involves initializing the speech recognition pipeline, the software resources occupied by each module in the pipeline can be released or the pipeline can be reset to an uninitialized state; if a model-resident approach is used, the model can be switched from an active state to a low-power standby state so that it can be quickly reused when the intent is predicted again. Through the above resource release or downgrading process, computing and memory resources can be promptly reclaimed when the prediction fails, leaving running space for other applications or services.

[0072] Step D20: If, after performing the second-level voice input wake-up operation, no audio input by the user is detected within the expected time window, it is determined that the prediction has failed, and the microphone is released, writing audio to the circular buffer is stopped, and unconfirmed audio in the circular buffer is discarded.

[0073] After performing the second-level voice input activation operation, the microphone is occupied, the circular buffer is activated, and audio data is continuously written. At this point, it is expected that the user will actually start speaking within a short period of time. Similar to step D10, an expected time window is also set to evaluate the predictive effectiveness at the second level.

[0074] If the microphone fails to capture a valid user voice signal within the expected time window, the prediction is considered unsuccessful, meaning that the microphone was fully prepared for voice recognition but the user ultimately did not initiate voice input.

[0075] At this point, it's crucial to promptly release the resources occupied by the second-level operation. Specifically, microphone resources should be released first, ensuring the microphone is no longer exclusively used by the current function and can be accessed by other applications or services. Simultaneously, writing audio data to the circular buffer should cease to terminate the continuous acquisition and storage of the audio stream. Furthermore, audio data already stored in the circular buffer but not yet confirmed as valid voice input should be discarded to clear the buffer area and prevent residual historical audio data from interfering with subsequent circular buffer write operations. Through these resource release and buffer clearing operations, the device can quickly exit the high-power second-level state when the prediction fails, returning to a low-power standby state. This ensures timely voice input activation while reasonably controlling the overall power consumption level of the device.

[0076] In one possible implementation, after the step of performing a first-level or second-level voice input activation operation based on the intent probability and a preset hierarchical gating, the method further includes: Step E10: Obtain user behavior feedback after this prediction, wherein the behavior feedback includes whether the user inputs audio within the expected time window and whether the second level of arousal preparation is adopted; After performing a first-level or second-level voice input activation operation, in order to further optimize the accuracy of subsequent intent prediction, behavioral feedback generated by the user during the actual interaction process is obtained.

[0077] The behavioral feedback includes at least the following two types of information: First, whether the user actually input audio within the expected time window, that is, whether the user initiated the voice input behavior within the time window predicted by the system; Second, whether the second-level wake-up preparation was adopted, that is, after performing the second-level voice input wake-up operation, whether the user actually used the ready pipeline to input voice, or whether the function was not used after the preparation was completed.

[0078] The behavioral feedback described above reflects the user's actual response to the prediction results and objectively indicates whether the intention prediction was correct. This behavioral feedback can be obtained by monitoring internal state changes and user interaction logs.

[0079] Step E20: Generate updated samples based on the behavioral feedback; After obtaining user behavior feedback, it is associated with the sequence of interface operation events and contextual features used in this prediction to generate training samples for model updates.

[0080] Specifically, if the behavioral feedback indicates that the user actually inputs audio within the expected time window, or the second-level arousal preparation is actually adopted by the user, then this prediction is marked as a positive sample, meaning that the model's output intent probability is consistent with the user's actual behavior; if the behavioral feedback indicates that the user does not input audio within the expected time window, or the second-level arousal preparation is not adopted, then this prediction is marked as a negative sample, meaning that the model's output intent probability is inconsistent with the user's actual behavior.

[0081] The generation of updated samples can be done online in real time, i.e., immediately after each prediction; or it can be done offline in batches, i.e., collecting prediction and feedback data over a period of time and then generating a unified sample set. By transforming feedback information into labeled training samples, the basic data required for supervised learning is provided for subsequent model parameter updates.

[0082] Step E30: Input the updated sample into the model used to predict the probability of the intent, so as to update the parameters of the model; The update involves updating the lightweight classification head or preference parameters of the model, while the pre-trained base model remains unchanged.

[0083] After generating updated samples, they are input into the intent prediction model for parameter updates, enabling the model to learn from users' real behavioral feedback and gradually improve the accuracy of predicting future intents.

[0084] It's important to note that this update operation involves updating the lightweight classifier head or preference parameters of the model, while the pre-trained base model remains unchanged. The lightweight classifier head refers to the top-level network layer in the model structure that outputs the final probability decision; it has relatively few parameters, requiring less computational and time overhead for updating. Preference parameters refer to the weights assigned to different feature dimensions in the model; adjusting these parameters can also optimize model behavior at a lower cost. Keeping the pre-trained base model unchanged means that the model's low-level feature extraction capabilities are preserved, eliminating the need for extensive full retraining, while also leveraging the general representation capabilities learned by the base model on large-scale data.

[0085] By using the aforementioned lightweight incremental update method, continuous model optimization can be achieved without significantly increasing computational overhead. This allows the intent prediction model to adapt to different users' operating habits and behavioral patterns, thereby further improving the accuracy of voice input activation timing and reducing false triggers and missed triggers.

[0086] In one possible implementation, the steps of predicting the probability of the user's intention to make voice input and performing the voice input activation operation are both performed before the user inputs voice or performs an explicit voice input activation action, wherein the explicit voice input activation action includes at least one of voice input wake word, press and hold input voice operation and tap voice button.

[0087] The intent probability prediction step and the step of performing hierarchical voice input activation based on the intent probability are both completed before the user inputs voice or actively performs an explicit voice input activation action. An explicit voice input activation action refers to the operation method by which the user actively triggers the voice input function through a clear interactive behavior, including but not limited to at least one of the following situations.

[0088] Voice input wake-up words refer to preset wake-up words spoken by the user. After the device recognizes the wake-up word, it starts the voice recognition process. This is a typical explicit voice triggering method. For example, the user says preset phrases such as "Hello Voice" or "Start Recording" to clearly instruct the device to enter voice input mode.

[0089] Press-and-hold voice input refers to users triggering voice input by physically pressing a button, such as pressing and holding a specific button on a touchscreen, pressing a physical button, or applying continuous pressure to a touchpad. While holding the button down, the user speaks, and releasing the button ends the input and triggers recognition. This type of operation also relies on the user actively performing a physical action to clearly express their voice input intention.

[0090] Tapping the voice button means that the user triggers voice input by clicking, such as clicking the microphone icon or voice input control on the screen. After clicking, audio is captured and recognized. Compared to pressing and holding input, tapping is usually a one-time trigger and does not require continuous pressing.

[0091] In the aforementioned explicit wake-up methods, the loading and initialization processes of the speech recognition pipeline are only triggered after the user performs the explicit action, resulting in pipeline startup time consuming the user's speaking window. However, in this embodiment, intent prediction and hierarchical wake-up operations are initiated before the explicit action is executed. When the user actually speaks the wake word, presses a button, or clicks the voice button, the speech recognition pipeline is already in a ready state, and the circular buffer has already stored the audio data. Therefore, even if the user starts speaking while performing the explicit wake-up action, the initial syllable of their speech can be completely captured by the circular buffer and processed immediately by the ready pipeline. This effectively avoids syllable loss caused by the cold start process occupying the speaking window in traditional solutions, allowing the speech recognition model to decode based on the complete audio signal, thereby improving the integrity and recognition accuracy of the speech input.

[0092] For example, to aid in understanding the technical concept or principle of the voice input activation method combined with the first embodiment described above, a specific embodiment is now provided. In this specific embodiment, the voice input activation method is applied to a voice interaction device, referring to... Figure 3 As shown in the figure, this diagram illustrates a comparison of the proposed solution and the traditional solution in terms of voice input timeline. In the traditional solution, after the user performs an explicit voice input action, there is a cold start waiting process. This process includes occupying microphone resources, loading the recognition model, and initializing the recognition pipeline. Only after these operations are completed can the system begin to receive the user's voice signal. This results in a significant delay window between the user speaking and the recognition of the first syllable on the screen. When the user is in a silent state waiting for the system to be ready, the initial syllable often falls within the start window and is lost. In this solution, through intent prediction and hierarchical wake-up mechanisms, the voice recognition pipeline is already in a ready state before the user speaks. At the same time, the circular buffer mechanism is activated and continuously retains audio data, achieving a pre-wake-up effect. When the user actually speaks, the system can backtrack and supplement words based on the audio data already stored in the circular buffer, enabling the first word to be recognized and displayed on the screen with near-zero latency. The latency perceived by the user is significantly reduced.

[0093] Specifically, refer to Figure 4 As shown in the figure, this is a schematic diagram of the state transition of the voice input wake-up method in this embodiment. The system is initially in an idle state, at which time no wake-up preparation operation is performed. When the intent probability rises and exceeds the pre-wake-up gate (i.e., the first gate threshold), the system transitions to the pre-warm-up state (i.e., the first level). In this state, at least one operation is performed, such as dwelling or loading the speech recognition model and initializing the speech recognition pipeline, but microphone resources are not occupied and no recording is performed. When the intent probability continues to rise and exceeds the pre-wake-up gate (i.e., the second gate threshold), the system transitions to the pre-wake-up state (i.e., the second level). In this state, microphone resources are occupied, the circular buffer is opened, and the speech recognition pipeline enters the ready state. When the user's mouth is detected (i.e., the user inputs audio), first word replacement can be performed and edge recognition can be performed. In the pre-warm-up state or the pre-wake-up state, if the intent probability falls below the corresponding gate threshold, the system reverts to the previous state or the idle state; if the prediction fails in the pre-wake-up state, that is, no user's mouth is detected within the expected time window, silent reclamation is triggered, releasing the occupied microphone and buffer resources.

[0094] Furthermore, refer to Figure 5 As shown, after performing intent prediction and hierarchical arousal operations, the speech intent prediction result is obtained, and the judgment is based on whether the user actually speaks within the expected time window and whether the pre-arousal result is adopted. If the prediction fails, a false arousal recovery operation is triggered, and the specific level of the recovery operation is determined according to the current arousal level. At the same time, updated samples are generated based on behavioral feedback for online updates to the predictor. This update is limited to adjusting the model's lightweight classification head or preference parameters, while the base model remains frozen. Abnormal feedback samples are downweighted or not updated, and a rollback mechanism for model parameters is supported. All the above operation sequence acquisition, intent prediction, unconfirmed audio buffering during pre-arousal, and online updates are all completed locally on the device, ensuring that user data is not transmitted externally. Through the above mechanism, the predictor can gradually adapt to the user's operating habits and usage patterns, further improving the accuracy of intent prediction and the precision of arousal timing in subsequent interactions.

[0095] It should be noted that the above examples are only used to help understand this embodiment and do not constitute a limitation on the voice input activation process of this embodiment. Any simple modifications based on this technical concept are within the protection scope of this application.

[0096] Based on the first and / or second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to the first and second embodiments described above can be referred to the above description and will not be repeated hereafter. Furthermore, before the step of predicting the probability of the user's intention to perform voice input based on the interface operation event sequence and the context features, the method further includes: Step F10: Obtain the time interval between adjacent operations in the interface operation event sequence, and the operation type transition probability between each adjacent operation. The operation type transition probability is obtained by querying a pre-configured operation type transition probability matrix. Any element in the operation type transition probability matrix is ​​used to characterize the probability of transitioning from one operation type to another. Before predicting a user's voice input intent, the coherence of the interface operation event sequence can be evaluated to determine whether the current operation sequence conforms to normal interaction logic, thereby deciding whether subsequent intent prediction is necessary. Specifically, the time interval between each pair of adjacent operations in the interface operation event sequence is first obtained. This time interval can be calculated by the difference between the timestamps of the subsequent and preceding operations. Simultaneously, the operation type transition probability between each adjacent operation needs to be obtained. This probability characterizes the likelihood of switching from one operation type to another within a historical interaction pattern.

[0097] The operation type transition probability is obtained by pre-configuring an operation type transition probability matrix. This matrix uses rows and columns to correspond to different operation types, and each element represents the probability of transitioning from the operation type corresponding to a row to the operation type corresponding to a column. This matrix can be pre-calculated by statistically analyzing a large amount of historical user interaction logs. For example, it can be calculated by counting the frequency of transitions from "clicking the input box" to "clicking the voice button" in all samples and normalizing these frequencies into transition probability values. When the transition probability of a pair of adjacent operations is needed, simply query the corresponding element value in the matrix based on the types of the preceding and following operations.

[0098] Step F20: For each pair of adjacent operations in the sequence of interface operation events, calculate the local consistency score of the adjacent operations based on the time interval between the adjacent operations and the operation type transition probability. After obtaining the time interval and operation type transition probability between adjacent operations, a local consistency score is calculated for each pair of adjacent operations in the interface operation event sequence. This score is used to measure the rationality and coherence of the temporal relationship between the pair of adjacent operations.

[0099] The specific calculation method can be based on the following logic: When the time interval between two operations is short, it indicates that the user is performing operations quickly and continuously. In this case, if the probability of operation type transition is high, it means that this operation transition is relatively common in historical interaction patterns, and the local consistency score will be correspondingly high. Conversely, if the probability of operation type transition is low, it means that this transition is relatively rare, and the local consistency score will be relatively low. In addition, when the time interval is long, even if the probability of operation type transition is high, the score weight can be appropriately reduced, because an excessively long time interval may mean that the user has interrupted the current operation process. Based on the above considerations, the local consistency score can be designed as the product or weighted combination of the operation type transition probability and the time interval decay factor to comprehensively reflect the coherence between adjacent operations.

[0100] Step F30: Generate a causal consistency score for the interface operation event sequence based on the local consistency scores of each pair of adjacent operations in the interface operation event sequence, wherein the causal consistency score is used to characterize the degree of logical coherence between operations before and after in the interface operation event sequence. After calculating the local consistency scores for each pair of adjacent operations in the sequence, these scores are aggregated to generate the causal consistency score for the entire sequence of interface operation events. This causal consistency score is used to characterize the degree of logical coherence between operations preceding and following each other in the current operation sequence.

[0101] Specifically, the causal consistency score can be obtained by averaging the local consistency scores of all adjacent operations in the sequence, or by using a weighted summation method, such as assigning higher weights to adjacent operations closer to the current time step, since recent interactions are more indicative of the user's current intent. A high causal consistency score indicates that the current operation sequence exhibits a relatively smooth logical progression, and the user is interacting according to a certain expected behavioral path, making it unlikely to be a misoperation or random click.

[0102] Step F40: If the causal consistency score is greater than or equal to a preset consistency threshold, then perform the prediction of the user's intent to perform voice input based on the interface operation event sequence and the context features.

[0103] After calculating the causal consistency score of the interface operation event sequence, it is compared with a preset consistency threshold. If the score is greater than or equal to the consistency threshold, it indicates that the current operation sequence has high logical coherence and conforms to the typical behavior pattern of users in real interaction scenarios. At this time, it is determined that the current interface operation state has the basic conditions for intention prediction, thus triggering the subsequent step of predicting the probability of the user's intention to make voice input based on the interface operation event sequence and contextual features.

[0104] Conversely, if the causal consistency score is below the consistency threshold, it indicates that there are many abnormal jumps, excessively long pauses, or unreasonable operation transitions in the current operation sequence, which may stem from user mis-touches, interface lag, or atypical interaction behavior. In this case, intention prediction based on such low-quality operation sequences lacks reliability, so subsequent intention prediction steps are not executed. This pre-screening mechanism can ensure prediction accuracy while avoiding unnecessary execution of intention prediction and voice input activation operations, thereby effectively reducing the resource overhead caused by invalid computation.

[0105] Furthermore, this application also proposes a voice interaction device, including a microphone, a processor, and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the steps of the voice input activation method described above.

[0106] The voice interaction device provided in this application, employing the voice input activation method described in the above embodiments, can solve the technical problem of how to improve the integrity and recognition accuracy of voice input. Compared with the prior art, the beneficial effects of the voice interaction device provided in this application are the same as those of the voice input activation method provided in the above embodiments, and other technical features of this voice interaction device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0107] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0108] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0109] In addition, to achieve the above objectives, embodiments of this application also provide a readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the voice input wake-up method in the above embodiments.

[0110] The computer-readable storage medium provided in this application embodiment may be, for example, a USB flash drive, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0111] The aforementioned computer-readable storage medium may be included in the voice interaction device; or it may exist independently and not be assembled into the voice interaction device.

[0112] The aforementioned computer-readable storage medium carries one or more programs that, when executed by a voice interaction device, cause the voice interaction device to perform the process steps of any embodiment of the aforementioned voice input activation method.

[0113] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0114] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0115] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules are not, in some cases, limiting the scope of the module itself.

[0116] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described voice input activation method, thereby solving the technical problem of how to improve the integrity and recognition accuracy of voice input. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the voice input activation method provided in the above embodiments, and will not be repeated here.

[0117] Furthermore, embodiments of this application also propose a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-described voice input activation method.

[0118] The specific implementation of the computer program product in this application is basically the same as the embodiments of the above-described voice input activation method, and will not be repeated here.

[0119] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0120] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0121] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases, the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software sensor. This computer software sensor is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a voice interaction device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0122] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for activating voice input, characterized in that, The voice input activation method includes the following steps: Obtain the sequence of interface operation events and context features, wherein the context features include at least one of input control type, application scenario and operation timing rhythm; Based on the interface operation event sequence and the context features, predict the probability of the user's intention to perform voice input; Based on the intent probability and the preset hierarchical gating, execute the first level or the second level of voice input activation operation; The first-level voice input activation operation includes at least one operation among executing a resident speech recognition model or loading a speech recognition model and initializing a speech recognition pipeline, and determines whether to upgrade from the first level to the second level or exit the voice input activation operation based on the change in the intent probability. The second-level voice input activation operation includes putting the voice recognition pipeline into a ready state and activating the circular buffer to retain the acquired audio through the circular buffer, and performing voice recognition based on the audio cached in the circular buffer through the voice recognition pipeline.

2. The voice input activation method as described in claim 1, characterized in that, After the step of performing a first-level or second-level voice input activation operation based on the intent probability and a preset hierarchical gating, the method further includes: If user input audio is detected after the second-level voice input activation operation has finished, then endpoint detection is performed on the input audio to obtain the endpoint detection result; Based on the endpoint detection results, the candidate audio start time is determined, and in the circular buffer, a backtracking window is defined forward from the candidate audio start time as the endpoint; The backtracking starting point is determined based on the acoustic characteristics of the audio within the backtracking window, wherein the acoustic characteristics include at least one of energy mutation characteristics, acoustic confidence characteristics, and noise level; Extract the audio segment from the backtracking starting point to the audio start candidate time from the circular buffer; The extracted audio segment is merged with the audio collected after the candidate time point to form a complete audio, and the complete audio is input into the speech recognition pipeline for speech recognition.

3. The voice input activation method as described in claim 1, characterized in that, The step of performing a first-level or second-level voice input activation operation based on the intent probability and a preset hierarchical gating includes: Obtain a preset hierarchical gating system, wherein the hierarchical gating system includes a first gating threshold and a second gating threshold, and the second gating threshold is greater than the first gating threshold; If the intent probability is greater than or equal to the first gating threshold and less than the second gating threshold, then the first level of voice input wake-up operation is performed. If the intent probability is greater than or equal to the second gating threshold, then the second level of voice input activation operation is performed.

4. The voice input activation method as described in claim 3, characterized in that, The method further includes: Configure a de-jitter mechanism to suppress frequent level switching caused by fluctuations in the intent probability around the gating threshold; The anti-shake mechanism includes at least one of the following: hysteresis threshold with different thresholds for upgrading and downgrading, minimum dwell time, cooldown time, consecutive hit count, and intent probability smoothing.

5. The voice input activation method as described in claim 1, characterized in that, After the step of performing a first-level or second-level voice input activation operation based on the intent probability and a preset hierarchical gating, the method further includes: If the first-level voice input wake-up operation is executed, but the second-level voice input wake-up operation is not upgraded within the expected time window, then the prediction is determined to be unsuccessful, and the resources occupied by the voice recognition model and the voice recognition pipeline are released or downgraded. If, after performing the second-level voice input wake-up operation, no audio input from the user is detected within the expected time window, the prediction is deemed unsuccessful, the microphone is released, writing audio to the circular buffer stops, and unconfirmed audio in the circular buffer is discarded.

6. The voice input activation method as described in claim 1, characterized in that, After the step of performing a first-level or second-level voice input activation operation based on the intent probability and a preset hierarchical gating, the method further includes: Obtain user behavior feedback after this prediction, wherein the behavior feedback includes whether the user inputs audio within the expected time window and whether the second level of arousal preparation is adopted; An updated sample is generated based on the behavioral feedback; The updated sample is input into the model used to predict the probability of the intent, so as to update the parameters of the model; The update involves updating the lightweight classification head or preference parameters of the model, while the pre-trained base model remains unchanged.

7. The voice input activation method as described in claim 1, characterized in that, The steps of predicting the probability of the user's intention to make voice input and the steps of performing voice input activation are both performed before the user inputs voice or performs an explicit voice input activation action. The explicit voice input activation action includes at least one of voice input wake-up word, pressing and holding input voice operation and pressing voice button.

8. The voice input activation method as described in any one of claims 1 to 7, characterized in that, Before the step of predicting the probability of the user's intent to perform voice input based on the interface operation event sequence and the context features, the method further includes: The time interval between adjacent operations in the interface operation event sequence and the operation type transition probability between each adjacent operation are obtained. The operation type transition probability is obtained by querying a pre-configured operation type transition probability matrix. Any element in the operation type transition probability matrix is ​​used to characterize the probability of transitioning from one operation type to another. For each pair of adjacent operations in the sequence of interface operation events, a local consistency score of the adjacent operations is calculated based on the time interval between the adjacent operations and the operation type transition probability. Based on the local consistency scores of each pair of adjacent operations in the interface operation event sequence, a causal consistency score is generated for the interface operation event sequence, wherein the causal consistency score is used to characterize the degree of logical coherence between operations before and after in the interface operation event sequence. If the causal consistency score is greater than or equal to a preset consistency threshold, then the probability of predicting the user's intention to perform voice input based on the interface operation event sequence and the context features is executed.

9. A voice interaction device, characterized in that, include: A microphone, a processor, and a memory, the memory storing a computer program that, when executed by the processor, implements the steps of the voice input activation method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the voice input activation method as described in any one of claims 1 to 8.

Images