Embedded terminal-oriented low-delay streaming voice wake-up method and embedded terminal-oriented low-delay streaming voice wake-up device

Abstract

The invention discloses a low-delay streaming voice wake-up method and device for an embedded terminal, is applied to the technical field of data processing, and adopts a GRU as a backbone architecture for the requirements of low power consumption and high real-time performance of voice wake-up of the embedded terminal. The original audio is segmented and reconstructed through time domain lightweight preprocessing, and the FFT overhead is eliminated through linear layer mapping; building a multi-layer GRU full-causal streaming coding network, wherein no sliding window cache exists, and the memory bandwidth occupies O (1); outputting Token probability distribution containing Blank through a CTCLoss training module, and directionally extracting a target Token probability; constructing a target index sequence based on the target wake-up word and a CTC protocol, and setting three types of state transition rules to generate anti-interference decoding information; and a non-Top-K pruning decoding mechanism is adopted, an attenuation factor and threshold judgment is combined, a judgment result with constant calculation complexity and low false and missing wakeup is generated, and subsequent interaction is triggered if the judgment result succeeds.

Description

A low-latency streaming voice wake-up method and apparatus for embedded terminals Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a low-latency streaming voice wake-up method and apparatus for embedded terminals. Background Technology

[0002] The core processing flow of current voice wake-up (KWS) systems mainly includes three key stages: feature extraction, model inference, and back-end decoding. The mainstream technical solutions for each stage are as follows: In the feature extraction stage, frequency domain feature extraction is commonly used. The original speech signal is converted into frequency domain features such as the Mel-bank or Mel-Cepstral Conversion (MFCC) through Fast Fourier Transform (FFT), relying on complex time-frequency transformation logic to achieve feature extraction. In the model inference stage, convolutional frameworks such as Deep Feedforward Sequential Memory Networks (DFSMN) are often used. By setting left and right look-ahead windows on the time axis, long-term context information is extracted using convolutional kernels, requiring a sliding window buffer to ensure computational continuity. In the back-end decoding stage, the mainstream solution is the Prefix Beam Search algorithm. This algorithm maintains a dynamic candidate dictionary at each time step, sorts probability paths, and performs Top-K pruning to ultimately determine the wake-up word sequence, relying on pruning operations to control the computational load.

[0003] Existing technologies in embedded terminal applications suffer from the following unavoidable technical shortcomings, making it difficult to meet the core requirements of low power consumption, high real-time performance, and high accuracy: High preprocessing computational overhead and difficulty in adapting to low-power edge devices: Traditional frequency domain feature extraction relies on FFT and complex nonlinear mappings, involving a large number of floating-point operations. This generates a significant computational load on low-power DSPs (Digital Signal Processors), consuming valuable hardware computing resources and shortening device battery life, thus failing to meet the low-power requirements of embedded terminals. Complex model inference cache management and excessive memory bandwidth consumption: During streaming inference, convolutional frameworks such as DFSMN and CNN require each layer to maintain a sliding window cache of tens of frames, and frequent concatenation, movement, and pointer addressing operations are required within each frame inference cycle. For resource-constrained edge chips, this frequent memory access causes a severe memory bandwidth bottleneck, further increasing device power consumption. The decoding stage suffers from high computational uncertainty and a high risk of missed wake-ups: The Prefix Beam Search algorithm relies on Top-K pruning to limit candidate paths, which presents an inherent contradiction between computational performance and wake-up effectiveness. If the Top-K value is too large, it will increase the burden and power consumption of the edge device. If the value is too small, it will easily lead to the pruning of correct candidate tokens in scenarios with low signal-to-noise ratio, resulting in missed wake-ups. At the same time, the algorithm maintains the prefix path with the highest probability. In scenarios where there are irrelevant insertions in the keywords, it will cause false wake-ups due to path filtering or forced merging. Furthermore, dynamic memory allocation is prone to memory fragmentation, and the computation time fluctuates greatly, which does not meet the real-time requirements of industrial embedded systems. The search logic based on sorting and hash lookup is also difficult to achieve vectorization acceleration on DSP or NPU. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides the following technical solution: a low-latency streaming voice wake-up method for embedded terminals, comprising: acquiring the core application requirements of low power consumption and high real-time performance for voice wake-up of embedded terminals, and determining a GRU backbone model architecture adapted to these requirements; performing time-domain lightweight preprocessing on the continuously input raw audio sampling point sequence, segmenting and reconstructing it into a two-dimensional vector according to a preset frame length, projecting it onto a high-dimensional feature space through a fully connected linear layer, and generating time-domain feature reconstruction and linear mapping information without FFT floating-point operations and cache overhead; constructing a multi-layer GRU fully causal streaming coding network, initializing the hidden state vector to all zeros, and updating each frame's inference only based on the current frame's feature vector and the hidden state at the previous time step, generating GRU hidden state dynamic update information without sliding window cache and with O(1) memory bandwidth usage; inputting the above two types of information into a character array trained by CTCLoss. The prediction module outputs the probability distribution of all tokens including Blank placeholders, extracts the probability of the target wake word token, and generates CTC character prediction and target token extraction probability distribution information. Based on the target wake word token sequence and CTC protocol, it constructs a target index sequence of length L containing Blank, defines a matching one-dimensional state probability vector, sets three types of closed-loop state transition constraint rules, and generates a decoded state vector and path transition information isolated from non-target token interference through forward probability accumulation calculation. Based on a streaming dynamic programming decoding mechanism without Top-K pruning, combined with long-term energy processing of attenuation factors and threshold triggering decision rules at the end of the state vector, and combined with terminal hardware state calculation, it generates voice wake-up judgment information with low computational complexity, low false wake-up errors, and high real-time performance. If the wake-up is successful, the subsequent voice interaction module is triggered.

[0005] A low-latency streaming voice wake-up device for embedded terminals is provided, the device being used to execute executable instructions to perform the aforementioned low-latency streaming voice wake-up method for embedded terminals.

[0006] Its beneficial effects are as follows: This invention proposes an end-to-end streaming solution for the low power consumption and high real-time performance requirements of voice wake-up in embedded terminals. First, the GRU backbone architecture is determined, and the original audio is preprocessed in the temporal domain. After segmentation and reconstruction, feature information without FFT overhead is generated by linear layer mapping. A multi-layer GRU fully causal streaming coding network is built with no sliding window buffer and memory bandwidth usage of O(1). The target token probability is extracted in a targeted manner through the character prediction module trained by CTCLoss. The target index sequence is constructed based on the target wake-up word and CTC protocol, and three types of state transition rules are set to generate anti-interference decoding information. A Top-K pruning decoding mechanism is adopted, combined with attenuation factor and threshold decision, to generate wake-up judgment information with a constant computational complexity of O(L). If successful, subsequent interaction is triggered.

[0007] This invention aims to eliminate FFT operations and sliding window caching, reduce memory access and floating-point operations, and adapt to the battery life requirements of embedded terminals; the computational complexity per frame is constant, with no dynamic path fluctuations, and low and stable wake-up latency; through path constraints and directional probability extraction, interference from non-target tokens is isolated, reducing the false wake-up rate; the computational logic is simple, supports vectorization acceleration, adapts to edge hardware such as DSP / NPU, and has high deployment flexibility. Attached Figure Description

[0008] Figure 1 is a flowchart of a low-latency streaming voice wake-up method for embedded terminals provided by an embodiment of the present invention; Figure 2 is a schematic diagram of a module of a low-latency streaming voice wake-up device for embedded terminals provided by an embodiment of the present invention. Detailed Implementation

[0009] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention. A low-latency streaming voice wake-up method for embedded terminals according to an exemplary embodiment of this application will be described below with reference to FIG1. ​​In one embodiment, this application also proposes a low-latency streaming voice wake-up method for embedded terminals.

[0010] In this embodiment of the application, a low-latency streaming voice wake-up method for embedded terminals is shown in Figure 1: S101, obtain the core application requirements of low power consumption and high real-time performance for voice wake-up of embedded terminals, and determine the GRU backbone model architecture that adapts to the requirements.

[0011] In one implementation, embedded terminals (such as low-power IoT devices and portable smart hardware) typically rely on battery power. It is necessary to control the computational resource consumption, memory bandwidth usage, and floating-point operations of the voice wake-up system to avoid shortened battery life due to excessive power consumption. This requirement addresses the high power consumption issues caused by FFT frequency domain transformation and convolutional architecture sliding window caching in existing technologies. Embedded terminal voice wake-up needs to respond quickly to user commands, requiring a constant processing time per frame with no computational uncertainty, avoiding wake-up delays caused by dynamic memory allocation and path search fluctuations, and meeting the real-time interaction requirements of industrial-grade embedded systems.

[0012] By analyzing the hardware parameters of embedded terminals (such as DSP / NPU computing power, memory capacity, and power consumption threshold), application scenarios (such as offline voice control and low-power standby wake-up), and user interaction requirements (such as fast response and low false wake-up), the core requirement indicators are determined comprehensively without relying on additional external data input, and are derived solely based on the characteristics of the terminal itself and the business scenario.

[0013] To address the core requirements of low power consumption and high real-time performance, and comparing the shortcomings of existing convolutional frameworks such as DFSMN and CNN, GRU (Gated Recurrent Unit) was chosen as the backbone model architecture. The selection logic is as follows: GRU does not require maintaining a lengthy sliding window cache like convolutional frameworks; it only needs to store a fixed-dimensional hidden state vector, greatly reducing memory access frequency and space occupation, and lowering hardware power consumption. Simultaneously, it avoids the frequent cache concatenation and shifting operations found in convolutional architectures, reducing unnecessary computational overhead. GRU inherently supports fully causal streaming computation; each frame's inference only depends on the current frame's feature vector and the hidden state from the previous time step. The computational logic is simple, with no dynamic search space fluctuations, ensuring stable processing complexity for each frame and meeting real-time requirements. GRU's recursive hidden state update mechanism does not require complex floating-point operations (such as FFT), making it compatible with the hardware computing characteristics of low-power DSPs / NPUs and facilitating vectorization acceleration.

[0014] A multi-layer GRU structure is adopted as the backbone network, and the number of layers is flexibly configured according to the computing power resources of the embedded terminal. Specifically, this application establishes a layer matching mechanism based on the effective peak computing power of the embedded terminal (determined by the CPU clock speed and instruction set parallel processing capability): under computing power resources with high-frequency vector operation capabilities (such as high-performance processors with a clock speed >= 240MHz), a dual-layer or higher GRU structure is configured to enhance the temporal modeling depth in complex acoustic environments; under limited computing power resources that only support basic scalar operations or have low clock speeds (such as processors with a clock speed <= 160MHz or no hardware acceleration module), a single-layer GRU is configured to prioritize inference real-time performance. The specific configuration process uses the ratio of single-frame inference time to audio frame length (real-time duty cycle) as a quantification criterion to ensure that the model computation overhead does not exceed 30% of the total computing power of the terminal, thereby achieving dynamic optimal adaptation among wake-up real-time performance, recognition accuracy, and computing power load on hardware platforms with different performance levels. Each GRU layer is connected in series. The output hidden state of the previous GRU layer serves as the input hidden state of the next GRU layer. The output hidden state of the final GRU layer is directly passed to the subsequent CTC character prediction module, forming an end-to-end feature encoding link. A fixed dimension is set according to the memory bandwidth limit of the embedded terminal to ensure that the memory bandwidth usage is reduced to the O(1) level (related only to the hidden layer dimension and not to the number of historical frames). The native Sigmoid gate activation function and Tanh activation function of GRU are used to balance the computational complexity and feature representation capability, avoiding additional floating-point operation overhead.

[0015] By comparing the memory access frequency and computational load ratio of GRU and DFSMN / CNN architectures, the study verifies the reduction effect of GRU on memory bandwidth usage and floating-point operation volume after eliminating the sliding window cache, ensuring that the low power consumption threshold of embedded terminals is met. The study also tests the processing time of each frame of GRU streaming inference, confirming that the computational complexity does not fluctuate with the inference duration and that there is no time uncertainty caused by dynamic memory allocation, thus meeting industrial-grade real-time requirements. Furthermore, considering the instruction set characteristics of low-power DSP / NPU, the study verifies whether GRU's hidden state update logic supports vectorization acceleration, avoiding the problem of traditional BeamSearch algorithms being difficult to adapt to edge hardware.

[0016] S102 performs time-domain lightweight preprocessing on the continuously input raw audio sampling point sequence, segments it according to the preset frame length and reconstructs it into a two-dimensional vector, projects it onto the high-dimensional feature space through a fully connected linear layer, and generates time-domain feature reconstruction and linear mapping information without FFT floating-point operation and cache overhead.

[0017] In one implementation, combining the low-power voice wake-up requirements of embedded terminals with the original audio time-domain processing logic, an audio frame length segmentation rule and a time-domain feature linear mapping mechanism are introduced to perform time-domain lightweight preprocessing on the continuously input original audio sampling point sequence. By combining the core requirements of low-power voice wake-up in embedded terminals (controlling computing resource usage and reducing energy consumption) with the continuous characteristics of the original audio time-domain data, an audio frame length segmentation rule and a time-domain feature linear mapping mechanism are specifically introduced to replace the complex FFT frequency domain transformation logic in traditional solutions, reducing floating-point operation overhead from the source and achieving a "lightweight" design in the preprocessing stage.

[0018] The audio frame length segmentation rules are as follows: based on the computing power and memory bandwidth limitations of the embedded terminal, a fixed frame length (chunk_size) is preset to ensure that the segmented audio segments can retain effective speech features without increasing the computational burden due to excessive segment length; the temporal feature linear mapping mechanism is as follows: a learnable linear mapping layer replaces the traditional physical feature extraction (such as Fbank / MFCC), enabling the system to adaptively extract wake-up-related representations from the original temporal waveform, avoiding the computational load caused by complex nonlinear mapping.

[0019] The original audio sampling point sequence is segmented according to a preset frame length, reconstructing the one-dimensional audio sampling point sequence into a two-dimensional vector, thus completing the dimensional reorganization of the audio temporal features. The system receives a continuous input sequence of original audio sampling points (one-dimensional time series) and segments it into equal lengths according to a preset frame length (chunk_size). This reorganization reconstructs the original continuous one-dimensional sampling point sequence into a two-dimensional vector of shape (N, chunk_size), where N is the number of frames after segmentation, and chunk_size is the number of sampling points in each frame. This dimensional reorganization makes the temporal features more suitable for parallel computation and feature extraction in subsequent models.

[0020] The frame length (chunk_size) setting needs to be combined with the sampling rate and hardware processing capabilities of the embedded terminal. At a sampling rate of 16kHz, when the chunk_size is set to 400, 1 second of audio can be divided into 40 frames, with each frame corresponding to a 25ms audio segment. This parameter balances feature integrity and computational efficiency.

[0021] A fully connected linear layer projects the temporal information of a two-dimensional vector into the high-dimensional feature space of the model's hidden layer, achieving a linear mapping transformation of temporal features. A single fully connected linear layer is used as the mapping module. The input of this module is the dimensionally reorganized two-dimensional vector (N, chunk_size), and the output is the high-dimensional feature vector of the model's hidden layer. The linear layer is directly connected in series with the subsequent GRU streaming encoder module. The mapped high-dimensional feature vector is used as the input of the GRU, forming an end-to-end front-end processing chain of "dimensionality reorganization - linear mapping - feature encoding".

[0022] Fully connected linear layers learn weight parameters to directly project the original temporal information of the chunk_size dimension into a high-dimensional feature space without undergoing FFT frequency domain transformation and complex nonlinear mapping. This reduces computational overhead while preserving key wake-up-related features in the original audio, achieving "lightweight" and "adaptive" feature extraction.

[0023] This process generates temporal feature reconstruction and linear mapping information that eliminates the overhead of FFT floating-point operations and has no additional cache overhead, thus completing the front-end feature extraction for voice wake-up in embedded terminals. The generated temporal feature reconstruction and linear mapping information is a feature set containing N frames of high-dimensional feature vectors (e.g., if the input X is 1*40*400, the linear layer is a simple matrix transformation; in this case, a W matrix is ​​defined as 400*128, and the result of X*B is a 1*40*128 feature matrix). This result must meet two core indicators: first, eliminating the overhead of FFT floating-point operations; and second, having no additional cache overhead, ensuring low power consumption and high efficiency in the front-end feature extraction process.

[0024] The power consumption verification is as follows: comparing the floating-point computation of this scheme with that of the traditional FFT frequency domain feature extraction scheme, it is verified that this scheme has no FFT-related calculations, reducing the floating-point computation by more than 80%, thus meeting the low-power requirements of embedded terminals. The feature effectiveness verification is as follows: the mapped high-dimensional features are input into the GRU streaming encoding module to verify that the features can effectively support subsequent character prediction and wake-up determination, ensuring that the wake-up accuracy is no less than that of the traditional scheme.

[0025] S103, a multi-layer GRU fully causal streaming coding network is built, the hidden state vector is initialized to all 0, and the inference of each frame is updated only according to the feature vector of the current frame and the hidden state of the previous time step, generating GRU hidden state dynamic update information without sliding window buffer and with memory bandwidth of O(1).

[0026] In one implementation, combining the low memory and high real-time requirements of embedded terminal voice wake-up streaming inference with the inherent recursive characteristics of GRU network hidden states, a multi-layer GRU fully causal coding rule and a frame-level hidden state update mechanism are introduced to build a multi-layer GRU fully causal streaming coding network adapted to the terminal side. Considering the core requirements of low-power voice wake-up in embedded terminals (reducing computational resource consumption and energy consumption) and the continuous and temporal characteristics of the original audio time-domain data, a targeted audio frame length segmentation rule and a linear mapping mechanism for time-domain features are introduced to replace the complex FFT frequency domain transformation logic in traditional solutions. This avoids redundant floating-point operations from the preprocessing source, achieving the goal of "lightweight" processing.

[0027] Among them, the audio frame length segmentation rule is constrained by the hardware computing power and memory bandwidth of the embedded terminal, and a fixed frame length (chunk_size) is preset to ensure that the segmented audio segments can retain effective speech features without increasing the hardware processing burden due to excessively long segments; the temporal feature linear mapping mechanism replaces the traditional physical feature extraction (such as Fbank / MFCC) with a learnable linear mapping layer, enabling the system to adaptively mine key representations related to wake-up from the original temporal waveform, avoiding the computational load caused by complex nonlinear mapping.

[0028] Following the initialization specifications for streaming inference, the hidden state vectors of the multi-layer GRU fully causal streaming encoder network are initialized to all zeros, completing the state preparation before network inference. The system receives a continuous sequence of raw audio sampling points (one-dimensional time series) and divides it into equal-length segments according to a preset frame length (chunk_size). This reconstructs the original continuous one-dimensional sampling point sequence into a two-dimensional vector of shape (N, chunk_size)—where N is the number of frames after segmentation, and chunk_size is the number of sampling points per frame. Through dimensionality reorganization, the temporal features are transformed into a matrix form suitable for parallel computation by the subsequent model, improving feature processing efficiency.

[0029] Regarding key parameters, the frame length (chunk_size) setting needs to be combined with the sampling rate and hardware processing capabilities of the embedded terminal. At a sampling rate of 16kHz, when chunk_size=400, 1 second of audio can be divided into 40 frames, with each frame corresponding to a 25ms audio segment. This parameter ensures the integrity of the features of a single frame while controlling the amount of computation per frame, achieving a balance between feature integrity and computational efficiency.

[0030] Based on the frame-level processing logic of GRU fully causal streaming inference, only the temporal feature vector of the current frame and the network hidden state of the previous time step are received as input during each frame inference process. A single fully connected linear layer is used as the mapping module. This module is directly connected in series with the preceding dimension reorganization step and the subsequent GRU streaming encoding module to form an end-to-end front-end processing link of "dimensional reorganization-linear mapping-feature encoding". The input of the linear layer is a dimension-reorganized two-dimensional vector (N, chunk_size), and the output is a high-dimensional feature vector of the model's hidden layer (the dimension is consistent with the dimension of the GRU hidden state).

[0031] In terms of mapping principle, the fully connected linear layer learns the optimal weight parameters through training, directly projecting the original time-domain information of the chunk_size dimension to the high-dimensional feature space without undergoing FFT frequency domain transformation and complex nonlinear mapping. This design reduces floating-point operation overhead and allows the model to adaptively learn key wake-up-related features in the original time-domain waveform, balancing "lightweight" and feature representation capabilities.

[0032] The input feature vector and the previous hidden state are calculated and processed using the GRU network gating mechanism to dynamically update the hidden state of the GRU network at the current time step. This generates GRU hidden state dynamic update information that does not require maintenance of convolutional sliding window cache and reduces memory bandwidth usage to O(1) level, realizing low memory overhead speech feature streaming encoding for embedded terminals. The generated temporal feature reconstruction and linear mapping information is a feature set containing N frames of high-dimensional feature vectors. This result must meet two core indicators: first, completely eliminate the overhead of FFT floating-point operations; second, have no additional cache overhead, ensuring low power consumption and high efficiency in the preprocessing stage.

[0033] Comparing the floating-point computation of this scheme with that of traditional FFT frequency domain feature extraction schemes, it is verified that this scheme has no FFT-related calculations, reducing floating-point computation by more than 80% compared to traditional schemes, thus meeting the low-power requirements of embedded terminals. The mapped high-dimensional feature vector is input into the subsequent GRU streaming encoding module. Through model training and inference verification, it is confirmed that the feature can effectively support subsequent character prediction and wake-up determination, with a wake-up accuracy no lower than that of traditional FFT frequency domain feature extraction schemes, ensuring the practicality of the feature.

[0034] S104. Input the above two types of information into the character prediction module trained by CTCLoss, output the probability distribution of the full token including the Blank placeholder, extract the probability of the target wake word token in a targeted manner, and generate CTC character prediction and target token targeted extraction probability distribution information.

[0035] In one implementation, combining the character sequence prediction requirements for embedded terminal voice wake-up with the feature adaptation characteristics of CTC (Connectionist Temporal Classification) sequence modeling, a CTCLoss training rule and a Token probability distribution output mechanism are introduced to build and train a CTC character prediction module adapted to the terminal side. By combining the character sequence prediction requirements for embedded terminal voice wake-up (requiring accurate output of character probability distributions to support subsequent decoding) with the feature adaptation characteristics of CTC (Connectionist Temporal Classification) sequence modeling (not requiring strict alignment of characters with audio frames, adapting to the temporal flexibility of streaming speech), a CTCLoss training rule and a Token probability distribution output mechanism are introduced to build a CTC character prediction module adapted to low-computing-power scenarios on the terminal side, replacing the traditional fixed wake-up word classification model and providing a foundation for "arbitrary word wake-up".

[0036] This scheme adopts a cascaded architecture of "two-layer mapping-mid-encoding". First, the original temporal features after reconstruction are reduced in dimension and aligned in dimension through the first feature mapping layer. Then, they are input into the GRU streaming encoding module to capture long temporal dependencies and generate hidden state vectors. This vector is further linearly projected to the feature space that is strictly consistent with the dimension of the token table through the second feature mapping layer. Finally, it is converted into the log confidence distribution of the target token through the LogSoftmax layer, so as to directly adapt to the log domain operation requirements of the back-end decoder and ensure the stability of numerical calculation.

[0037] To address the practical application scenarios of voice wake-up on embedded terminals, a systematic collection of multi-dimensional voice data was developed to ensure data coverage and diversity, thereby enhancing the model's generalization ability. The accent dimension covers standard Mandarin accents, regional accents, and pronunciation characteristics of different age groups (children, youth, and the elderly), adapting to the usage scenarios of different user groups. The noise environment dimension includes typical scenarios such as quiet indoor environments, home environments (TV sound, footsteps), outdoor environments (traffic noise, crowd noise), and in-vehicle environments (engine noise, wind noise), simulating noise interference in actual use of embedded terminals. The total training data is ensured to be no less than 100,000 voice segments, with each segment ranging from 1 to 3 seconds in length (covering the complete pronunciation duration of the wake-up word), and a uniform sampling rate of 16kHz (adapting to common sampling standards on edge hardware).

[0038] Precisely annotate each voice segment, with the annotation content being the corresponding character or pinyin Token sequence (for example, if the voice segment is "你好小蝉", it is annotated as the pinyin Token sequence [ni,hao,xiao,chan]), ensuring that the annotation is strictly aligned with the voice content. Divide the annotated data into a training set, a validation set, and a test set according to the ratio of 8:1:1. The training set is used for learning model parameters, the validation set is used to monitor the overfitting situation during training, and the test set is used for the final evaluation of model performance. Normalize the collected voice data (normalize the amplitude of audio sampling points to the interval [-1,1]), and remove overly long silent segments (directly truncate segments with a silent duration exceeding 500ms) to ensure data quality.

[0039] The Token table contains three types of core elements - Chinese pinyin (covering the pinyin and combinations of initials and finals involved in common wake-up words, approximately 63), basic characters (numbers, common wake-up keyword characters), and Blank placeholders (used to accommodate the non-strict alignment characteristics of the CTC protocol); the Token table only covers the phoneme components, necessary basic characters, and Blank placeholders involved in the target wake-up word (the total dimension is controlled within 60 - 120 dimensions). By artificially removing redundant characters irrelevant to the wake-up task, the matrix calculation amount of the model output layer is forced to be limited to the minimum level, avoiding the ineffective computational loss caused by thousands of-dimensional word tables in general speech models.

[0040] This solution uses CTCLoss as the training criterion. Calculate the cumulative probability of all possible alignment paths between the Token probability sequence predicted by the model and the true annotation sequence through the forward-backward algorithm. The loss function quantifies the probability distribution difference between the two by minimizing the negative log-likelihood loss between the predicted sequence and the target sequence, thereby guiding the model to establish the ability of temporal modeling and probability prediction for wake-up word-related Tokens. Preferentially use the AdamW optimizer (the learning rate range is set to 0.0001 to 0.001). Utilize its decoupled weight decay mechanism (WeightDecay is set to 0.0001), while ensuring the model converges quickly in the voice feature space, effectively suppressing the overfitting risk of small-parameter models on specific wake-up word sets. Model parameters (such as hidden layer dimension, word table size) need to be pre-matched with the memory specifications of the target embedded terminal. Ensure that the model file generated during training and the intermediate states required during its inference (such as the HiddenState of GRU) can be completely loaded into the terminal memory (such as in the scenario where the embedded terminal memory ≤ 512MB); set the maximum number of iterations to 50 - 100 rounds. When the CTCLoss on the validation set does not decrease continuously for 5 rounds (the decrease amplitude < 0.0001), it is determined that the model training converges, stop the iteration and save the optimal model parameters.

[0041] An end-to-end training mode is adopted, treating the preceding temporal mapping layer, the intermediate GRU encoding module, and the final fully connected mapping layer as a unified computation graph. During backpropagation, gradients are propagated backward from CTCLoss to each layer of the network, realizing the joint update of feature extraction and sequence modeling parameters, ensuring that the model can adaptively extract discriminative features from the original temporal signal.

[0042] A Dropout mechanism (dropout rate set to 0.1-0.2) is introduced into the fully connected layers and GRU recurrent layers to effectively suppress the risk of overfitting of small-parameter models on specific wake-up word sets by randomly deactivating neurons. An exponential learning rate decay strategy is implemented (e.g., a decay factor of 0.9 every 10 rounds), coupled with a validation set loss monitoring mechanism. If the validation set loss function value does not decrease further within a preset number of rounds, an automatic learning rate reduction or early stopping is triggered, and the model is backtracked to the historically best model weight version. After training, the trigger probability of the model's character sequences is comprehensively evaluated using a test set. By validating the model's recall and false wake-up rate balance under different noise interferences, the probability stream output by the model is ensured to have sufficient discriminative power to meet the threshold decision requirements of the subsequent backend decoder.

[0043] An end-to-end training mode is adopted, treating the preceding temporal mapping layer, the intermediate GRU encoding module, and the final fully connected mapping layer as a unified computation graph. During backpropagation, gradients are propagated backward from CTCLoss to each layer of the network, realizing the joint update of feature extraction and sequence modeling parameters, ensuring that the model can adaptively extract discriminative features from the original temporal signal.

[0044] A Dropout mechanism (dropout rate set to 0.1-0.2) is introduced into the fully connected layers and GRU recurrent layers to effectively suppress the risk of overfitting of small-parameter models on specific wake-up word sets by randomly deactivating neurons. An exponential learning rate decay strategy is implemented (e.g., a decay factor of 0.9 every 10 rounds), coupled with a validation set loss monitoring mechanism. If the validation set loss function value does not decrease further within a preset number of rounds, an automatic learning rate reduction or early stopping is triggered, and the model is backtracked to the historically best model weight version. After training, the trigger probability of the model's character sequences is comprehensively evaluated using a test set. By validating the model's recall and false wake-up rate balance under different noise interferences, the probability stream output by the model is ensured to have sufficient discriminative power to meet the threshold decision requirements of the subsequent backend decoder.

[0045] The main parameters of the trained model are quantized using INT8 weight quantization (PTQ) or quantization-aware training (QAT). Floating-point parameters are mapped to 8-bit integer values, compressing the model storage size to approximately 25% of the original floating-point version while ensuring extremely low wake-up rate loss, and significantly improving vector operation efficiency on embedded systems. Furthermore, during quantization, components used only for training, such as the Dropout layer, are removed, and the LogSoftmax output layer maintains floating-point (Float32) precision without integer quantization. This strategy avoids the numerical saturation and precision loss issues of exponential and logarithmic operations in LogSoftmax under low-bit-width quantization, ensuring that the output token probability stream has sufficient dynamic range to support high-precision logarithmic domain accumulation decisions in the backend decoder.

[0046] The temporal feature recombination and linear mapping information, along with the GRU hidden state dynamic update information, are input to a character prediction module trained with CTCLOss for feature calculation. The module receives two types of feature information: temporal feature recombination and linear mapping information (e.g., a high-dimensional feature vector of dimension (40, 128), corresponding to the temporal mapping result of 40 frames of audio), and GRU hidden state dynamic update information (with the same dimension as the GRU hidden state, such as 128 dimensions). These two types of information are concatenated by dimension before being input to the module. The input features undergo linear transformation and feature fusion through a fully connected mapping layer, mapping the fused features to the Token table dimension space. The mapping result is then converted into probability values ​​using a Softmax activation function, completing the character sequence probability calculation. The entire process involves no complex floating-point operations, making it suitable for low-computing-power hardware on the edge.

[0047] Leveraging the module's character sequence probability prediction capabilities, the system outputs the current probability distribution of all tokens, including placeholders and Chinese pinyin, thus completing the probability prediction of all tokens. Based on the module's trained character sequence probability prediction capabilities, the system outputs the current probability distribution of all tokens, including placeholders and Chinese pinyin—a two-dimensional matrix where rows correspond to audio frames and columns correspond to each token in the token table. Matrix elements represent the probability of a specific token appearing in a given frame. This output provides a complete data foundation for subsequent targeted extraction of target tokens.

[0048] According to the Token sequence corresponding to the target wake-up word, the probability values of the target Tokens are directionally extracted from the full-scale Token probability distribution, and the probability calculation and extraction of non-target Tokens are masked, generating CTC character prediction information and Token directional extraction probability distribution information that only focuses on the target wake-up word, providing accurate target Token probability data for the subsequent decoding stage. According to the Token sequence corresponding to the target wake-up word (for example, the pinyin Token sequence corresponding to the wake-up word "Hello, Xiaochai" is [ni, hao, xiao, chan]), the probability values corresponding to the target Tokens are directionally extracted from the full-scale Token probability distribution through Token indexes, and the probability values of non-target Tokens are directly masked without additional calculation; this extraction process only focuses on the Tokens related to the target wake-up word, eliminating the invalid calculation overhead of non-target Tokens, reducing the burden on the end-side hardware, and at the same time ensuring that the extracted probability data accurately matches the subsequent decoding requirements, providing the core input for the path calculation in the decoding stage.

[0049] S105, Based on the target wake-up word Token sequence and the CTC protocol, construct a target index sequence of length L containing Blank, define a matching one-dimensional state probability vector, set three types of closed-loop state transition constraint rules, and through forward probability cumulative calculation, generate a decoding state vector and path transition information that isolate the interference of non-target Tokens.

[0050] In one implementation, combining the requirements of anti-interference and low computational complexity for voice wake-up decoding in embedded terminals with the CTC protocol encoding specification, introduce a target Token sequence construction mechanism and a forward probability accumulation logic of the state vector to determine the core calculation rules in the decoding stage. Combining the dual core requirements of voice wake-up decoding in embedded terminals - anti-interference (masking the interference of non-target Tokens and reducing false wake-up) and low computational complexity (adapting to low-power hardware on the end side), following the CTC protocol encoding specification (supporting non-strict alignment of Tokens and audio frames), introduce a target Token sequence construction mechanism and a forward probability accumulation logic of the state vector to determine the core calculation rules in the decoding stage.

[0051] The core logic focuses on "local state space calculation", abandoning the traditional full-scale quantifier table search mode, only carrying out probability calculations around the Tokens related to the target wake-up word, fundamentally reducing invalid operations, and at the same time achieving anti-interference through state transition constraints, ensuring that the computational complexity is constantly O(L) (L is the length of the target index sequence), adapting to the real-time and low-power requirements of embedded terminals.

[0052] Taking the voice wake-up decoding scenario of a low-power smart wearable device as an example, the device has a low-computing-power DSP built-in, and the computing workload during the decoding process needs to be controllable and resistant to environmental noise interference. Based on the device's hardware computing power (the limit for single-frame processing is 5 ms) and the CTC protocol specifications, the core computing rules are determined: only probability accumulation is performed on the Token sequence corresponding to the target wake-up word, and the state transition is strictly limited within the target index sequence. Non-target Tokens do not participate in any calculations, ensuring that the decoding time for each frame does not exceed 3 ms, while shielding the interference caused by environmental noise.

[0053] Extract the Token sequence corresponding to the target wake-up word, insert Blank placeholders between each Token according to the CTC protocol, construct a target index sequence with length L, and delimit the local state space for decoding. The Token sequence extraction is as follows: first, clarify the basic Token sequence corresponding to the target wake-up word (such as a pinyin or phoneme sequence), and keep the Token type consistent with the Token table of the previous CTC character prediction module to ensure the compatibility of probability extraction. The Blank placeholder insertion is as follows: according to the requirements of the CTC protocol, insert Blank placeholders before and after each Token in the basic Token sequence (for scenarios where the audio frame and Token are not strictly aligned), forming a mixed sequence containing target Tokens and Blanks.

[0054] Map the mixed sequence to a target index sequence (Target) with a fixed length L, where each element corresponds to a unique index in the Token table (including the target Token index and the Blank index). This sequence delimits the local state space for decoding - all subsequent calculations are only carried out around the Tokens corresponding to the indices in this sequence, independent of the full quantifier table.

[0055] Taking the wake-up word "Hello, Xiaochai" as an example, its corresponding pinyin Token sequence is [ni, hao, xiao, chan]. According to the CTC protocol, insert Blank placeholders (numbered 0) between each Token, obtaining the mixed sequence [0, ni, 0, hao, 0, xiao, 0, chan]; then map each element to an index in the Token table (assuming ni corresponds to 15, hao corresponds to 22, xiao corresponds to 31, and chan corresponds to 45), finally generating a target index sequence with length L = 8, [0, 15, 0, 22, 0, 31, 0, 45], delimiting the decoding to calculate only within the Token dimensions corresponding to these 8 indices, excluding the interference of other Tokens in the full quantifier table.

[0056] Based on the length L of the target index sequence, a one-dimensional state probability vector for the matching dimension is defined, clarifying the meaning of the cumulative token probability of each element in the vector corresponding to the target index sequence. Based on the length L of the target index sequence, a one-dimensional state probability vector S for perfect matching is defined, where each element in the vector... (j from 0 to L-1) has a clear physical meaning: it represents the cumulative probability of decoding and aligning to the j-th token in the target index sequence at the current moment. The dimension of the vector is strictly consistent with the length of the target index sequence, ensuring that each element corresponds one-to-one with the target token / Blank. No additional dimension is needed to store non-target information, further controlling memory usage and adapting to low memory resource scenarios on the edge.

[0057] Continuing with the target index sequence [0,15,0,22,0,31,0,45] of length L=8, we define a one-dimensional state probability vector S=[S0,S1,S2,S3,S4,S5,S6,S7]. Here, S0 corresponds to the cumulative probability aligned to the first Blank (index 0), S1 corresponds to the cumulative probability aligned to "ni" (index 15), S2 corresponds to the cumulative probability aligned to the second Blank (index 0), and so on, with S7 corresponding to the cumulative probability aligned to "chan" (index 45). The meaning of each element is clear and unique.

[0058] By combining the correlation constraints between keyword tokens, three types of closed-loop state transition rules are set: dwell, normal jump, and cross-jump jump, limiting the path range for probability accumulation. Based on the correlation between keyword tokens (temporal continuity and non-repetition characteristics), these three types of closed-loop state transition rules limit the update paths of elements in the state probability vector, ensuring that probability accumulation only occurs on reasonable paths within the target index sequence, isolating interference from non-target tokens. Current State The cumulative probability can inherit its own state from the previous moment. The probability corresponds to the scenario where the token is continuously pronounced (long tone or repeated frame).

[0059] Current status The cumulative probability can be inherited from the previous state at the previous time step. The probability corresponds to the scenario where the target index sequence jumps normally from the previous Token / Blank to the current Token / Blank; the jump rule (only for non-BlankTokens): if the current state If the corresponding token is not Blank and is different from the (j-2)th token in the target index sequence, then the current state can additionally inherit the previous two states. The probability corresponds to the scenario where, after crossing an intermediate Blank, the user directly jumps to the current non-BlankToken.

[0060] The three types of rules form a closed-loop path constraint, ensuring that state transitions occur only on reasonable paths within the target index sequence, with no additional invalid paths. (Based on the target index sequence...) Taking (corresponding to Token "hao", index 22) as an example, its state transition follows the following rules: The dwell rule is as follows, inheriting from the previous time step. The probability (the previous frame already aligned to "hao", the current frame still shows "hao"). The normal jump rule is as follows: inherit from the previous frame. The probability (aligning the Blank token before "hao" in the previous time step, jumping to "hao" in the current frame). The jump rule is as follows: since "hao" is a non-BlankToken and is different from the (j-2)th Token "ni" (index 15), it additionally inherits from the previous time step. The probability of (skipping the intermediate Blank to jump directly from "ni" to "hao"). Through these three types of rules, The probability comes only from the relevant states within the target index sequence and is independent of other irrelevant states.

[0061] Based on the forward probability accumulation algorithm, the target token-oriented extraction probability distribution information is substituted into the state transition rules for frame-by-frame calculation, completing the dynamic update of the state probability vector and generating a streaming dynamic programming decoding state vector and path transition information that is isolated from non-target token interference and revolves only around the target index sequence. The input of the forward probability accumulation is the target token-oriented extraction probability distribution information—that is, the probability value of the corresponding token extracted from the target index sequence from the full token probability distribution output by the CTC character prediction module (updated once per frame); based on the forward probability accumulation algorithm, the target token probability value of each frame is substituted into the above three types of state transition rules, and each element of the state probability vector S is updated frame-by-frame: for each S_j, the inheritable previous state is first determined according to the rules ( , Add if necessary The summation of the probabilities of the preceding states is multiplied by the probability value of the token in the current frame to obtain the current time step. The probability of [the event / determined outcome]. The entire calculation process involves only element-wise updates of a one-dimensional vector, without dynamic memory allocation or sorting / pruning, with a constant computational load. It focuses solely on the target index sequence, completely isolating interference from non-target tokens. The generated state vector and path transition information directly support subsequent wake-up determination.

[0062] Following on from S3 above, let's assume the probability of directional extraction of "hao" at time t is... =0.7, in the state vector of the previous time step =0.3、 =0.4、 =0.2. According to the transfer rules, ,Finish The update is performed. Similarly, each other element in the vector is calculated according to the corresponding rules to obtain the state vector S(t) at time t. This vector and the transition paths of each element are the decoded state vector and path transition information.

[0063] S106 is based on a streaming dynamic programming decoding mechanism without Top-K pruning. It combines long-term energy processing of attenuation factors and threshold triggering decision rules at the end of the state vector with terminal hardware state calculation to generate voice wake-up judgment information with low computational complexity, low false wake-up error, and high real-time performance. If the wake-up is successful, the subsequent voice interaction module is triggered.

[0064] In one implementation, combining the low latency and high accuracy decision requirements of embedded terminal voice wake-up with the pruning-free characteristics of streaming dynamic programming decoding, a long-term energy processing mechanism for attenuation factors and a threshold-triggered decision rule at the end of the state vector are introduced to determine the core execution logic of wake-up determination. Combining the core requirements of embedded terminal voice wake-up—low latency (fast response to wake-up commands) and high accuracy decision (reducing false and missed wake-ups)—the core execution logic of wake-up determination is clearly defined by fully utilizing the pruning-free characteristics of streaming dynamic programming decoding (constant computational load, no path loss risk).

[0065] The core logic focuses on "deterministic threshold judgment," abandoning the multi-path scoring mode of traditional BeamSearch. By monitoring the cumulative probability at the end of the state vector (corresponding to the probability of fully aligned wake words), combined with long-term energy stabilization processing and threshold triggering, it achieves low-latency and high-precision wake-up judgment, which is fully adapted to the real-time and reliability requirements of embedded terminals.

[0066] Based on a streaming dynamic programming decoding mechanism without Top-K pruning, and combined with the real-time hardware operation status of the embedded terminal, frame-by-frame wake-up probability calculation is performed on the decoded state vector and path transition information. This mechanism eliminates the need to maintain dynamic candidate paths, focusing solely on the previously generated decoded state vector and path transition information, ensuring a constant computational complexity of O(L) per frame (where L is the length of the target index sequence). The computational priority is dynamically adjusted based on the real-time hardware operation status of the embedded terminal (e.g., CPU utilization, memory bandwidth usage) to avoid wake-up delays due to excessive hardware resource consumption, ensuring the computation process is compatible with hardware capabilities. The decoded state vector and path transition information are read frame by frame. Based on the cumulative forward probability results, the probability values ​​of each element in the current frame's corresponding state vector are updated, with a focus on the real-time update of the final element of the state vector (the cumulative probability of fully aligned wake-up words), providing data support for subsequent decisions.

[0067] Continuing the target index sequence (length L=8) corresponding to the wake word "Hello Xiao Chan", under normal real-time hardware conditions (CPU utilization ≤30%, memory bandwidth utilization ≤50MB / s), probability calculations are performed frame-by-frame on the decoded state vector S=[S0,S1,...,S7]. Each frame only needs to update the probability values ​​of 8 elements according to the path transition rules. The calculation process involves no pruning or dynamic path adjustment, and the calculation time per frame is ≤2ms, which is fully adapted to the low computing power characteristics of embedded terminals.

[0068] To address the potential numerical drift of the probability at the end of the state vector in scenarios with no wake-up commands, a decay factor is introduced for frame-level long-term energy processing, ensuring numerical stability and static convergence over an infinite time axis. This involves fully aligning the accumulated probability value of the wake-up word at the end of the state vector and applying a decay factor. After each frame update, the probability value at the end is multiplied by the decay factor. This ensures that, over an infinite time axis, the probability value gradually converges to a stable range when no wake-up word is detected, avoiding misjudgments caused by numerical overflow or invalid accumulation, thus guaranteeing numerical stability and static convergence. An example decay factor value of 0.998 is used, which effectively suppresses numerical drift in long-term wake-up-free scenarios without excessively attenuating the probability value of valid wake-up signals, balancing stability and sensitivity.

[0069] Assume that the final element S7 of the state vector at time t (the cumulative probability of aligning with "chan") is 0.5, and the decay factor is set to 0.998. If no token related to the wake-up word is detected in frame t+1, S7 is updated and multiplied by the decay factor, resulting in 0.5 × 0.998 = 0.499. In subsequent frames where no valid signal is detected, S7 decays frame by frame according to this logic, gradually converging to a stable value close to 0, avoiding abnormal values ​​due to prolonged periods without wake-up. If a valid signal is detected, the new cumulative probability value will overwrite the decayed value, ensuring that the wake-up signal is not missed.

[0070] Based on the application scenario requirements of the embedded terminal, a wake-up determination threshold is preset, and a deterministic decision rule is set to trigger the wake-up when the cumulative probability at the end of the state vector exceeds the threshold. The wake-up determination threshold is preset according to the application scenario requirements of the embedded terminal (such as noisy environments and wake-up sensitivity requirements). An example threshold is -20 (logarithmic probability form). The threshold can be flexibly adjusted according to the actual scenario—in noisy environments, the threshold can be appropriately increased (e.g., -18) to improve anti-interference capabilities, while in quiet environments, the threshold can be decreased (e.g., -22) to improve wake-up sensitivity. Setting a deterministic decision rule of "triggered when the cumulative probability at the end of the state vector exceeds the threshold" eliminates the need for confidence scoring of multiple paths; it only requires real-time monitoring of the probability value of the element at the end of the state vector. When this value exceeds the preset threshold, wake-up is immediately determined to be successful; otherwise, monitoring continues, ensuring a simple and low-latency decision logic.

[0071] The cumulative probability at the end of the processed state vector is determined in real time using decision rules, generating high-real-time voice wake-up determination information with a constant computational complexity of O(L) and low false wake-up errors. The decision rules determine the cumulative probability at the end of the state vector after attenuation factor processing in real time: if the end probability value exceeds a preset threshold, a "wake-up successful" determination is generated, which includes key data such as the wake-up trigger time and target wake-up word matching degree. The computational complexity remains constant at O(L), ensuring high real-time performance. If the end probability value does not reach the threshold, a "not woken up" determination is generated, and the process returns to the frame-by-frame calculation stage to continuously monitor probability changes. The core characteristics of the determination information are: low false wake-up errors (no path loss due to pruning, threshold triggering avoids misjudgment), high real-time performance (single frame determination time ≤ 1ms), and constant computational load (adapting to low computing power on the device side).

[0072] If the wake-up result is successful, a wake-up success signal is immediately output and the subsequent voice interaction module is triggered, completing the entire process of embedded terminal streaming voice wake-up. If the wake-up result is successful, a standardized wake-up success signal (such as a high-level signal or a specific protocol command) is immediately output, with a signal transmission delay of ≤1ms to ensure rapid response; the wake-up success signal directly triggers the subsequent voice interaction module (such as a speech recognition or command execution module), initiating the voice interaction process and completing the entire closed loop of "audio input - feature extraction - encoding - prediction - decoding - judgment - interaction"; if the wake-up result is not successful, the system remains in standby mode, continues to receive audio input and repeats the above process to ensure continuous monitoring of wake-up commands.

[0073] As shown in Figure 2, a low-latency streaming voice wake-up device for embedded terminals includes: a requirement architecture adaptation module 201, used to obtain the core application requirements of low power consumption and high real-time performance for voice wake-up of embedded terminals, and determine the GRU backbone model architecture adapted to the requirements based on the characteristics of terminal hardware resources; a time-domain feature lightweight processing module 202, used to receive the continuously input raw audio sampling point sequence, divide it according to the preset frame length and reconstruct it into a two-dimensional vector, and project the time-domain information to the high-dimensional feature space through a fully connected linear layer to generate time-domain feature reconstruction and linear mapping information without FFT floating-point operation and cache overhead; a GRU streaming coding module 203, used to build a multi-layer GRU fully causal streaming coding network, initialize the hidden state vector to all 0, and update the inference of each frame only based on the feature vector of the current frame and the hidden state of the previous time step to generate GRU hidden state dynamic update information without sliding window cache and memory bandwidth occupation of O(1); and a CTC character prediction module 204, used to receive time-domain feature reconstruction and linear mapping information and GRU hidden state dynamic update information, and project the time-domain information to the high-dimensional feature space through a fully connected linear layer to generate time-domain feature reconstruction and linear mapping information and GRU hidden state dynamic update information through a CTC module 204. The character prediction model trained by CLoss outputs the probability distribution of all tokens including Blank placeholders, extracts the probability of the target wake word token, and generates CTC character prediction and target token extraction probability distribution information. The decoding path construction module 205 is used to construct a target index sequence of length L containing Blank based on the target wake word token sequence and CTC protocol, defines a one-dimensional state probability vector with matching dimension, sets three types of closed-loop state transition constraint rules: pause, normal jump, and cross jump, and generates a decoded state vector and path transition information isolated from non-target token interference through forward probability accumulation calculation. The wake-up judgment trigger module 206 is used to generate voice wake-up judgment information with constant computational complexity of O(L), low false wake-up error, and high real-time performance based on a streaming dynamic programming decoding mechanism without Top-K pruning, combined with long-term energy processing of attenuation factor and threshold triggering judgment rule at the end of state vector, and correlated with the real-time hardware status of embedded terminal. If the wake-up is successfully judged, the subsequent voice interaction module is triggered.

[0074] A computing device includes a memory for storing computer program instructions and a processor for executing the computer program instructions, wherein when the computer program instructions are executed by the processor, the device is triggered to execute any low-latency streaming voice wake-up method for an embedded terminal.

[0075] The methods and / or embodiments in this application can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. When the computer program is executed by a processing unit, it performs the functions defined in the methods of this application.

[0076] It should be noted that the computer-readable medium described in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0077] It will be apparent to those skilled in the art that this application is not limited to the details of the exemplary embodiments described above, and that this application can be implemented in other specific forms without departing from the spirit or essential characteristics of this application.

Claims

1. A low-latency streaming voice wake-up method for embedded terminals, characterized in that, include: To obtain the core application requirements of low power consumption and high real-time performance for voice wake-up in embedded terminals, and to determine the GRU backbone model architecture that adapts to these requirements. The original audio sampling point sequence of continuous input is preprocessed in the temporal domain with lightweight quantization. It is then segmented and reconstructed into a two-dimensional vector according to the preset frame length. The vector is then projected into a high-dimensional feature space through a fully connected linear layer to generate temporal feature reconstruction and linear mapping information without FFT floating-point operation and cache overhead. A multi-layer GRU fully causal streaming coding network is built. The hidden state vector is initialized to all zeros. The inference of each frame is updated only according to the feature vector of the current frame and the hidden state of the previous time step to generate GRU hidden state dynamic update information without sliding window cache and with memory bandwidth of O(1). The above two types of information are input into the character prediction module trained by CTCLoss, and the full token probability distribution containing Blank placeholders is output. The target wake word token probability is extracted in a targeted manner to generate CTC character prediction and target token targeted extraction probability distribution information. Based on the target wake word token sequence and CTC protocol, a target index sequence of length L containing Blank is constructed. A matching one-dimensional state probability vector is defined, and three types of closed-loop state transition constraint rules are set. After forward probability accumulation calculation, a decoding state vector and path transition information that isolate non-target token interference are generated. Based on a streaming dynamic programming decoding mechanism without Top-K pruning, combined with long-term energy processing of attenuation factors and threshold triggering decision rules at the end of the state vector, and combined with terminal hardware state calculation, voice wake-up determination information with low computational complexity, low false wake-up error, and high real-time performance is generated. If the wake-up is successful, the subsequent voice interaction module is triggered.

2. The low-latency streaming voice wake-up method for embedded terminals according to claim 1, characterized in that, The process involves performing time-domain lightweight preprocessing on the continuously input raw audio sampling point sequence, segmenting it according to a preset frame length and reconstructing it into a two-dimensional vector, then projecting it onto a high-dimensional feature space through a fully connected linear layer. This generates time-domain feature reconstruction and linear mapping information without FFT floating-point operations or cache overhead. The process includes: combining the low-power voice wake-up requirements of embedded terminals with the original audio time-domain processing logic, introducing audio frame length segmentation rules and a time-domain feature linear mapping mechanism to perform time-domain lightweight preprocessing on the continuously input raw audio sampling point sequence; segmenting the raw audio sampling point sequence according to a preset frame length, reconstructing the one-dimensional audio sampling point sequence into a two-dimensional vector, completing the dimensional reconstruction of the audio time-domain features; projecting the time-domain information of the two-dimensional vector onto the high-dimensional feature space of the model's hidden layer through a fully connected linear layer, achieving linear mapping transformation of the time-domain features; and generating time-domain feature reconstruction and linear mapping information that eliminates FFT floating-point operation overhead and has no additional cache overhead, thus completing the front-end feature extraction for embedded terminal voice wake-up.

3. The low-latency streaming voice wake-up method for embedded terminals according to claim 1, characterized in that, A multi-layer GRU fully causal streaming coding network was constructed, and the hidden state vector was initialized to all zeros. Each frame inference was updated only based on the feature vector of the current frame and the hidden state of the previous time step, generating GRU hidden state dynamic update information with no sliding window buffer and O(1) memory bandwidth usage. This included: combining the low memory and high real-time requirements of embedded terminal voice wake-up streaming inference with the natural recursive characteristics of GRU network hidden states, introducing multi-layer GRU fully causal coding rules and frame-level hidden state update mechanism, and constructing a multi-layer GRU fully causal streaming coding network adapted to the terminal side; according to the initialization specification of streaming inference, the multi-layer GRU fully causal streaming coding network was... The hidden state vector of the network is initialized to an all-zero vector to complete the state preparation before network inference; according to the frame-level processing logic of GRU fully causal streaming inference, only the temporal feature vector of the current frame and the network hidden state of the previous time step are received as input during the inference process of each frame; the input feature vector and the hidden state of the previous time step are calculated and processed through the GRU network gating mechanism to complete the dynamic update of the GRU network hidden state at the current time step, and generate GRU hidden state dynamic update information that does not require maintenance of convolution sliding window cache and reduces memory bandwidth occupation to O(1) level, so as to realize the low memory overhead of speech feature streaming encoding in embedded terminals.

4. The low-latency streaming voice wake-up method for embedded terminals according to claim 1, characterized in that, The above two types of information are input into the character prediction module trained by CTCLoss, which outputs the probability distribution of all tokens including Blank placeholders, and extracts the probability of the target wake-up word token in a targeted manner. This generates CTC character prediction and target token targeted extraction probability distribution information, including: combining the feature adaptation characteristics of embedded terminal voice wake-up character sequence prediction requirements and CTC sequence modeling, introducing CTCLoss training rules and token probability distribution output mechanism, and building and training a CTC character prediction module adapted to the terminal side; taking temporal feature recombination and linear mapping information and GRU hidden state dynamic update information as input, and feeding them into the character prediction module trained by CTCLoss for feature calculation; relying on the character sequence probability prediction capability of the module, outputting the probability distribution of all tokens at the current time including Blank placeholders and Chinese pinyin, and completing the probability prediction of all tokens; based on the token sequence corresponding to the target wake-up word, extracting the probability value of the target token in a targeted manner from the probability distribution of all tokens, shielding the probability calculation and extraction of non-target tokens, generating CTC character prediction information and token targeted extraction probability distribution information focusing only on the target wake-up word, providing accurate target token probability data for the subsequent decoding stage.

5. The low-latency streaming voice wake-up method for embedded terminals according to claim 4, characterized in that, Based on the target wake-up word token sequence and the CTC protocol, a target index sequence of length L containing Blank is constructed. A one-dimensional state probability vector for matching is defined, and three types of closed-loop state transition constraint rules are set. Through forward probability accumulation calculation, a decoded state vector and path transition information isolated from non-target token interference are generated. This includes: combining the requirements of anti-interference and low computational load of embedded terminal voice wake-up decoding with the CTC protocol encoding specifications, introducing a target token sequence construction mechanism and forward probability accumulation logic of the state vector, and determining the core calculation rules of the decoding stage; extracting the token sequence corresponding to the target wake-up word, inserting Blank placeholders between each token according to the CTC protocol, and constructing a target index sequence of length L. For the target index sequence L, define the local state space for decoding; based on the length L of the target index sequence, define a one-dimensional state probability vector for the matching dimension, clarifying the meaning of the token accumulation probability of each element in the vector corresponding to the target index sequence; combined with the correlation constraints between keyword tokens, set three types of closed-loop state transition rules: stay, normal jump, and cross-jump jump, limiting the path range of probability accumulation; based on the forward probability accumulation algorithm, extract the probability distribution information of the target token and substitute it into the state transition rules for frame-by-frame calculation, completing the dynamic update of the state probability vector, generating a streaming dynamic programming decoding state vector and path transition information that isolates non-target token interference and revolves only around the target index sequence.

6. The low-latency streaming voice wake-up method for embedded terminals according to claim 1, characterized in that, Based on a streaming dynamic programming decoding mechanism without Top-K pruning, and combining long-term energy processing of the attenuation factor and threshold triggering rules at the end of the state vector, along with terminal hardware state calculations, this mechanism generates voice wake-up determination information with low computational complexity, low false alarms, and high real-time performance. Successful wake-up triggers subsequent voice interaction modules, including: combining the low-latency, high-accuracy voice wake-up determination requirements of embedded terminals with the pruning-free characteristics of streaming dynamic programming decoding, introducing a long-term energy processing mechanism of the attenuation factor and threshold triggering rules at the end of the state vector to determine the core execution logic of wake-up determination; and based on the streaming dynamic programming decoding mechanism without Top-K pruning, and combined with the real-time hardware operating status of the embedded terminal, processing the decoded state vector and path transformation... The system performs frame-by-frame wake-up probability calculations on the information; it fully aligns the cumulative probability values ​​of the wake-up words at the end of the state vector, introduces a decay factor for frame-level long-term energy processing, and ensures numerical stability and static convergence on the infinite time axis; it presets a wake-up judgment threshold according to the application scenario requirements of the embedded terminal, and sets a deterministic decision rule that triggers when the cumulative probability at the end of the state vector exceeds the threshold; it performs real-time judgment on the cumulative probability at the end of the processed state vector through the decision rule, and generates high-real-time voice wake-up judgment information with a constant computational complexity of O(L) and low false wake-up errors; if the judgment result is a successful wake-up, it immediately outputs a wake-up success signal and triggers the subsequent voice interaction module, completing the entire process of streaming voice wake-up of the embedded terminal.

7. A low-latency streaming voice wake-up device for embedded terminals, characterized in that, The device is used to execute executable instructions to perform the low-latency streaming voice wake-up method for embedded terminals as described in any one of claims 1 to 6.

8. An electronic device, characterized in that, include: First processor; And a memory for storing executable instructions of the first processor; wherein the first processor is configured to execute the low-latency streaming voice wake-up method for embedded terminals according to any one of claims 1 to 6 by executing the executable instructions.

9. A computing device, the device comprising a memory for storing computer program instructions and a processor for executing the computer program instructions, wherein, When the computer program instructions are executed by the processor, the device is triggered to execute the low-latency streaming voice wake-up method for embedded terminals as described in any one of claims 1 to 6.

Images