Method and device for acquiring wake-up keyword and embedded equipment

Abstract

The invention relates to a method and device for obtaining a wake-up keyword and embedded equipment, and the method comprises the steps: carrying out the framing of audio data, extracting audio features, and obtaining an audio feature sequence; inputting a CNN encoder to obtain an encoding feature sequence; performing compact attention processing on the coded feature sequence, wherein local causal attention calculation is performed on the feature sequence to obtain a local attention weight sequence; channels contained in the coding feature sequence are grouped, and according to the number of channels contained in each feature channel group, copying of the number of channels is carried out on the local attention weight sequence in the channel dimension, and an attention weight tensor sequence is obtained; the coding feature sequence is remodeled, a remodeled feature sequence is obtained, the product of the remodeled feature sequence and the attention weight tensor sequence is calculated, the product is processed through a full-connection classifier, the probability of the wake-up keyword is output, and the contained wake-up keyword is obtained based on the probability. The wake-up accuracy and the environment robustness can be improved.

Description

Technical Field

[0001] This invention relates to the fields of speech signal processing and artificial intelligence technology, and in particular to a method, apparatus and embedded device for acquiring wake-up keywords. Background Technology

[0002] Keyword Spotting (KWS) is characterized by low power consumption, always-on availability, and millisecond-level response. By embedding wake-up keywords into the keyword spotting model of a smart terminal, the model identifies wake-up keywords based on user-input voice data. Upon confirming the presence of a wake-up keyword, it triggers the wake-up function, achieving contactless, low-power, and instantaneous voice interaction. It is widely used in scenarios requiring immediate triggering, such as smart homes, in-car navigation, smart wearables, and medical monitoring. However, related keyword spotting models, such as those based on Convolutional Neural Networks (CNNs), extract local features from voice data through convolutional layers. However, a single convolutional kernel can only capture information within a limited time window. Therefore, to understand a complete wake-up keyword (e.g., "Xiao Ai Tongxue"), the keyword spotting model needs to stack multiple convolutional layers to obtain a sufficient "receptive field," leading to the following problems: First, the dependence on long-distance relationships makes the modeling ability weak, making it difficult to capture the association between two syllables that are far apart in time, resulting in low recognition accuracy of wake-up keywords. Second, the wake-up keyword recognition method that uses stacked multi-layer convolution may introduce irrelevant noise. Since the keyword wake-up model processes the signal at all time steps equally, it cannot avoid the influence of background noise or irrelevant speech, resulting in low accuracy of the recognized wake-up keywords. Summary of the Invention

[0003] In view of this, the present invention provides a method, apparatus and embedded device for obtaining wake-up keywords.

[0004] Specifically, the present invention is achieved through the following technical solution: According to a first aspect of the present invention, a method for obtaining wake-up keywords is provided, applicable to embedded devices, the method comprising: The user's audio data is segmented into frames, and the audio features of each audio data frame are extracted to obtain an audio feature sequence. The acquired audio feature sequence is input into a pre-built CNN encoder to obtain the encoded feature sequence; Compact attention processing is performed on the encoded feature sequence, wherein: local causal attention calculation is performed on the input feature sequence to obtain a normalized local attention weight sequence; the channels contained in the encoded feature sequence are grouped to obtain multiple feature channel groups; based on the number of channels contained in each feature channel group, the normalized local attention weight sequence is copied in the channel dimension to obtain the number of channels, thereby obtaining an attention weight tensor sequence; the encoded feature sequence is reshaped by performing matrix operations matching the attention weight tensor sequence to obtain a reshaped feature sequence; the product of the reshaped feature sequence and the attention weight tensor sequence is obtained to obtain a compact attention processing feature sequence. Based on the features processed by compact attention, a fully connected classifier is used to process and output the probability of whether it is a wake-up keyword. Based on the probability of whether it is a wake-up word, the wake-up keyword contained in the audio data is obtained.

[0005] Optionally, the step of performing local causal attention calculation on the input feature sequence to obtain a normalized local attention weight sequence includes: wherein, using a pre-set one-dimensional convolution kernel, performing causal convolution on the input feature sequence to obtain unnormalized probability weights, and obtaining a normalized local attention weight sequence based on the unnormalized probability weights and the classification normalization function.

[0006] Optionally: the compact attention processing on the encoded feature sequence further includes: The encoded feature sequence is projected onto a linear layer to obtain a compressed feature sequence with reduced feature dimensions; the compressed feature sequence is then used as the input feature sequence for the local causal attention calculation. The compact attention processing feature sequence is transformed into a transformed feature sequence with the same dimension as the encoded feature sequence. The transformed feature sequence and the encoded feature sequence are then joined by a residual connection to obtain the reconstructed feature sequence. The features based on compact attention processing are used with a fully connected classifier to output the probability of whether it is a wake-up keyword, including: The reconstructed feature sequence is used as the feature after compact attention processing. After global average pooling, it is input into a pre-built fully connected classifier to output the probability of whether it is a wake word.

[0007] Optionally, the step of performing causal convolution on the input feature sequence using a pre-set one-dimensional convolution kernel to obtain unnormalized probability weights includes: Using the one-dimensional convolution kernel, for the audio data frame corresponding to the t-th time step in the input feature sequence, calculate the causal convolution of the t-th audio data frame with all audio data frames in the range [t, t+W] to obtain the unnormalized probability weight corresponding to the t-th audio data frame, where W is the number of audio data frames in the time window.

[0008] Optionally, all audio data frames within the range [t, t+W] include: W audio data frames centered on the t-th audio data frame on the time axis; or, All audio data frames within the range [t, t+W] include: W audio data frames on the time axis, with the t-th audio data frame as the starting audio data frame.

[0009] Optionally, the kernel size of the one-dimensional convolution kernel is the same as the compression ratio of the linear layer projection.

[0010] Optionally, the dimension of the encoded feature sequence is 128; the compression ratio is 8. The one-dimensional convolution kernel has a kernel size of 8, an output channel number of 1, and a stride of 1. The channels contained in the encoded feature sequence are grouped to obtain 8 feature channel groups. Based on the number of channels (16) contained in each feature channel group, the normalized local attention weight sequence is copied in the channel dimension to obtain the attention weight tensor sequence. The classification normalization function is the Softmax function.

[0011] Optionally, the step of performing residual concatenation on the transformed feature sequence and the encoded feature sequence to obtain the reconstructed feature sequence includes: The reconstructed feature sequence is obtained by summing the transformed feature sequence and the encoded feature sequence.

[0012] The method for obtaining wake-up keywords in this technical solution involves: segmenting the user's audio data into frames, extracting audio features from each audio data frame to obtain an audio feature sequence; inputting the obtained audio feature sequence into a pre-constructed CNN encoder to obtain an encoded feature sequence; performing compact attention processing on the encoded feature sequence, wherein: local causal attention calculation is performed on the input feature sequence to obtain a normalized local attention weight sequence; the channels contained in the encoded feature sequence are grouped to obtain multiple feature channel groups, and the normalized local attention weight sequence is copied along the channel dimension according to the number of channels contained in each feature channel group to obtain an attention weight tensor sequence; the encoded feature sequence is reshaped by performing matrix operations matching the attention weight tensor sequence to obtain a reshaped feature sequence, and the product of the reshaped feature sequence and the attention weight tensor sequence is obtained to obtain a compact attention processed feature sequence; based on the features after compact attention processing, a fully connected classifier is used to process and output the probability of whether it is a wake-up keyword, and the wake-up keyword contained in the audio data is obtained based on the probability of whether it is a wake-up word. In this way, after the CNN encoder and before the classifier, a novel compact attention processing module with a group attention processing mechanism can be used to introduce the global perception and dynamic focusing capabilities of the attention mechanism while maintaining extremely low computational overhead in embedded devices. This allows for better attention to key phonemes or syllables in the wake word, thereby significantly improving wake-up accuracy and environmental robustness.

[0013] According to a second aspect of the present invention, an apparatus for acquiring wake-up keywords is provided, the apparatus comprising: The data framing module is used to segment the user's audio data into frames, extract the audio features of each audio data frame, and obtain the audio feature sequence. The encoding module is used to input the acquired audio feature sequence into a pre-built CNN encoder to obtain the encoded feature sequence; An attention processing module is used to perform compact attention processing on the encoded feature sequence, wherein: local causal attention calculation is performed on the input feature sequence to obtain a normalized local attention weight sequence; the channels contained in the encoded feature sequence are grouped to obtain multiple feature channel groups; based on the number of channels contained in each feature channel group, the normalized local attention weight sequence is copied in the channel dimension to obtain the number of channels, thereby obtaining an attention weight tensor sequence; the encoded feature sequence is reshaped by performing matrix operations matching the attention weight tensor sequence to obtain a reshaped feature sequence; and the product of the reshaped feature sequence and the attention weight tensor sequence is obtained to obtain a compact attention processing feature sequence. The probability calculation module is used to process the features after compact attention processing using a fully connected classifier and output the probability of whether it is a wake-up keyword. The wake-up keyword determination module is used to obtain the wake-up keyword contained in the audio data based on the probability of whether it is a wake-up word.

[0014] According to a third aspect of the present invention, an embedded device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method for acquiring a wake-up keyword in any possible implementation of the first aspect. Attached Figure Description

[0015] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating a method for obtaining wake-up keywords according to an embodiment of the present invention; Figure 2 A schematic diagram of a device for acquiring wake-up keywords provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an embedded device provided in an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] The keyword wake-up model based on convolutional neural network architecture extracts local features from the speech data by stacking multiple convolutional layers when recognizing wake-up keywords from user-input speech data. The extracted local features are then input into a classifier for wake-up keyword recognition. However, the method of wake-up based on the recognized wake-up keywords suffers from the fact that the convolutional kernels of the convolutional layers can only acquire information within a limited time window. This results in long-distance dependencies among the multiple convolutional layers, making it difficult to capture the correlation between two syllables that are far apart in time. Furthermore, it may introduce irrelevant noise, leading to low accuracy in recognizing wake-up keywords.

[0020] To improve the recognition accuracy of wake-up keywords in keyword wake-up models, related technologies can utilize the self-attention mechanism in Transformer to solve long-distance dependency problems. Although this can improve the recognition accuracy of wake-up keywords, the computational complexity of the self-attention mechanism is too high (O(T²), where T is the sequence length of the input keyword wake-up model). For wake-up tasks that need to remain always on and have low power consumption, the required power consumption is relatively large. Therefore, directly adding the self-attention mechanism to the keyword wake-up model is not suitable for embedded device application scenarios with extremely limited computing resources and high low power consumption requirements.

[0021] The term "embedded system" as used in this article refers to a dedicated computing system embedded within an object's architecture. Its core characteristic is that it is "designed specifically for a particular task," and it typically has strict requirements for specialization, resource constraints, real-time performance, high reliability and stability, low power consumption, and direct hardware interaction. Its basic components include hardware and software. The hardware is usually centered around a microcontroller (MCU) or microprocessor (MPU), while the software is primarily firmware, i.e., a program written to read-only memory. "Embedded device" refers to a physical device or product that integrates an embedded system. It is generally based on an ARM core and architecture or other low-power cores and architectures, and is used to implement specific functions and applications. It is a device relative to a general-purpose PC with multiple functions. Specific examples include headphones, speakers, smartwatches, smart locks, in-vehicle navigation devices, in-vehicle entertainment devices, smart home appliances, wearable medical monitors (such as blood glucose meters and blood pressure monitors), etc.

[0022] In this embodiment, based on a keyword wake-up model using a CNN framework, a compact attention module designed in this embodiment is inserted after the CNN encoder and before the classifier. This introduces the intelligent focusing capability of the attention mechanism with minimal additional computational resources, improving the performance of the keyword wake-up model with only a slight increase in computational complexity, thus achieving a balance between performance, complexity, and efficiency. This makes it widely applicable to various specific application scenarios in embedded devices and significantly improves the wake-up accuracy and environmental robustness of embedded devices.

[0023] See Figure 1 This invention provides a method for obtaining wake-up keywords, applicable to embedded devices, which may include the following steps: S101. Divide the user's audio data into frames, extract the audio features of each audio data frame, and obtain the audio feature sequence. In this embodiment, as an optional implementation, the audio features include, but are not limited to, Mel-frequency cepstral coefficients (MFCC) features. The audio data includes, but is not limited to, 16kHz mono audio.

[0024] In this embodiment, as an optional embodiment, the frame length of the audio data frame is 32ms.

[0025] In this embodiment, as an optional implementation, the user's audio data is segmented into frames, including: From the beginning position of the audio data, audio data of a preset frame length is extracted to obtain the first audio data frame; Starting from the first position of the audio data corresponding to the first position of the first audio data frame, backtrack the audio data for a preset frame shift time in the direction of the beginning position of the audio data to obtain the starting position of the second audio data frame; Starting from the beginning of the second audio data frame, audio data of a certain duration of the frame is extracted to obtain the second audio data frame, until the audio data is completely segmented into frames.

[0026] In this embodiment, as an optional implementation, the frame length is set to 32ms and the frame shift time is set to 16ms. By retaining a portion of overlapping audio data between audio data frames, the frame shift of the overlapping audio data between adjacent audio data frames is set to 16ms. In this way, a local fixed window is formed using the frame length of the overlapping audio data, effectively reducing computational complexity.

[0027] In this embodiment, the audio data is preprocessed: 16kHz mono audio data is received, and the received mono audio data is divided into frames with a frame length of 32ms and a frame shift of 16ms. For each audio data frame, 12-dimensional MFCC features are extracted. The MFCC features corresponding to each audio data frame contained in the audio data constitute an audio feature sequence, and 12 dimensions represent the number of MFCC features contained in the audio data.

[0028] S102. Input the acquired audio feature sequence into the pre-built CNN encoder to obtain the encoded feature sequence; In this embodiment, audio feature encoding is performed. A lightweight CNN-based model can be used as the CNN encoder to adapt to embedded devices.

[0029] As an optional embodiment, the CNN encoder employs the TC-ResNet model. The audio feature sequence, composed of MFCC features corresponding to each audio data frame, is input into an 8-layer TC-ResNet sub-model. Taking an audio data set containing 16 audio frames as an example, the output is a encoded feature sequence X with dimensions [T, 128], where T is the time step, i.e., the number of audio data frames. In this embodiment, T=16, and 128 is the number of channels in the output encoded feature sequence.

[0030] S103. Perform compact attention processing on the encoded feature sequence, wherein: local causal attention calculation is performed on the input feature sequence to obtain a normalized local attention weight sequence; the channels contained in the encoded feature sequence are grouped to obtain multiple feature channel groups; based on the number of channels contained in each feature channel group, the normalized local attention weight sequence is copied in the channel dimension to obtain the number of channels, thereby obtaining an attention weight tensor sequence; the encoded feature sequence is reshaped to match the matrix operation performed on the attention weight tensor sequence to obtain a reshaped feature sequence; the product of the reshaped feature sequence and the attention weight tensor sequence is obtained to obtain a compact attention processing feature sequence. In this embodiment, compact attention processing is applied to the encoded feature sequence to obtain a compact attention-processed feature sequence. Specifically, compact attention processing is used to reconstruct the encoded feature sequence based on local causal attention and grouped channel attention.

[0031] In this embodiment, considering that for most four-character wake-up keywords, a window covering 200-300 milliseconds is sufficient to capture the key inter-syllable context, it is not necessary to focus on the entire sequence at each time point according to the standard attention mechanism. Instead, it is only necessary to focus on the small window adjacent to it, which reduces the computational complexity from O(T²) to O(T*W), where T is the length of the audio feature sequence and W is the size of the small window adjacent to it. Since W is a constant much smaller than the sequence length, the computational complexity can be made linear O(T). Thus, with a small amount of additional computation, long-distance dependencies are avoided and background noise or irrelevant speech is effectively filtered out.

[0032] In this embodiment, as an optional implementation, local causal attention calculation is performed on the input feature sequence to obtain a normalized local attention weight sequence. This includes: using a pre-set one-dimensional convolution kernel to perform causal convolution on the input feature sequence to obtain unnormalized probability weights; and based on the unnormalized probability weights and a classification normalization function, obtaining a normalized local attention weight sequence. In this embodiment, the compact attention module performs local causal attention through a one-dimensional causal convolution layer. A fixed window size W is set (e.g., W = 12 or 15 time frames). For the t-th frame in the sequence, only its attention score is calculated relative to all frames within the range [t, t+W]. A one-dimensional causal convolution layer is used to efficiently generate the attention weights for this local window.

[0033] In this embodiment, as an optional implementation, a bottleneck structure can be introduced to further compress parameters and stabilize training before performing local causal attention calculation, specifically including: A11, perform linear layer projection on the encoded feature sequence to obtain a compressed feature sequence with reduced feature dimension; In this embodiment, before entering the local causality calculation, the input feature sequence is first passed through a bottleneck compression layer (a linear layer of C -> C / R, where R is the compression ratio, the value of which needs to be set, typically 4 or 8) to reduce the feature dimension. As an optional embodiment, the compression ratio of the linear layer projection is set to 8, that is, the 128-dimensional encoded feature sequence X[T, 128] is projected to 16 dimensions through the linear layer, thereby compressing the bottleneck dimension of the input feature (encoded feature sequence) before attention calculation, resulting in a compressed feature sequence [T, 16].

[0034] A12, the compressed feature sequence is used as the input feature sequence for the local causal attention calculation, so that the local causal calculation process can use a pre-set one-dimensional convolution kernel to perform causal convolution on the compressed feature sequence to obtain unnormalized probability weights.

[0035] In this embodiment, the one-dimensional convolution kernel performs causal convolution on the input compressed feature sequence using adjacent small windows (local fixed windows). As an optional embodiment, the local fixed window size W contains 8 audio data frames, i.e., W=8. Thus, for the audio data frame corresponding to the t-th time step in the audio feature sequence, the causal convolution of the t-th audio data frame with all audio data frames in the range [t, t+W] is calculated to obtain the unnormalized probability weight corresponding to the t-th audio data frame. As an optional embodiment, all audio data frames in the range [t, t+W] include W audio data frames centered on the t-th audio data frame. As another optional embodiment, all audio data frames in the range [t, t+W] include W audio data frames on the time axis starting with the t-th audio data frame. This embodiment does not limit the number of audio data frames in the range [t, t+W].

[0036] As an alternative embodiment, the kernel size of the one-dimensional convolution kernel is the same as the compression ratio of the linear layer projection.

[0037] For example, when the compression ratio of the linear layer projection is set to 8, the kernel size of the one-dimensional convolution kernel is configured to be 8, the number of output channels is 1, and the stride is 1.

[0038] As an optional embodiment, the classification normalization function can be configured as the Softmax function.

[0039] In this embodiment, a causal convolution with left-padding zeros is performed on the compressed feature sequence [T, 16] to obtain unnormalized probability weights (logits) of [T, 1, 8]. A Softmax function is applied to the last dimension to obtain normalized local attention weights AR^(T×8). Then, a convolutional layer with kernel size M and 1 output channel is followed by the Softmax function to directly generate normalized local attention weights for the corresponding local fixed window [t: t+W] for each time step t corresponding to the t-th audio data frame (time frame). That is, the normalized attention weights A_t ∈ R^W, and the normalized attention weights A_t corresponding to each audio data frame constitute the local attention weight sequence A. In this way, the parallel computation characteristics of convolution can be utilized to generate corresponding local attention weights for all time steps at once, which is far more efficient than iterative computation.

[0040] In the compact attention processing of this embodiment, a grouped attention processing mechanism is further adopted. The feature channels are grouped, and attention weights calculated in the time dimension are applied independently in each group. Without significantly increasing the number of parameters, the attention mechanism is differentiated in the channel dimension, which enhances the expressive power of the model.

[0041] Specifically, the channels contained in the encoded feature sequence are grouped to obtain multiple feature channel groups. Based on the number of channels contained in each feature channel group, the normalized local attention weight sequence is copied along the channel dimension to obtain the number of channels, thus obtaining the attention weight tensor sequence. The encoded feature sequence is then reshaped by performing matrix operations on the attention weight tensor sequence to obtain the reshaped feature sequence. Finally, the product of the reshaped feature sequence and the attention weight tensor sequence is obtained to obtain the compact attention processing feature sequence.

[0042] As a specific implementation: For the encoded feature sequence X with feature dimensions [T, C], where T is the time step and C is the number of channels, the C channels are divided into G feature channel groups, each containing C / G channels. The local attention weight sequence A ∈ R^(T×W) calculated by the local causal attention module is copied C / G times along the channel dimension to form G independent attention weight tensors. The weight tensor of each feature channel group is: X_g ∈ R^(T×(C / G)). This is multiplied by the corresponding attention weight tensor sequence to achieve the effect of group weighting.

[0043] In this embodiment, a grouping weighting operation is performed. As an optional embodiment, the encoded feature sequence X contains 128 channels, and the number of feature channel groups is 8. Specifically, the original 128-dimensional channels contained in the encoded feature sequence are divided into 8 feature channel groups (G=8), and each feature channel group includes 16 channels. By copying the normalized local attention weight sequence A 16 times in the channel dimension, the attention weight tensor corresponding to the feature channel group is formed, and the attention weight tensors corresponding to each feature channel group constitute the attention weight tensor sequence [T, 16, 8].

[0044] In this embodiment, the input features, namely the encoded feature sequence X [T, 128], are reshaped so that they can be used for matrix operations with the attention weight tensor sequence. For example, if the attention weight tensor sequence is [T, 16, 8], then after reshaping the encoded feature sequence, the reshaped feature sequence is [T, 8, 16]. Then, the reshaped feature sequence is multiplied by the attention weight tensor sequence to obtain the compact attention processing feature sequence.

[0045] In this embodiment, as an optional implementation, since a bottleneck compression layer is introduced before the local attention calculation, the local attention calculation is performed in this compressed low-dimensional space. Correspondingly, a bottleneck recovery layer is introduced at the output of the compact attention processing, and the attention-weighted features are then restored to their original dimension through a bottleneck recovery layer (a C / R -> C linear layer). Finally, the module's output is residually concatenated with the original input X. Specifically, the compact attention processing feature sequence is transformed into a transformed feature sequence with the same dimension as the encoded feature sequence, and a residual concatenation is performed between the transformed feature sequence and the encoded feature sequence to obtain the reconstructed feature sequence.

[0046] As an optional embodiment, the weighted features (compact attention processing feature sequence) are transformed and reconstructed back into the transformed feature sequence [T, 128], for example, by restoring it to 128 dimensions through a linear layer. Finally, it is added to the encoded feature sequence X obtained by the TC-ResNet sub-model to obtain the grouped weighted feature sequence, i.e., the reconstructed feature sequence.

[0047] S104. Based on the features after compact attention processing, a fully connected classifier is used to process and output the probability of whether it is a wake-up keyword; In some specific embodiments, the features after compact attention processing are the aforementioned compact attention processing feature sequence.

[0048] In other specific embodiments, the features after compact attention processing are the aforementioned reconstructed feature sequence.

[0049] The features based on compact attention processing are used with a fully connected classifier to output the probability of whether it is a wake-up keyword, including: The reconstructed feature sequence is used as the feature after compact attention processing. After global average pooling, it is input into a pre-built fully connected classifier to output the probability of whether it is a wake word.

[0050] It is understandable that this step can be implemented using the processing flow of various existing fully connected classifiers, which will not be elaborated here.

[0051] S105. Based on the probability of whether it is a wake-up word, obtain the wake-up keyword contained in the audio data.

[0052] In this embodiment, if the probability exceeds a preset probability threshold, the keyword corresponding to that probability is the wake-up keyword, and a wake-up operation can be performed based on the obtained wake-up keyword.

[0053] In this embodiment, instead of simply utilizing an attention mechanism, it addresses the strict resource constraints and core challenges of the edge-side keyword wake-up task by striking a key balance point. This approach achieves maximum performance improvement for the keyword wake-up model with minimal computational cost, significantly simplifying computational complexity and reducing required computing resources while effectively ensuring the accuracy of wake-up keyword recognition. When applied to embedded devices, this approach offers the following beneficial technical effects: a. Performance Improvement: On public datasets (such as Google Speech Commands), by adding the compact attention module of this embodiment to the baseline CNN model, the keyword wake-up model can better focus on the speech subject and suppress noise, thereby effectively reducing the error rate, especially in low signal-to-noise ratio environments (+5dB SNR), where the wake-up rate is effectively improved.

[0054] b. Minimal additional overhead: The compact attention module added in this embodiment has fewer than 1K parameters and increases the computational cost (FLOPs) by less than 5% of the baseline model, thus meeting real-time requirements.

[0055] c. Enhanced model robustness: Since the attention mechanism can dynamically focus on the core part of the keyword, it can effectively increase the robustness of the keyword wake-up model to changes in pronunciation speed and accent differences.

[0056] Based on the same inventive concept, such as Figure 2 As shown, this embodiment of the invention also provides a device for obtaining wake-up keywords, the device comprising: The data framing module 201 is used to segment the user's audio data into frames, extract the audio features of each audio data frame, and obtain the audio feature sequence. In this embodiment, as an optional embodiment, the data framing module 201 is specifically used for: From the beginning position of the audio data, audio data of a preset frame length is extracted to obtain the first audio data frame; Starting from the first position of the audio data corresponding to the first position of the first audio data frame, backtrack the audio data for a preset frame shift time in the direction of the beginning position of the audio data to obtain the starting position of the second audio data frame; Starting from the beginning of the second audio data frame, audio data of a certain duration of the frame is extracted to obtain the second audio data frame, until the audio data is completely segmented into frames.

[0057] The encoding module 202 is used to input the acquired audio feature sequence into a pre-built CNN encoder to obtain the encoded feature sequence; In this embodiment, as an optional implementation, the audio feature sequence composed of the MFCC features corresponding to each audio data frame is input into an 8-layer TC-ResNet sub-model. Taking 16 audio data frames as an example, a coded feature sequence X with dimensions [T, 128] is output, where T is the time step, i.e., the number of audio data frames contained in the audio data. In this embodiment, T=16, and 128 is the number of channels contained in the output coded feature sequence.

[0058] Attention processing module 203 is used to perform compact attention processing on the encoded feature sequence, wherein: local causal attention calculation is performed on the input feature sequence to obtain a normalized local attention weight sequence; the channels contained in the encoded feature sequence are grouped to obtain multiple feature channel groups; based on the number of channels contained in each feature channel group, the normalized local attention weight sequence is copied in the channel dimension to obtain the number of channels, thereby obtaining an attention weight tensor sequence; the encoded feature sequence is reshaped by performing matrix operations matching the attention weight tensor sequence to obtain a reshaped feature sequence; and the product of the reshaped feature sequence and the attention weight tensor sequence is obtained to obtain a compact attention processing feature sequence. In this embodiment, as an optional embodiment, the attention processing module 203 is specifically used to: perform causal convolution on the input feature sequence using a pre-set one-dimensional convolution kernel to obtain unnormalized probability weights, and obtain a normalized local attention weight sequence based on the unnormalized probability weights and the classification normalization function.

[0059] In this embodiment, as another optional embodiment, the attention processing module 203 is specifically used for: Using the one-dimensional convolution kernel, for the audio data frame corresponding to the t-th time step in the input feature sequence, calculate the causal convolution of the t-th audio data frame with all audio data frames in the range [t, t+W] to obtain the unnormalized probability weight corresponding to the t-th audio data frame, where W is the number of audio data frames in the time window.

[0060] In this embodiment, as an optional embodiment, all audio data frames in the range [t, t+W] include: W audio data frames centered on the t-th audio data frame on the time axis.

[0061] In this embodiment, as another optional embodiment, all audio data frames in the range [t, t+W] include: W audio data frames on the time axis starting with the t-th audio data frame.

[0062] In this embodiment, the channels contained in the encoded feature sequence are grouped to obtain multiple feature channel groups. Based on the number of channels contained in each feature channel group, the normalized local attention weight sequence is copied in the channel dimension to obtain the number of channels, and an attention weight tensor sequence is obtained. The encoded feature sequence is reshaped to match the matrix operation of the attention weight tensor sequence to obtain the reshaped feature sequence. The product of the reshaped feature sequence and the attention weight tensor sequence is obtained to obtain the compact attention processing feature sequence. The classification normalization function is the Softmax function.

[0063] In this embodiment, as an optional implementation, the attention processing module 203 is specifically used to: introduce a bottleneck structure to further compress parameters and stabilize training before performing local causal attention calculation. Specifically, this includes: projecting the encoded feature sequence onto a linear layer to obtain a compressed feature sequence with reduced feature dimensions; using the compressed feature sequence as the input feature sequence for the local causal attention calculation, so that the local causal calculation process can utilize a pre-set one-dimensional convolution kernel to perform causal convolution on the compressed feature sequence to obtain unnormalized probability weights. A bottleneck recovery layer is introduced at the output of the compact attention processing, specifically including: transforming the compact attention processing feature sequence into a transformed feature sequence with the same dimension as the encoded feature sequence; performing a residual connection between the transformed feature sequence and the encoded feature sequence to obtain a reconstructed feature sequence.

[0064] The probability calculation module 204 is used to process and output the probability of whether a feature is a wake-up keyword based on the features after compact attention processing using a fully connected classifier. In some specific embodiments, the features after compact attention processing are the aforementioned compact attention processing feature sequence.

[0065] In other specific embodiments, the features after compact attention processing are the aforementioned reconstructed feature sequence.

[0066] As an optional embodiment, the probability calculation module 204 is also used for: The reconstructed feature sequence is used as the feature after compact attention processing. After global average pooling, it is input into a pre-built fully connected classifier to output the probability of whether it is a wake word.

[0067] The wake-up keyword determination module 205 is used to obtain the wake-up keyword contained in the audio data based on the probability of whether it is a wake-up word.

[0068] In this embodiment, if the probability exceeds a preset probability threshold, the keyword corresponding to that probability is the wake-up keyword, and a wake-up operation can be performed based on the obtained wake-up keyword.

[0069] In this embodiment, as an optional implementation, the kernel size of the one-dimensional convolution kernel is the same as the compression ratio of the linear layer projection.

[0070] In this embodiment, as an optional implementation, the dimension of the encoded feature sequence is 128; the compression ratio is 8. The one-dimensional convolution kernel has a kernel size of 8, an output channel number of 1, and a stride of 1.

[0071] It is understood that the device for obtaining the wake-up keyword can be used to implement the steps of the method for obtaining the wake-up keyword in any of the above possible implementations, and for the sake of simplicity, it will not be described in detail here.

[0072] Based on the same inventive concept, see [link to inventive concept] Figure 3 The present invention also provides an embedded device, including a memory 101 (e.g., non-volatile memory), a processor 102, and a computer program stored on the memory 101 and executable on the processor 102. When the processor 102 executes the program, it implements the steps of the method for obtaining wake-up keywords in any of the above possible implementations, which can be equivalent to the aforementioned device for obtaining wake-up keywords. Of course, the processor can also be used to process other data or perform calculations.

[0073] like Figure 3 As shown, the embedded device may also include: memory 103, network interface 104, and internal bus 105. Other hardware may also be included besides these components, which will not be described in detail here.

[0074] It should be noted that the device for obtaining wake-up keywords can be implemented by software. As a logical device, it is formed by the processor 102 of the embedded device reading the computer program instructions stored in the non-volatile memory into the memory 103 for execution.

[0075] The embodiments of the subject matter and functional operation described in this specification can be implemented in the following ways: digital electronic circuits, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or combinations thereof. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by a data processing apparatus or for controlling the operation of a data processing apparatus. Alternatively or additionally, the program instructions may be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information and transmit it to a suitable receiving device for execution by the data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or combinations thereof.

[0076] The processing and logic flow described in this specification can be executed by one or more programmable computers that execute one or more computer programs to perform corresponding functions by operating on input data and generating output. The processing and logic flow can also be executed by special-purpose logic circuitry—such as FPGA (Field Programmable Gate Array) or ASIC (Application-Specific Integrated Circuit), and the device can also be implemented as special-purpose logic circuitry.

[0077] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily used to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0078] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0079] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0080] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0081] The above are merely specific embodiments of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for obtaining wake-up keywords, applicable to embedded devices, characterized in that, include: The user's audio data is segmented into frames, and the audio features of each audio data frame are extracted to obtain an audio feature sequence. The acquired audio feature sequence is input into a pre-built CNN encoder to obtain the encoded feature sequence; Compact attention processing is performed on the encoded feature sequence, wherein: local causal attention calculation is performed on the input feature sequence to obtain a normalized local attention weight sequence; the channels contained in the encoded feature sequence are grouped to obtain multiple feature channel groups; based on the number of channels contained in each feature channel group, the normalized local attention weight sequence is copied in the channel dimension to obtain the number of channels, thereby obtaining an attention weight tensor sequence; the encoded feature sequence is reshaped by performing matrix operations matching the attention weight tensor sequence to obtain a reshaped feature sequence; the product of the reshaped feature sequence and the attention weight tensor sequence is obtained to obtain a compact attention processing feature sequence. Based on the features processed by compact attention, a fully connected classifier is used to process and output the probability of whether it is a wake-up keyword. Based on the probability of whether it is a wake-up word, the wake-up keyword contained in the audio data is obtained.

2. The method according to claim 1, characterized in that: The step of performing local causal attention calculation on the input feature sequence to obtain a normalized local attention weight sequence includes: using a pre-set one-dimensional convolution kernel to perform causal convolution on the input feature sequence to obtain unnormalized probability weights, and obtaining a normalized local attention weight sequence based on the unnormalized probability weights and the classification normalization function.

3. The method according to claim 1 or 2, characterized in that: The compact attention processing of the encoded feature sequence further includes: The encoded feature sequence is projected onto a linear layer to obtain a compressed feature sequence with reduced feature dimensions; the compressed feature sequence is then used as the input feature sequence for the local causal attention calculation. The compact attention processing feature sequence is transformed into a transformed feature sequence with the same dimension as the encoded feature sequence. The transformed feature sequence and the encoded feature sequence are then joined by a residual connection to obtain the reconstructed feature sequence. The features based on compact attention processing are used with a fully connected classifier to output the probability of whether it is a wake-up keyword, including: The reconstructed feature sequence is used as the feature after compact attention processing. After global average pooling, it is input into a pre-built fully connected classifier to output the probability of whether it is a wake word.

4. The method for obtaining wake-up keywords according to claim 2, characterized in that, The step of performing causal convolution on the input feature sequence using a pre-set one-dimensional convolution kernel to obtain unnormalized probability weights includes: Using the one-dimensional convolution kernel, for the audio data frame corresponding to the t-th time step in the input feature sequence, calculate the causal convolution of the t-th audio data frame with all audio data frames in the range [t, t+W] to obtain the unnormalized probability weight corresponding to the t-th audio data frame, where W is the number of audio data frames in the time window.

5. The method for obtaining wake-up keywords according to claim 4, characterized in that, The range [t, t+W] includes all audio data frames, centered on the t-th audio data frame on the time axis; or... All audio data frames within the range [t, t+W] include: W audio data frames on the time axis, with the t-th audio data frame as the starting audio data frame.

6. The method for obtaining wake-up keywords according to claim 3, characterized in that, The size of the one-dimensional convolution kernel is the same as the compression ratio of the linear layer projection.

7. The method for obtaining wake-up keywords according to claim 6, characterized in that, The dimension of the encoded feature sequence is 128; the compression ratio is 8. The one-dimensional convolution kernel has a kernel size of 8, an output channel number of 1, and a stride of 1. The channels contained in the encoded feature sequence are grouped to obtain 8 feature channel groups. Based on the number of channels (16) contained in each feature channel group, the normalized local attention weight sequence is copied in the channel dimension to obtain the attention weight tensor sequence. The classification normalization function is the Softmax function; The CNN encoder uses the TC-ResNet model.

8. The method for obtaining wake-up keywords according to claim 3, characterized in that, The step of performing residual concatenation on the transformed feature sequence and the encoded feature sequence to obtain the reconstructed feature sequence includes: The reconstructed feature sequence is obtained by summing the transformed feature sequence and the encoded feature sequence.

9. A device for acquiring wake-up keywords, characterized in that, The device for acquiring wake-up keywords includes: The data framing module is used to segment the user's audio data into frames, extract the audio features of each audio data frame, and obtain the audio feature sequence. The encoding module is used to input the acquired audio feature sequence into a pre-built CNN encoder to obtain the encoded feature sequence; An attention processing module is used to perform compact attention processing on the encoded feature sequence, wherein: local causal attention calculation is performed on the input feature sequence to obtain a normalized local attention weight sequence; the channels contained in the encoded feature sequence are grouped to obtain multiple feature channel groups; based on the number of channels contained in each feature channel group, the normalized local attention weight sequence is copied in the channel dimension to obtain the number of channels, thereby obtaining an attention weight tensor sequence; the encoded feature sequence is reshaped by performing matrix operations matching the attention weight tensor sequence to obtain a reshaped feature sequence; and the product of the reshaped feature sequence and the attention weight tensor sequence is obtained to obtain a compact attention processing feature sequence. The probability calculation module is used to process the features after compact attention processing using a fully connected classifier and output the probability of whether it is a wake-up keyword. The wake-up keyword determination module is used to obtain the wake-up keyword contained in the audio data based on the probability of whether it is a wake-up word.

10. An embedded device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method for obtaining wake-up keywords as described in any one of claims 1 to 8.

Images