Voice wake-up method and model training method for voice diversity conditions, and related devices

Abstract

The application provides a speech wake-up method and model training method under a speech diversity condition and related devices, and belongs to the field of speech recognition. The method first pre-processes the acquired speech signal and extracts acoustic features, which can avoid noise and redundant information in the original speech signal from interfering with the determination result. Then, the speech wake-up model can decouple the acoustic features into wake-up word features and speaker voice characteristics, and use the speaker voice characteristics to constrain the model to perform wake-up determination only based on the wake-up word features. This design can effectively separate the individual differences of the speaker (such as accent and voice tone) from the wake-up core information, avoid the influence of the model by the speaker change and accent difference, thereby significantly improving the robustness and generalization ability of the model in complex speech diversity scenarios, and reducing the occurrence of false wake-up and missed wake-up.

Description

Technical Field

[0001] This invention belongs to the field of speech recognition technology, specifically relating to a speech wake-up method, model training method, and related apparatus for speech diversity conditions. Background Technology

[0002] Voice recognition technology is widely used in many fields such as smart devices and smart homes. Voice wake-up, as a crucial component of voice recognition, aims to quickly activate a device from standby mode using a specific wake word, bringing it into working mode and providing users with a convenient interactive experience. With the increasing prevalence of smart devices and the continuous expansion of application scenarios, the requirements for voice wake-up technology are also rising.

[0003] In practical applications, speech signals are affected by a variety of factors, such as accent differences among people from different regions, variations in speech rate, potential ambiguity in pronunciation, and individual differences among speakers. These speech diversity factors pose a significant challenge to voice wake-up, making it difficult for traditional voice wake-up models to work accurately and stably in complex and ever-changing speech environments.

[0004] Traditional voice wake-up models exhibit poor robustness in complex speech diversity scenarios, such as diverse accents, uneven speech rates, unclear pronunciation, and speaker variations. They lack generalization ability for different speakers and accents, and are prone to false or missed wake-ups. Furthermore, traditional models are typically large, with high computational and parameter requirements, making it difficult to meet the lightweight requirements of embedded devices and limiting their widespread application. Summary of the Invention

[0005] Therefore, it is necessary to provide a voice wake-up method, model training method and related device for speech diversity conditions to address the above-mentioned technical problems, so as to improve the robustness and generalization ability of the voice wake-up model in complex scenarios, while realizing the lightweighting of the model so that it can meet the deployment requirements of embedded devices.

[0006] In a first aspect, the present invention provides a voice wake-up method for conditions of voice diversity, comprising the following steps:

[0007] Acquire speech signals, preprocess the speech signals, and extract acoustic features;

[0008] The trained voice wake-up model is used to determine voice wake-up based on acoustic features, and a wake-up result is generated.

[0009] Among them, the voice wake-up model is used to decouple acoustic features into wake word features and speaker timbre features, and uses speaker timbre features to constrain the voice wake-up model to perform voice wake-up determination only based on wake word features.

[0010] Furthermore, before the voice wake-up model decouples acoustic features into wake word features and speaker timbre features, it also includes:

[0011] Temporal features are extracted from acoustic features to obtain temporal features;

[0012] The extracted temporal features are subjected to adaptive weighting to obtain attention-enhanced features; these attention-enhanced features are then decoupled into wake word features and speaker timbre features.

[0013] Furthermore, voice wake-up determination is performed based on wake-up word features, including:

[0014] By performing marginal optimization on the wake word features, the optimized wake decision boundary is obtained;

[0015] Based on the wake-up decision boundary, the wake-up word features are subjected to differential constraint processing to obtain a wake-up determination rule that balances the missed wake-up rate and the false wake-up rate.

[0016] The wake-up word features are used to determine the wake-up function according to the wake-up determination rules, and the corresponding wake-up result is generated.

[0017] Secondly, the present invention provides a lightweight voice wake-up model training method for speech diversity conditions, comprising the following steps:

[0018] Construct a trainable speech wake-up model that includes an adversarial feature decoupling branch; the adversarial feature decoupling branch includes a gradient inversion layer and a speaker discrimination branch;

[0019] Construct a speech diversity training dataset and preprocess the speech signals in the speech diversity training dataset to obtain acoustic features for training.

[0020] The acoustic features used for training are input into the speech wake-up model to be trained. The features are decoupled through a gradient inversion layer to obtain the wake-up word features and the speaker timbre features used for training. Through the speaker discrimination branch, the speaker timbre features used for training are used to constrain the speech wake-up model to be trained to perform wake-up only based on the wake-up word features.

[0021] The model parameters of the voice wake-up model to be trained are optimized by utilizing wake word features, resulting in a well-trained voice wake-up model;

[0022] The trained voice wake-up model is then lightweighted to obtain a lightweight voice wake-up model.

[0023] Furthermore, the speech signals in the speech diversity training dataset are preprocessed to obtain acoustic features for training, including:

[0024] The speech signal undergoes basic preprocessing to obtain the basic speech signal;

[0025] Extract FBank features from the basic speech signal and normalize the FBank features;

[0026] Data augmentation processing is performed on the normalized FBank features to obtain training acoustic features that meet the needs of speech diversity.

[0027] Furthermore, the voice wake-up model to be trained also includes a temporal feature extraction branch and an adaptive weighting branch; before feature decoupling through the gradient reversal layer, it also includes:

[0028] Through the temporal feature extraction branch, temporal features are extracted from the acoustic features used for training to obtain the temporal features used for training.

[0029] By using an adaptive weighted branch, the training temporal features are adaptively weighted to obtain the training attention-enhanced features;

[0030] The attention-enhanced features used in training are input into the adversarial feature decoupling branch to decouple the features.

[0031] Furthermore, the trained voice wake-up model is lightweighted to obtain a lightweight voice wake-up model, including:

[0032] The trained voice wake-up model is quantized and pruned to obtain a voice wake-up model with optimized parameters.

[0033] Export the files and configuration parameters of the optimized voice wake-up model to obtain a deployable lightweight voice wake-up model.

[0034] Thirdly, the present invention provides a voice wake-up device for conditions of voice diversity, comprising:

[0035] The signal acquisition and preprocessing module is used to acquire speech signals, preprocess the speech signals and extract acoustic features;

[0036] The voice wake-up determination module is used to determine voice wake-up based on acoustic features using a trained voice wake-up model and generate a wake-up result.

[0037] Among them, the voice wake-up model is used to decouple acoustic features into wake word features and speaker timbre features. The speaker timbre features are used to constrain the voice wake-up model to perform wake-up only based on wake word features, and the voice wake-up determination is based on wake word features.

[0038] Furthermore, before the voice wake-up model decouples acoustic features into wake word features and speaker timbre features, it also includes:

[0039] Temporal features are extracted from acoustic features to obtain temporal features;

[0040] The extracted temporal features are subjected to adaptive weighting to obtain attention-enhanced features; these attention-enhanced features are then decoupled into wake word features and speaker timbre features.

[0041] Furthermore, voice wake-up determination is performed based on wake-up word features, including:

[0042] By performing marginal optimization on the wake word features, the optimized wake decision boundary is obtained;

[0043] Based on the wake-up decision boundary, the wake-up word features are subjected to differential constraint processing to obtain a wake-up determination rule that balances the missed wake-up rate and the false wake-up rate.

[0044] The wake-up word features are used to determine the wake-up function according to the wake-up determination rules, and the corresponding wake-up result is generated.

[0045] Fourthly, the present invention provides a computer device, the device including a processor and a memory:

[0046] The memory is used to store computer programs and send the instructions of the computer programs to the processor;

[0047] The processor executes the following steps according to the instructions of the computer program:

[0048] Acquire speech signals, preprocess the speech signals, and extract acoustic features;

[0049] The trained voice wake-up model is used to determine voice wake-up based on acoustic features, and a wake-up result is generated.

[0050] Among them, the voice wake-up model is used to decouple acoustic features into wake word features and speaker timbre features, and uses speaker timbre features to constrain the voice wake-up model to perform voice wake-up determination only based on wake word features.

[0051] Fifthly, the present invention provides a computer-readable storage medium on which a computer program is stored, and when executed by a processor, the computer program performs the following steps:

[0052] Acquire speech signals, preprocess the speech signals, and extract acoustic features;

[0053] The trained voice wake-up model is used to determine voice wake-up based on acoustic features, and a wake-up result is generated.

[0054] Among them, the voice wake-up model is used to decouple acoustic features into wake word features and speaker timbre features, and uses speaker timbre features to constrain the voice wake-up model to perform voice wake-up determination only based on wake word features.

[0055] Sixthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0056] Acquire speech signals, preprocess the speech signals, and extract acoustic features;

[0057] The trained voice wake-up model is used to determine voice wake-up based on acoustic features, and a wake-up result is generated.

[0058] Among them, the voice wake-up model is used to decouple acoustic features into wake word features and speaker timbre features, and uses speaker timbre features to constrain the voice wake-up model to perform voice wake-up determination only based on wake word features.

[0059] In summary, this invention provides a voice wake-up method for speech diversity conditions. This method first preprocesses the acquired speech signal and extracts acoustic features, avoiding interference from noise and redundant information in the original speech signal. Then, through a voice wake-up model, the acoustic features are decoupled into wake-up word features and speaker timbre features. The speaker timbre features constrain the model to perform wake-up judgments only based on wake-up word features. This design effectively separates individual speaker differences (such as accent and timbre) from the core wake-up information, avoiding the model being affected by speaker changes and accent differences. This significantly improves the robustness and generalization ability of the model in complex speech diversity scenarios, reducing false wake-ups and missed wake-ups.

[0060] This invention also provides a training method for a voice wake-up model under conditions of speech diversity. This method constructs a training model that includes an adversarial feature decoupling branch (containing a gradient inversion layer and a speaker discrimination branch). Utilizing a speech diversity training dataset, the model can fully learn the features of complex speech scenarios during training. The gradient inversion layer achieves feature decoupling, and the speaker discrimination branch provides constraints, further strengthening the model's focus on learning wake-up word features and ensuring that the trained model can adapt to diverse speech scenarios. At the same time, by lightweighting the trained model, the number of model parameters and computational cost are effectively reduced, solving the problem that traditional models are too large to adapt to embedded devices. This allows the lightweight trained model to meet the deployment requirements of embedded devices.

[0061] The present invention also provides a voice wake-up device, computer equipment, computer-readable storage medium and computer program product for conditions of voice diversity, which have similar effects to the above-mentioned voice wake-up method when implemented, and will not be described in detail here. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 This is a flowchart illustrating a voice wake-up method for speech diversity conditions in one embodiment of the present invention.

[0064] Figure 2 This is a flowchart illustrating a voice wake-up model training method under conditions of speech diversity, as described in one embodiment of the present invention.

[0065] Figure 3 This is a block diagram of a voice wake-up device for speech diversity conditions in one embodiment of the present invention;

[0066] Figure 4 This is a block diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0067] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0068] It should be noted that the user information (including but not limited to user images, user portrait information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with relevant regulations.

[0069] The background technology of this invention will be further introduced below.

[0070] With the rapid development of voice interaction technology, voice wake-up, as the entry point for voice interaction systems, has been widely applied in various scenarios such as smart homes, wearable devices, and embedded terminals. Its core requirement is to accurately identify preset wake words in complex environments to achieve rapid device response and interaction. Currently, voice wake-up technology has mainly formed three mainstream implementation schemes: traditional methods based on template matching, end-to-end methods based on deep learning, and the CTC (Connectionist Temporal Classification) scheme, which balances performance and efficiency. Among them, the deep learning-based scheme has become the mainstream technology direction due to its high recognition accuracy.

[0071] In current technological practices, the design of traditional voice wake-up models has significant limitations, making it difficult to adapt to complex and diverse speech scenarios. On the one hand, most traditional models fail to effectively decouple speech features, making them susceptible to interference from individual speaker differences (such as timbre and accent) when faced with diverse accents, uneven speech rates, unclear pronunciation, and changes in speaker identity. This makes them unable to focus on the core features of the wake word, leading to false wake-ups and missed wake-ups, significantly reducing robustness and generalization ability. For example, traditional template-matching methods, which only compare the input speech with a preset wake word template, cannot adapt to the differences in accents and pronunciation habits of different speakers, resulting in a significant drop in recognition accuracy in non-standard pronunciation scenarios. While some end-to-end deep learning-based models can improve recognition accuracy to some extent, they are still affected by speaker variations due to the lack of feature decoupling, resulting in insufficient generalization ability.

[0072] On the other hand, existing voice wake-up models generally suffer from large parameter counts and high computational complexity, making it difficult to meet the deployment requirements of embedded devices. As voice wake-up technology becomes more widespread in low-resource devices such as wearable devices, in-vehicle embedded terminals, and small smart home devices, the demand for lightweight models is increasing. However, traditional deep learning models, in pursuit of recognition accuracy, often design complex network structures, resulting in large model sizes and high computational costs. This not only consumes a large amount of device storage resources but also increases device power consumption, making it impossible to run stably in low-resource embedded environments and severely limiting the expansion of application scenarios for voice wake-up technology.

[0073] Furthermore, while some existing models attempt to achieve lightweight optimization, this often comes at the cost of sacrificing recognition accuracy, failing to balance model lightweighting with recognition robustness, further highlighting the shortcomings of existing technologies.

[0074] To address the shortcomings of existing voice wake-up models, such as poor robustness and insufficient generalization ability in complex and diverse speech scenarios, large model size making them difficult to adapt to embedded device deployments, and the inability to balance lightweight design with recognition accuracy, this invention proposes a voice wake-up method, model training method, and related apparatus for speech diversity conditions. The following provides a detailed description of various embodiments of this invention.

[0075] Please see Figure 1 This embodiment provides a voice wake-up method for conditions of voice diversity, including the following steps:

[0076] S101: Acquire speech signals, preprocess the speech signals and extract acoustic features.

[0077] Among them, speech signal preprocessing refers to operations such as denoising, framing, and amplitude normalization on the original speech signal. The purpose is to remove environmental noise and redundant information from the speech signal to ensure the accuracy of subsequent feature extraction. Acoustic feature extraction refers to extracting features that can reflect the essence of speech from the preprocessed speech signal.

[0078] Optionally, speech signal preprocessing can be implemented based on signal processing technology. Specifically, it can eliminate interference factors in the original speech signal by filtering invalid noise and unifying signal amplitude, so that the extracted acoustic features can truly reflect the core information of the speech. The principle of acoustic feature extraction is to select features from the preprocessed speech signal that can distinguish between wake words and non-wake words and can be adapted to various speech scenarios. Its core is to construct a reliable input basis for wake-up determination through feature selection, so as to ensure that speech signals in different scenarios can be effectively recognized.

[0079] For example, after acquiring the user's voice signal, the voice signal is preprocessed to remove environmental interference noise, and then acoustic features are extracted to obtain effective feature inputs for subsequent wake-up determination.

[0080] S102: The acoustic features are used to determine the wake-up function by using the trained voice wake-up model, and the wake-up result is generated. The voice wake-up model is used to decouple the acoustic features into wake-up word features and speaker timbre features. The speaker timbre features are used to constrain the voice wake-up model to perform the voice wake-up determination only based on the wake-up word features.

[0081] Among them, the voice wake-up model refers to a model that, after training, can achieve feature decoupling and wake-up determination. Its function is to separate wake-up-related features from irrelevant features. Decoupling means splitting acoustic features into wake-up core features (wake-up word features) and individual difference features (speaker timbre features) to avoid interference from individual speaker differences on wake-up determination. Speaker timbre feature constraint means forming a constraint mechanism through this feature to force the model to focus on wake-up word features and ignore interference caused by individual speaker differences.

[0082] Optionally, feature decoupling in this step involves using an internal feature separation mechanism within the model to split the mixed acoustic features into wake-up-related features (wake-up word features) and speaker-independent features (speaker timbre features). The principle is to separate features based on semantic differences, ensuring that the two types of features do not interfere with each other. The principle of speaker timbre feature constraint is to generate a constraint signal through this feature, which acts inversely on the model's decision process, forcing the model to focus only on wake-up word features and avoiding decision bias caused by differences in speaker timbre and accent. This design aims to solve the problem of insufficient generalization ability of traditional models, enabling the model to maintain stable wake-up accuracy when facing different speakers and accents.

[0083] For example, the preprocessed acoustic features are input into the trained voice wake-up model. The model decouples the acoustic features into wake-up word features and speaker timbre features. The speaker timbre features are used to constrain the model to ensure that the model performs wake-up judgment only based on the wake-up word features, and finally generates an accurate wake-up result.

[0084] The voice wake-up method provided in this embodiment first preprocesses the acquired speech signal and extracts acoustic features, which can avoid noise and redundant information in the original speech signal from interfering with the judgment result. Then, the voice wake-up model can decouple the acoustic features into wake-up word features and speaker timbre features, and use speaker timbre features to constrain the model to perform wake-up judgment only based on wake-up word features. This design can effectively separate individual differences of speakers (such as accent and timbre) from the core wake-up information, avoid the model being affected by speaker changes and accent differences, thereby significantly improving the robustness and generalization ability of the model in complex speech diversity scenarios and reducing the occurrence of false wake-ups and missed wake-ups.

[0085] In an exemplary embodiment, before the voice wake-up model decouples the acoustic features into wake word features and speaker timbre features, it further includes:

[0086] S201: Extract temporal features from acoustic features to obtain temporal features.

[0087] Temporal features refer to features that can reflect changes in the time dimension of speech signals, including dynamic change information of speech signals. The core purpose of temporal feature extraction is to capture the temporal correlation of speech signals and avoid the limitation that a single static feature cannot fully reflect the dynamic changes of speech.

[0088] Optionally, the temporal feature extraction in this step can be based on time series analysis technology. By utilizing the temporal correlation of speech signals, discrete acoustic features can be integrated into a feature sequence with temporal correlation. The principle is that speech signals have short-term stationarity. By capturing the changing patterns of acoustic features at different times, the complete information of speech can be reflected more comprehensively, avoiding feature fragmentation caused by ignoring the time dimension, thereby improving the accuracy of subsequent feature decoupling.

[0089] For example, temporal feature extraction is performed on the preprocessed acoustic features to capture the temporal dimension change information of the speech signal and obtain temporal features containing dynamic changes.

[0090] S202: Adaptive weighting is applied to the extracted temporal features to obtain attention-enhanced features; these attention-enhanced features are used to decouple wake word features and speaker timbre features.

[0091] Among them, adaptive weighting refers to assigning high weights to effective features and low weights to redundant features based on the importance of temporal features, thereby achieving effective feature selection. Its core purpose is to highlight effective features related to wake words, suppress redundant and interfering features, and improve the targeting of feature decoupling.

[0092] Optionally, the core of adaptive weighting in this step is based on the attention mechanism. By calculating the correlation between different dimensions of temporal features and the wake word, high weights are given to effective features with high correlation, while low weights are given to redundant and interfering features with low correlation, or even filtered out. Its principle is consistent with the attention allocation logic, prioritizing features that are meaningful to wake-up and ignoring irrelevant interference, thereby improving feature quality.

[0093] For example, the temporal features are subjected to adaptive weighting, with high weights assigned to effective features related to the wake word and low weights assigned to redundant and interfering features, to obtain attention-enhanced features for subsequent feature decoupling operations.

[0094] In this embodiment, the quality of acoustic features is further optimized by extracting temporal features and applying adaptive weights, thus solving the problem of weak feature specificity in traditional wake-up methods.

[0095] In one exemplary embodiment, performing voice wake-up determination based on wake-up word features includes:

[0096] S301: Perform marginal optimization on the wake word features to obtain the optimized wake decision boundary.

[0097] Among them, marginal optimization refers to adapting to speech scenarios with different accents and pronunciation habits by adjusting the threshold range of feature determination; wake-up decision boundary refers to the judgment criteria that distinguish wake words from non-wake words. Its function is to determine the judgment range of wake word features and ensure that speech signals with different pronunciation methods can be accurately recognized.

[0098] Optionally, based on boundary optimization theory in statistical learning, the threshold parameters of the decision boundary can be adjusted through training with a large number of diverse speech samples, so that the decision boundary can cover the range of speech features with different accents and pronunciation habits. The principle is to minimize the error of wake-up determination, expand the effective determination range, and avoid missed or false wake-ups caused by pronunciation deviations, which can effectively adapt to diverse speech scenarios.

[0099] For example, marginal optimization processing is performed on the wake word features to adjust the threshold range of the wake decision boundary so that it can adapt to the pronunciation differences of different accents and ensure accurate recognition even if there are slight deviations in pronunciation.

[0100] S302: Based on the wake-up decision boundary, the wake-up word features are subjected to differential constraint processing to obtain a wake-up determination rule that balances the missed wake-up rate and the false wake-up rate.

[0101] Among them, differentiated constraint processing refers to using different judgment criteria for different types of wake-up word features based on the characteristics of the speech signal; the missed wake-up rate refers to the probability of not recognizing a wake-up word, and the false wake-up rate refers to the probability of misrecognizing a non-wake-up word. The purpose of this step is to balance the two errors and improve the reliability of wake-up judgment.

[0102] Optionally, this step can be based on the cost-sensitive learning principle, setting different constraint weights for two different types of errors: missed wake-up and false wake-up. By adjusting the constraint parameters, the model can prioritize avoiding errors that have a greater impact on the user experience (such as missed wake-up) while controlling the occurrence of the other type of error (such as false wake-up). The principle is to use differentiated penalty strategies to allow the model to flexibly adjust the judgment criteria according to different speech features during the judgment process, thereby achieving a balance between the two types of errors and improving the accuracy and reliability of the judgment.

[0103] For example, based on the optimized wake-up decision boundary, a strict judgment standard is adopted for wake-up word features with clear pronunciation to reduce false wake-ups; a lenient judgment standard is adopted for wake-up word features with ambiguous pronunciation to avoid missed wake-ups, and finally a wake-up judgment rule that balances the missed wake-up rate and the false wake-up rate is obtained.

[0104] S303: Perform voice wake-up determination on the wake-up word features according to the wake-up determination rules, and generate the corresponding wake-up result.

[0105] This step involves matching the wake word features with the optimized judgment rules, and generating a wake-up command or a non-wake-up command based on the matching results to ensure the accuracy of the wake-up results.

[0106] Optionally, based on feature matching theory, the optimized wake word features can be compared with the standard features in the wake-up determination rules. If the feature matching degree reaches the determination threshold, a wake-up result is generated; otherwise, no wake-up result is generated. The principle is to ensure the accuracy of wake-up determination through feature matching, while adapting to speech scenarios with different accents and pronunciation methods.

[0107] For example, according to the wake-up determination rule after balancing the missed wake-up rate and the false wake-up rate, the wake-up word features are used to determine voice wake-up. If the wake-up word features meet the determination rule, a wake-up command is generated and the device is triggered to respond; if they do not meet the rule, wake-up is not triggered and the corresponding wake-up result is generated.

[0108] In this embodiment, by performing marginal optimization and differential constraint processing on the wake-up word features, the problems of poor adaptability and high error rate in traditional wake-up determination are solved, and adaptation to different pronunciations and different speaker scenarios is achieved, thereby improving the accuracy and robustness of voice wake-up.

[0109] Secondly, the present invention provides a lightweight voice wake-up model training method for speech diversity conditions, comprising the following steps:

[0110] S401: Construct a voice wake-up model to be trained, including an adversarial feature decoupling branch; the adversarial feature decoupling branch includes a gradient inversion layer and a speaker discrimination branch.

[0111] Among them, the voice wake-up model to be trained refers to a model whose parameters have not been optimized and whose wake-up performance can be gradually improved through training; the adversarial feature decoupling branch is a module in the model used to achieve feature decoupling and model constraints; the gradient reversal layer is a component used to achieve feature decoupling; and the speaker discrimination branch is a component used to generate constraint signals to constrain the training direction of the model.

[0112] Optionally, the principle of constructing the model to be trained is to achieve feature decoupling and model constraint by setting an adversarial feature decoupling branch, ensuring that the model can focus on the core features of the wake word. The principle of the adversarial feature decoupling branch is based on the adversarial training idea. Through the synergy of the gradient inversion layer and the speaker discrimination branch, the effective separation of features and the constraint of the model are achieved. The gradient inversion layer is responsible for feature decoupling, and the speaker discrimination branch is responsible for generating constraint signals. The two work together to form a complete decoupling-constraint mechanism to guide the direction of model training.

[0113] For example, a voice wake-up model to be trained is constructed, which includes an adversarial feature decoupling branch consisting of a gradient inversion layer and a speaker discrimination branch.

[0114] S402: Construct a speech diversity training dataset and preprocess the speech signals in the speech diversity training dataset to obtain acoustic features for training.

[0115] Among them, the speech diversity training dataset refers to a dataset containing diverse speech scene samples with different accents, different speaking speeds, and different pronunciation clarity. Its purpose is to provide diverse inputs for model training and improve the model's generalization ability. The training acoustic features refer to the speech features used for model training after preprocessing.

[0116] Optionally, the principle of constructing a speech diversity training dataset is to allow the model to learn the speech feature patterns in different scenarios during training by covering samples from different speech scenarios, thereby avoiding overfitting and improving the model's generalization ability. The principle of preprocessing the speech signal is the same as the preprocessing principle in the speech wake-up method in the aforementioned embodiment. The purpose is to remove interference, unify feature scale, ensure the accuracy and consistency of acoustic features used for training, and provide reliable input for model training.

[0117] For example, a speech diversity training dataset containing different accents, speech rates, and pronunciation intelligibility is constructed. The speech signals in the dataset are preprocessed to remove environmental noise and unify feature scales to obtain acoustic features for training.

[0118] S403: Input the acoustic features used for training into the speech wake-up model to be trained, decouple the features through the gradient inversion layer to obtain the wake-up word features used for training and the speaker timbre features used for training; through the speaker discrimination branch, use the speaker timbre features used for training to constrain the speech wake-up model to be trained to perform wake-up only based on the wake-up word features.

[0119] Among them, the wake-up word features used for training refer to the features used to optimize model parameters and improve wake-up accuracy; the speaker timbre features used for training refer to the auxiliary features used to generate constraint signals and standardize the direction of model training; the core of this step is to achieve feature decoupling and model constraint to ensure the targeted nature of model training.

[0120] Optionally, the principle of feature decoupling in the gradient inversion layer in this step is that, during forward propagation, the input acoustic features used for training are not modified. Only during backward propagation, the gradient is multiplied by a negative coefficient, thereby changing the direction of gradient propagation. This achieves effective separation of the wake-up word features used for training and the speaker timbre features used for training, ensuring that the two types of features do not interfere with each other. The principle of constraint implementation in the speaker discrimination branch is that the speaker timbre features used for training are identified, a constraint signal is generated, and it acts in reverse on the model training process, forcing the model to ignore the speaker timbre features and focus only on the wake-up word features used for training. This ensures that the model training direction is consistent with the wake-up requirements, which can effectively improve the model's generalization ability.

[0121] For example, the acoustic features used for training are input into the speech wake-up model to be trained, and the features are decoupled through a gradient inversion layer to obtain the wake-up word features used for training and the speaker timbre features used for training. Through the speaker discrimination branch, the speaker timbre features used for training are used to generate a constraint signal to constrain the model to be trained to perform wake-up only based on the wake-up word features used for training, thus standardizing the training direction of the model.

[0122] S404: Optimize the model parameters of the voice wake-up model to be trained using wake-up word features to obtain a trained voice wake-up model.

[0123] Here, model parameters refer to the parameters in the model that affect wake-up performance (such as network weights, decision thresholds, etc.); a well-trained voice wake-up model refers to a model that has been optimized in terms of parameters and can stably perform wake-up decisions and adapt to diverse voice scenarios; the core of this step is to improve the wake-up accuracy of the model by training and optimizing the model parameters.

[0124] Optionally, the principle of model parameter optimization in this step is based on the backpropagation algorithm. By inputting the wake-up word features used in training into the model to be trained, the wake-up judgment error of the model is calculated. Based on the error feedback, the network parameters and judgment threshold of the model are adjusted so that the wake-up error of the model is gradually reduced and the wake-up accuracy is gradually improved.

[0125] For example, by utilizing the wake-up word features used in training, and through multiple rounds of iterative training, the network parameters and wake-up judgment threshold of the model to be trained are adjusted, the wake-up error is gradually reduced, and finally a well-trained voice wake-up model is obtained.

[0126] S405: Perform lightweight processing on the trained voice wake-up model to obtain a lightweight voice wake-up model.

[0127] Lightweight processing refers to the method of reducing the number of model parameters and computational load while ensuring that the model wake-up performance is not degraded; a lightweight voice wake-up model refers to a voice wake-up model that can adapt to the deployment requirements of embedded devices, with a small size and low computational load; the core of this step is to achieve model lightweighting to meet the needs of embedded deployment.

[0128] Optionally, the principle of lightweight processing in this step is to reduce the number of model parameters and computational load by optimizing parameters and simplifying the structure, while ensuring that the model wake-up performance remains basically unchanged. The logic is to remove irrelevant redundant modules and parameters in the model, and retain the core modules that realize feature decoupling and wake-up determination. This ensures that the model can still stably adapt to diverse speech scenarios while reducing its size and computational load, and can solve the problem that traditional models are large and cannot be adapted to embedded devices.

[0129] For example, the trained voice wake-up model is lightweighted to reduce the number of model parameters and computational load without compromising wake-up performance, resulting in a lightweight voice wake-up model that can be adapted for deployment on embedded devices.

[0130] In this embodiment, the model training is carried out by voice wake-up, realizing a closed loop from model construction and training to lightweight deployment. This not only solves the problem of poor generalization ability of traditional models, but also meets the needs of embedded deployment.

[0131] In an exemplary embodiment, the speech signals in the speech diversity training dataset are preprocessed to obtain training acoustic features, including:

[0132] S501: Perform basic preprocessing on the speech signal to obtain the basic speech signal.

[0133] Basic preprocessing refers to basic operations such as denoising, framing, and amplitude normalization on the original speech signal. Its purpose is to remove environmental noise and redundant information from the original speech signal and unify the signal format. The basic speech signal refers to the standardized speech signal after basic preprocessing.

[0134] Optionally, the basic preprocessing principle in this step is based on signal processing technology. By filtering out invalid noise and unifying signal amplitude and format, the accuracy and consistency of subsequent feature extraction are ensured. The specific principles include: removing interference noise in the environment through filtering technology and retaining the effective components of the speech signal; dividing the continuous speech signal into fixed-length frames through framing operations and utilizing the short-term stationarity of the speech signal to improve the accuracy of subsequent feature extraction; and eliminating the influence of differences in the volume of different speakers' pronunciations through amplitude normalization to ensure that the speech signals of different samples have a uniform feature scale.

[0135] For example, basic preprocessing is performed on the speech signals in the speech diversity training dataset to remove environmental noise, perform frame segmentation and amplitude normalization to obtain standardized basic speech signals.

[0136] S502: Extract the FBank features of the basic speech signal and normalize the FBank features.

[0137] Among them, FBank features refer to frequency domain features extracted based on the characteristics of human hearing. They can effectively preserve the core information of speech and are commonly used acoustic features in the training of speech wake-up models. Normalization processing refers to mapping the amplitude of FBank features to a fixed range to eliminate feature differences between different samples and ensure feature consistency.

[0138] Optionally, the principle of FBank feature extraction is based on the auditory characteristics of the human ear, simulating the sensitivity of the human ear to different frequencies. It converts the linear frequency of the speech signal into Mel frequency, then divides the frequency domain into multiple Mel-scale frequency bands, and extracts the energy of each frequency band as FBank features. This feature can effectively preserve the core information of the speech while filtering out irrelevant high-frequency or low-frequency interference, adapting to the feature requirements of diverse speech scenarios. The principle of normalization processing is based on statistical normalization theory. By calculating the mean and standard deviation of the FBank features, the feature values ​​are standardized, eliminating amplitude differences between different samples, avoiding model training bias caused by sample amplitude differences, and ensuring the stability and accuracy of model training.

[0139] For example, FBank features are extracted from the basic speech signal, and the extracted FBank features are normalized to eliminate feature differences between different samples, thus obtaining standardized FBank features.

[0140] S503: Perform data augmentation on the normalized FBank features to obtain training acoustic features that meet the needs of speech diversity.

[0141] Among them, data augmentation refers to the processing method of reasonably transforming the normalized FBank features to expand the sample diversity; training acoustic features adapted to speech diversity requirements refer to acoustic features that, after data augmentation, can cover different speech scenarios and are used for model training, and their role is to improve the generalization ability of the model.

[0142] Optionally, the principle of data augmentation in this step is to reasonably transform the features of existing training samples to generate more diverse training samples, so that the model can learn more comprehensive speech feature rules during training, avoid model overfitting, and improve the model's generalization ability. The logic is that sample expansion improves generalization ability. This transformation will not change the core information of speech (wake word), but only change the presentation form of speech, ensuring that the model can adapt to speech scenarios with different speech speeds, accents, and pronunciation clarity.

[0143] For example, data augmentation is performed on the normalized FBank features to expand sample diversity and obtain training acoustic features that meet the requirements of speech diversity for subsequent model training.

[0144] In this embodiment, the quality of the training features can be ensured by acquiring the acoustic features used for training.

[0145] In an exemplary embodiment, the voice wake-up model to be trained further includes a temporal feature extraction branch and an adaptive weighting branch; before feature decoupling through the gradient reversal layer, it also includes:

[0146] S601: Through the temporal feature extraction branch, temporal feature extraction is performed on the acoustic features used for training to obtain the temporal features used for training.

[0147] Among them, the temporal feature extraction branch is the module used to extract temporal features in the voice wake-up model to be trained; the training temporal features refer to features that contain information on the temporal dimension changes of the training acoustic features, which can more comprehensively reflect the dynamic changes of the training voice signal.

[0148] Optionally, the implementation principle of the temporal feature extraction branch in this step is based on time series analysis technology. By constructing a temporal modeling network, the acoustic features used for training are analyzed frame by frame to capture the correlation between features at different times and generate temporal features for training that contain dynamic temporal information. Its core principle is to use the temporal correlation of speech signals to integrate discrete acoustic features used for training into a feature sequence with temporal correlation, thereby avoiding model training bias caused by feature fragmentation.

[0149] S602: Through adaptive weighted branching, the training temporal features are adaptively weighted to obtain the training attention-enhanced features.

[0150] Among them, the adaptive weighted branch is the module used to implement weight allocation in the voice wake-up model to be trained; the training attention enhancement feature refers to the training feature that highlights the core features of the wake-up word and suppresses redundant interference features after weight allocation, which can improve the targeting of subsequent feature decoupling and model training.

[0151] Optionally, the implementation principle of the adaptive weighted branch in this step is based on the attention mechanism. By calculating the correlation between different dimensions of the training time-series features and the wake word, high weights are given to effective features with high correlation (such as features corresponding to the core syllables of the wake word), and low weights are given to redundant and interfering features with low correlation.

[0152] S603: Input the training attention-enhanced features into the adversarial feature decoupling branch to perform feature decoupling.

[0153] Optionally, in this step, the attention enhancement features used for training are input into the adversarial feature decoupling branch. Through the synergistic effect of the gradient inversion layer and the speaker discrimination branch in the branch, the attention enhancement features are decoupled to obtain the wake word features used for training and the speaker timbre features used for training.

[0154] In this embodiment, the quality of training features is further optimized through temporal feature extraction and adaptive weighting, which solves the problems of weak feature specificity and excessive interference in traditional model training.

[0155] In one exemplary embodiment, the trained voice wake-up model is subjected to lightweight processing to obtain a lightweight voice wake-up model, including:

[0156] S701: Perform quantization and pruning on the trained voice wake-up model to obtain a voice wake-up model with optimized parameters.

[0157] Among them, quantization refers to the process of converting floating-point parameters in the model into low-precision integer parameters; pruning refers to the process of removing redundant network structures in the model and retaining core functional modules; the parameter-optimized voice wake-up model refers to a model with reduced parameter and computational loads without degrading wake-up performance, and its purpose is to adapt to the resource limitations of embedded devices.

[0158] Optionally, the quantization process in this step is based on numerical approximation theory. While ensuring that the model's wake-up performance remains basically unchanged, floating-point parameters (such as network weights and biases) in the model are converted into low-precision integer parameters to reduce the storage and computational complexity of the parameters. The pruning process is based on the analysis of the model's network structure. It identifies and removes redundant network branches with low contribution (such as irrelevant convolutional layers and fully connected layers) while retaining modules related to feature decoupling and wake-up determination. The two work together to achieve lightweighting of the model while preserving its performance to the maximum extent, thus solving the problem of traditional models being large and unable to adapt to embedded devices.

[0159] S702: Export the file and configuration parameters of the optimized voice wake-up model to obtain a deployable lightweight voice wake-up model.

[0160] Among them, the model file refers to the file containing the core parameters and network structure of the model after lightweight optimization; the configuration parameters refer to the parameters required for model deployment and operation (such as wake-up decision boundary, constraint signal threshold, etc.); the deployable lightweight voice wake-up model refers to the voice wake-up model that can be adapted to the operating environment of embedded devices and can be directly deployed and used.

[0161] Optionally, this step is implemented by packaging and exporting the quantized and pruned model parameters and network structure together with the configuration parameters required for model operation. This adapts to the hardware resources and operating environment of embedded devices, ensuring that the model can still accurately perform wake-up determination in low-memory, low-power embedded environments, and adapt to diverse voice scenarios.

[0162] In this embodiment, lightweight processing such as quantization and pruning solves the problems of traditional models being large in size, computationally intensive, and unable to adapt to embedded devices.

[0163] In one specific embodiment of the present invention, a flowchart design for a voice wake-up model training method under conditions of speech diversity is proposed based on the above embodiments, as follows: Figure 2 As shown, it includes the following steps:

[0164] Step 1: Construct a speech diversity training dataset.

[0165] It can collect wake word speech samples covering different accents from the north and south, double-speed speech, unclear pronunciation / sound swallowing while wearing a mask, and multiple speakers. At the same time, it constructs a negative sample set of non-wake words (environmental noise, daily conversation, etc.) and divides the training set, validation set, and test set in an 8:1:1 ratio. The training set is used to iteratively optimize the model parameters, the validation set is used to adjust the hyperparameters, and the test set is used to finally evaluate the model performance.

[0166] Step 2: Extract the speech frame sequence.

[0167] The pre-emphasized speech signal is framed and Hamming windowed to reduce inter-frame signal leakage, and the framed speech frame sequence is output.

[0168] Step 3: Standardize the feature sequence.

[0169] 80-dimensional FBank features are extracted from each frame of speech. Global mean and variance normalization (CMVN) is performed on the feature sequence to eliminate feature distribution differences caused by environment and device, and generate a standardized acoustic feature sequence.

[0170] Step 4: Enhance the features of the standard feature sequence.

[0171] Multi-dimensional data augmentation of standardized feature sequences: random fundamental frequency perturbation, random fine-tuning of speech rate within the range of 0.8 to 1.2 times speed, and random selection of feature sequences from two different speakers for Mixup linear fusion to generate augmented feature sequences, thereby improving the model's generalization ability.

[0172] Step 5: Build a lightweight dilated convolutional network (TCN).

[0173] The enhanced feature sequence is input into the lightweight dilated convolutional TCN network: the network consists of three stacked temporal convolutional blocks, each containing two depthwise separable dilated convolutional layers with dilation rates of 1, 2, and 4 respectively. Gradient vanishing is avoided through residual connections, and the output is an intermediate feature sequence Xseq containing dynamic temporal information.

[0174] It should be noted that this step constructs a lightweight dilated convolutional TCN temporal modeling network for adapting to uneven speech rates. It uses depthwise separable convolution combined with 1, 2, and 4 levels of increasing dilation rate to construct a dynamic receptive field, replacing the traditional fixed window temporal feature extraction method. This achieves long-range dependency capture of speech at different rates under the premise of lightweight design.

[0175] In addition, lightweight LSTM (Long Short-Term Memory) / GRU (Gated Recurrent Unit) networks can be used. A common feature is that they can capture the temporal contextual dependencies of speech signals and adapt to speech feature extraction at different speech rates. However, lightweight LSTM / GRU still suffers from high computational complexity of the gating units and is prone to gradient vanishing in long speech sequences. In fast-paced, continuous long-temporal speech scenarios, their feature capture capability is weaker than dilated convolutional TCN, while their parameter count and computational cost are slightly higher than the dilated convolutional TCN proposed in this scheme.

[0176] Step 6: Connect the lightweight time attention module.

[0177] Input Xseq into the lightweight temporal attention module: map the features to query vector Q and key vector K=V through 1×1 convolution, calculate the scaled dot product attention weights, assign high weights to core syllable frames with clear pronunciation and low weights to blurred / swallowed frames, and output the attention-enhanced feature sequence, which is then processed by global temporal pooling to obtain the aggregated feature Xatt.

[0178] It should be noted that this step constructs a lightweight temporal attention feature enhancement module for scenarios with unclear pronunciation. It achieves adaptive allocation of speech frame weights through scaling dot product attention of 1×1 convolutional mapping, focuses on clear syllables and suppresses unclear syllables, and realizes soft alignment of speech features, replacing the traditional fixed-weight feature aggregation method.

[0179] In addition, Convolutional Block Attention Module (CBAM / SE (Squeeze-and-Excitation)) can be used. A common feature is that they can all achieve adaptive weight allocation of features, focusing on key effective features and suppressing ineffective interfering features. CBAM / SE attention mainly allocates weights based on the channel dimension, and cannot achieve precise soft alignment based on the temporal frame (syllable) dimension of speech. Its adaptability to scenarios where local temporal frame features are damaged due to unclear pronunciation is far weaker than the temporal attention mechanism of this solution, and channel attention introduces a higher computational cost.

[0180] Step 7: Build a speaker-independent feature decoupling module.

[0181] Xatt is input into the speaker-independent feature decoupling module: the main branch outputs the wake word core feature Xmain through a lightweight fully connected layer; at the same time, Xatt is input into the speaker discrimination branch through a gradient reversal layer (GRL, Gradient Reversal Layer; the gradient reversal coefficient is set to -1), and the speaker identity is predicted through two fully connected layers. The speaker discrimination loss Lspk is calculated to achieve decoupling of wake word features and speaker timbre features.

[0182] It should be noted that this step constructs an adversarial feature decoupling architecture with a gradient inversion layer for multi-speaker generalization. It consists of a main wake-up feature extraction branch and a speaker discrimination branch. The gradient inversion layer realizes reverse gradient update, forcibly decouples wake-up word features and speaker timbre features, and improves the model's cross-speaker generalization ability.

[0183] In addition, contrastive learning-based speaker feature decoupling methods can be used. A common feature of these methods is their ability to separate wake-up word features from speaker timbre features, improving the model's cross-speaker generalization ability. However, contrastive learning requires constructing a large number of positive / negative speaker sample pairs, placing high demands on training data. This makes it difficult to implement in low-resource speech wake-up scenarios. Furthermore, the training process for contrastive learning is complex, with computational overhead far exceeding that of the adversarial training architecture in this solution, hindering lightweight deployment.

[0184] Step 8: Construct the joint loss function and train the model.

[0185] The Xmain algorithm is input into the wake-up decision layer, and the predicted wake-up word probability p is output. A FAR-MR tradeoff loss function is constructed to balance the false wake-up rate (FAR) and the missed wake-up rate (MR) to improve the model's wake-up performance. The total loss is calculated by combining the speaker discrimination loss, and the model parameters are updated through backpropagation.

[0186] It should be noted that this step constructs a FAR-MR trade-off marginal loss function for accent diversity adaptation, introduces decision margin to soften hard classification boundaries, and achieves a precise trade-off between missed wake-ups and false wake-ups through differentiated MR / FAR penalty weights, increasing the penalty for accent-related missed wake-ups to replace the traditional single cross-entropy loss function.

[0187] In addition, the weighted cross-entropy loss function can be used. A common feature of these functions is that they can differentiate penalties for different error types (missed wake-ups / false wake-ups) through weight adjustment. However, weighted cross-entropy only assigns weights to sample categories, without introducing a decision margin to soften the hard classification boundary. This fails to fundamentally solve the problem of the model's strict judgment of non-standard accents, and it is difficult to achieve a precise quantitative trade-off between FAR and MR. Furthermore, it is prone to the problem that after parameter tuning, the missed wake-up rate decreases, but the false wake-up rate increases significantly.

[0188] Step 9: Lightweight model compression.

[0189] The trained model is quantized to 8-bit, non-critical layers are pruned, the number of parameters is controlled, and a deployable lightweight model file is generated.

[0190] Step 10: End training.

[0191] Export the model file and configuration parameters (threshold θ, loss weights, etc.) to the embedded device.

[0192] The above design demonstrates a lightweight, end-to-end voice wake-up architecture for deployment on low-resource devices. It integrates multi-dimensional voice data enhancement, dynamic temporal modeling, attention feature enhancement, speaker feature decoupling, and FAR-MR trade-off decision-making into a lightweight design, achieving synergistic optimization of robustness and lightweightness.

[0193] Through the aforementioned core technologies such as lightweight temporal modeling, attention feature enhancement, speaker-independent feature decoupling, and FAR-MR tradeoff loss, the present invention offers the following significant practical advantages and beneficial effects compared to traditional lightweight voice wake-up technologies based on fixed windows and static feature extraction: First, it constructs a dynamic temporal receptive field, solving the sensitivity of fixed-window models to speech rate and enabling adaptation to both fast and slow speech. Second, it achieves soft alignment of speech features and focus on key information, suppressing feature interference caused by unclear pronunciation and swallowing sounds due to masks, thus improving wake-up accuracy in ambiguous speech scenarios. Third, it softens the model's decision boundary and achieves precise FAR-MR tradeoffs, enhancing the model's inclusiveness of different accents from the north and south and reducing the missed wake-up rate caused by accent differences. Fourth, it decouples wake-up word features from speaker timbre features, completely solving the model's sensitivity to speaker changes and significantly improving the model's generalization ability in multi-speaker continuous interaction scenarios. Fifth, the lightweight architecture design ensures robustness while strictly controlling the number of model parameters and computational load, allowing direct deployment on low-resource devices such as embedded systems and IoT devices, balancing detection performance and engineering feasibility.

[0194] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0195] Based on the same inventive concept, this application also provides a voice wake-up device for implementing the voice wake-up method for speech diversity conditions as described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in the embodiments of the voice wake-up device for speech diversity conditions provided below can be found in the limitations of the voice wake-up method for speech diversity conditions described above, and will not be repeated here.

[0196] This invention also provides a voice wake-up device for conditions of voice diversity, comprising:

[0197] The signal acquisition and preprocessing module is used to acquire speech signals, preprocess the speech signals and extract acoustic features;

[0198] The voice wake-up determination module is used to determine voice wake-up based on acoustic features using a trained voice wake-up model and generate a wake-up result.

[0199] Among them, the voice wake-up model is used to decouple acoustic features into wake word features and speaker timbre features. The speaker timbre features are used to constrain the voice wake-up model to perform wake-up only based on wake word features, and the voice wake-up determination is based on wake word features.

[0200] In one embodiment, before the voice wake-up model decouples the acoustic features into wake-up word features and speaker timbre features, it further includes:

[0201] Temporal features are extracted from acoustic features to obtain temporal features;

[0202] The extracted temporal features are subjected to adaptive weighting to obtain attention-enhanced features; these attention-enhanced features are then decoupled into wake word features and speaker timbre features.

[0203] In one embodiment, performing voice wake-up determination based on wake-up word features includes:

[0204] By performing marginal optimization on the wake word features, the optimized wake decision boundary is obtained;

[0205] Based on the wake-up decision boundary, the wake-up word features are subjected to differential constraint processing to obtain a wake-up determination rule that balances the missed wake-up rate and the false wake-up rate.

[0206] The wake-up word features are used to determine the wake-up function according to the wake-up determination rules, and the corresponding wake-up result is generated.

[0207] Please see Figure 3 , Figure 3 This paper presents the design of a voice wake-up device for diverse speech conditions. The overall architecture of the device consists of five modules: a voice input and preprocessing module, a lightweight temporal feature extraction module, an attention feature enhancement module, a speaker-independent feature decoupling module, and a FAR-MR tradeoff wake-up decision module. These modules sequentially perform speech signal preprocessing, dynamic temporal feature extraction, key speech feature focusing, speaker timbre feature decoupling, and robust wake-up determination, while maintaining a lightweight model design throughout to meet the deployment requirements of embedded devices. The following sections describe each module.

[0208] Speech Input and Preprocessing Module: This module serves as the model's input layer, responsible for acquiring and standardizing the original speech signal. Simultaneously, it enhances the model's generalization ability to speech diversity and speaker variations through multi-dimensional data augmentation techniques, providing high-quality and diverse speech input for subsequent feature extraction. First, the original speech signal is acquired and subjected to conventional preprocessing such as pre-emphasis, framing, and Hamming windowing to eliminate DC components and high-frequency attenuation, reducing inter-frame redundancy. Then, a Mel filter bank is used to extract FBank features, outputting 80-dimensional Mel spectral features to replace traditional MFCC (Mel-Frequency Cepstral Coefficients) features, preserving richer speech spectral information. Global mean-variance normalization (CMVN) is also applied to the features to eliminate feature distribution differences caused by different acquisition environments and devices. To address the model's sensitivity to speaker variations and weak generalization ability, this module introduces multi-dimensional speech data augmentation, specifically including: 1) Pitch Shifting: randomly adjusting the speech fundamental frequency to simulate differences in timbre and pitch among different speakers; 2) Speed ​​Perturbation: randomly adjusting the speech speed to further enrich the speech speed variation samples and complementing the subsequent temporal modeling module; 3) Mixup: linearly fusing speech samples from two different speakers with random weights, forcing the model to ignore the timbre features of a single speaker and focus on the core speech content of the wake word. The preprocessed and augmented FBank features serve as the input feature sequence for subsequent modules, denoted as... , where T is the number of frames and F is the feature dimension.

[0209] Lightweight Temporal Feature Extraction Module: This module addresses the issue of uneven speech rate. Its core is the construction of a lightweight temporal modeling network with a dynamic receptive field, replacing the traditional fixed-window model. This enables accurate capture of the temporal contextual dependencies of speech signals at different rates. Simultaneously, the lightweight network design strictly controls the number of model parameters and computational cost. This module uses a lightweight dilated convolutional temporal convolutional network (Dilated TCN) as its core architecture. It introduces dilated convolutions into the traditional TCN and combines them with depthwise separable convolutions for lightweight modification. Furthermore, to address the vanishing gradient problem in deep network training, residual connections are introduced into each convolutional block, adding the input features and convolutional output features element-wise, followed by a non-linear transformation using the Swish activation function. Global temporal pooling is added to the last layer of the network to aggregate the temporal feature sequences, outputting intermediate features containing global temporal information, denoted as . , where C is the number of network output channels.

[0210] Attention Feature Enhancement Module: This module is designed to address the issues of "unclear pronunciation" and "slurred speech due to wearing a mask". It introduces a lightweight temporal attention mechanism to achieve soft alignment of speech features and focus on key information. It breaks the rigid matching mode of the traditional "fixed keyword template" and allows the model to adaptively give more weight to syllables with clear pronunciation and high discrimination, while ignoring unclear or damaged speech segments.

[0211] Speaker-independent feature decoupling module: Designed to address the problem of "the model being sensitive to changes in the speaker and having weak generalization ability", this module introduces a gradient inversion layer (GRL) and an adversarial training mechanism on the basis of attention enhancement features. This decouples the voice wake word features from the speaker's timbre features, allowing the main network of the model to extract only the core features related to the wake word and completely remove the speaker's identity information.

[0212] FAR-MR Trade-off Wake-up Decision Module: This module is designed to address the issue of "diverse accents". Its core is to introduce the FAR-MR trade-off loss and boundary optimization strategy to replace the traditional cross-entropy loss. This softens the decision boundary of the model, making the model more tolerant of non-standard accents. At the same time, it achieves a precise trade-off between false wake-up rate (FAR) and missed wake-up rate (MR), significantly reducing wake-up failures caused by accent differences.

[0213] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0214] Reference Figure 4 This invention also provides a computer device, including: a memory and a processor, and a computer program stored in the memory. When the computer program is executed on the processor, it performs the following steps:

[0215] Acquire speech signals, preprocess the speech signals, and extract acoustic features;

[0216] The trained voice wake-up model is used to determine voice wake-up based on acoustic features, and a wake-up result is generated.

[0217] Among them, the voice wake-up model is used to decouple acoustic features into wake word features and speaker timbre features, and uses speaker timbre features to constrain the voice wake-up model to perform voice wake-up determination only based on wake word features.

[0218] The computer device may be a desktop computer, laptop, handheld computer, or cloud server, etc. This computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 4 The examples of computer devices are merely examples and do not constitute a limitation on computer devices. They may include more or fewer components than shown in the illustration, or combinations of certain components, or different components. For example, they may also include input / output devices, network access devices, etc.

[0219] The processor referred to can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0220] In some embodiments, the memory may be an internal storage unit of the computer device, such as a hard drive or RAM. In other embodiments, the memory may be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory may include both internal and external storage units of the computer device. The memory is used to store the operating system, applications, boot loader, data, and other programs, such as the program code of the computer program. The memory can also be used to temporarily store data that has been output or will be output.

[0221] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, performs the following steps:

[0222] Acquire speech signals, preprocess the speech signals, and extract acoustic features;

[0223] The trained voice wake-up model is used to determine voice wake-up based on acoustic features, and a wake-up result is generated.

[0224] Among them, the voice wake-up model is used to decouple acoustic features into wake word features and speaker timbre features, and uses speaker timbre features to constrain the voice wake-up model to perform voice wake-up determination only based on wake word features.

[0225] In this embodiment, if the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0226] This invention provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0227] Acquire speech signals, preprocess the speech signals, and extract acoustic features;

[0228] The trained voice wake-up model is used to determine voice wake-up based on acoustic features, and a wake-up result is generated.

[0229] Among them, the voice wake-up model is used to decouple acoustic features into wake word features and speaker timbre features, and uses speaker timbre features to constrain the voice wake-up model to perform voice wake-up determination only based on wake word features.

[0230] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0231] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0232] In the embodiments disclosed in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0233] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A voice wake-up method for conditions of speech diversity, characterized in that, Includes the following steps: Acquire speech signals, preprocess the speech signals and extract acoustic features; The acoustic features are used to determine voice wake-up by a trained voice wake-up model, and a wake-up result is generated. The voice wake-up model is used to decouple the acoustic features into wake-up word features and speaker timbre features, and uses the speaker timbre features to constrain the voice wake-up model to perform voice wake-up determination only based on the wake-up word features.

2. The voice wake-up method for speech diversity conditions according to claim 1, characterized in that, Before the voice wake-up model decouples the acoustic features into wake-up word features and speaker timbre features, it also includes: Temporal features are extracted from the acoustic features to obtain temporal features; The extracted temporal features are subjected to adaptive weighting to obtain attention-enhanced features; these attention-enhanced features are used to decouple the wake word features and the speaker timbre features.

3. The voice wake-up method for speech diversity conditions according to claim 1, characterized in that, Performing voice wake-up determination based on the wake-up word features includes: The wake-up word features are subjected to marginal optimization processing to obtain the optimized wake-up decision boundary; Based on the wake-up decision boundary, the wake-up word features are subjected to differential constraint processing to obtain a wake-up determination rule that balances the missed wake-up rate and the false wake-up rate. The wake-up word features are used to determine the wake-up effect according to the wake-up determination rules, and a corresponding wake-up result is generated.

4. A method for training a voice wake-up model under conditions of speech diversity, characterized in that, Includes the following steps: A trainable speech wake-up model is constructed, which includes an adversarial feature decoupling branch; the adversarial feature decoupling branch includes a gradient inversion layer and a speaker discrimination branch; Construct a speech diversity training dataset and preprocess the speech signals in the speech diversity training dataset to obtain acoustic features for training. The training acoustic features are input into the speech wake-up model to be trained, and the features are decoupled through the gradient inversion layer to obtain the training wake-up word features and the training speaker timbre features. Through the speaker discrimination branch, the training speaker timbre features are used to constrain the speech wake-up model to be trained to perform wake-up only based on the wake-up word features. The model parameters of the voice wake-up model to be trained are optimized using the wake-up word features to obtain a trained voice wake-up model; The trained voice wake-up model is then lightweighted to obtain a lightweight voice wake-up model.

5. The method for training a voice wake-up model under conditions of speech diversity according to claim 4, characterized in that, The speech signals in the speech diversity training dataset are preprocessed to obtain training acoustic features, including: The speech signal is preprocessed to obtain the basic speech signal; Extract the FBank features of the basic speech signal and normalize the FBank features; The normalized FBank features are subjected to data augmentation processing to obtain the training acoustic features adapted to the needs of speech diversity.

6. The method for training a voice wake-up model under conditions of speech diversity according to claim 4, characterized in that, The voice wake-up model to be trained also includes a temporal feature extraction branch and an adaptive weighted branch; Before feature decoupling via the gradient inversion layer, the following is also included: Through the temporal feature extraction branch, temporal features are extracted from the acoustic features used for training to obtain the temporal features used for training. The training temporal features are adaptively weighted through the adaptive weighting branch to obtain the training attention enhancement features; The training attention-enhanced features are input into the adversarial feature decoupling branch to perform feature decoupling.

7. The method for training a voice wake-up model under conditions of speech diversity according to claim 4, characterized in that, The trained voice wake-up model is then lightweighted to obtain a lightweight voice wake-up model, which includes: The trained voice wake-up model is quantized and pruned to obtain a voice wake-up model with optimized parameters. Export the files and configuration parameters of the optimized voice wake-up model to obtain a deployable lightweight voice wake-up model.

8. A voice wake-up device for conditions of diverse speech, characterized in that, include: The signal acquisition and preprocessing module is used to acquire speech signals, preprocess the speech signals and extract acoustic features; The voice wake-up determination module is used to determine the voice wake-up status of the acoustic features using a trained voice wake-up model and generate a wake-up result. The voice wake-up model is used to decouple the acoustic features into wake-up word features and speaker timbre features. The speaker timbre features are used to constrain the voice wake-up model to perform wake-up only based on the wake-up word features, and the voice wake-up determination is made based on the wake-up word features.

9. A computer device, characterized in that, The device includes a processor and a memory: The memory is used to store computer programs and send the instructions of the computer programs to the processor; The processor executes a voice wake-up method for speech diversity conditions as described in any one of the claims 1-3 according to the instructions of the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements a voice wake-up method for speech diversity conditions as described in any one of claims 1-3.

Images