A keyword wake-up method and device for an intelligent terminal device, and a storage medium

Abstract

This invention discloses a keyword wake-up method, device, and storage medium for intelligent terminal devices. The method includes: executing a keyword registration process, encoding multiple speech samples of a new keyword into sample features using a pre-trained feature extraction model with fixed parameters, and then calculating and storing baseline prototype features representing the keyword; executing a real-time wake-up recognition process, continuously encoding real-time audio input into real-time input features, and comparing the distance with all locally stored baseline prototype features; triggering the terminal wake-up function when the comparison result meets a preset wake-up judgment condition. This invention achieves rapid and efficient keyword self-learning on low-resource devices such as microcontrollers by using a fixed feature extraction model and incrementally registering prototype features. Users only need a small number of samples to complete new word registration, offering advantages such as low resource consumption, high flexibility, and high accuracy.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a keyword wake-up method, device, and storage medium for a smart terminal device. Background Technology

[0002] With the rapid development of information technology, intelligent terminal devices integrating display, touch, and computing functions, such as smart speakers and wearable devices, have been widely used in various scenarios including education, meetings, homes, and vehicles. Voice wake-up, as a crucial entry point for human-computer interaction, greatly enhances the convenience and naturalness of the user experience.

[0003] However, in many applications, especially for power- and cost-sensitive embedded devices, voice wake-up functionality is typically deployed on resource-constrained microcontroller units (MCUs). These hardware platforms often face strict resource constraints, and existing microcontroller-based keyword wake-up solutions mostly employ a fixed model approach, supporting only a few preset wake-up words (such as "Hello, Xiao X"). When users want to add new, personalized wake-up words (such as device names, user names, or specific commands), the traditional process is quite complex. It usually requires users to record and upload a large number of voice samples to a cloud server, where the manufacturer uses high-performance computing resources, such as graphics processing units (GPUs), to retrain the entire acoustic model before finally distributing the new model to the terminal device. This entire process not only takes days or even weeks but also heavily relies on network connectivity and cloud services, failing to meet users' needs for immediate, localized custom wake-up words.

[0004] Therefore, existing keyword wake-up technologies generally have a technical bottleneck: on low-resource terminal devices that meet stringent constraints such as low power consumption, low computing power, and small memory, it is difficult to achieve flexible, fast, and cloud-free incremental keyword registration and self-learning functions. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a keyword wake-up method, device, and storage medium for smart terminal devices.

[0006] In a first aspect, the present invention provides a keyword wake-up method for a smart terminal device, comprising: The keyword registration process is executed to obtain multiple speech samples corresponding to the new keyword. Each speech sample is encoded into a sample feature by a pre-trained feature extraction model. A baseline prototype feature is calculated based on the multiple sample features. The baseline prototype feature is then associated with the new keyword and stored locally. The real-time wake-up recognition process is executed, audio input information is continuously collected, the audio input information is encoded using the pre-trained feature extraction model to obtain real-time input features, the real-time input features are compared with all the benchmark prototype features stored locally, and the comparison result is determined to meet the preset wake-up judgment conditions to trigger the terminal wake-up function. The parameters of the pre-trained feature extraction model remain fixed after it is deployed on the terminal. Adding new keywords does not require retraining the model; incremental registration is completed simply by adding and storing the corresponding prototype features.

[0007] In some embodiments, calculating the baseline prototype features based on the features of the multiple samples includes: Calculate the average feature of the multiple sample features, and use the average feature as the baseline prototype feature.

[0008] In some embodiments, comparing the real-time input features with all locally stored baseline prototype features includes: Calculate the cosine distance between the real-time input feature and each of the benchmark prototype features in the set of all benchmark prototype features.

[0009] In some embodiments, determining that the comparison result meets the preset wake-up judgment condition to trigger the terminal wake-up function includes: When the cosine distance between the real-time input feature and any locally stored benchmark prototype feature is less than a preset distance threshold, the terminal wake-up function is triggered.

[0010] In some embodiments, the method further includes: An acoustic feature extraction network is pre-constructed and trained offline using a triplet loss function to obtain the pre-trained feature extraction model.

[0011] In some embodiments, the number of network parameters of the acoustic feature extraction network is less than 50K.

[0012] In some embodiments, obtaining multiple voice samples corresponding to the newly added keyword includes: At least 2 to 10 speech samples are collected as the multiple speech samples.

[0013] In some embodiments, the method is operated on a smart terminal device equipped with a microcontroller, wherein the on-chip static random access memory of the microcontroller is less than 1MB.

[0014] Secondly, the present invention also provides a smart terminal device, comprising: The registration module is used to obtain multiple speech samples corresponding to the new keywords, encode each speech sample into sample features through a pre-trained feature extraction model, and generate and store the baseline prototype features corresponding to the new keywords locally based on the multiple sample features. The wake-up module is used to continuously collect audio input information, encode the audio input information using the pre-trained feature extraction model to obtain real-time input features, compare the real-time input features with all the benchmark prototype features stored locally, and determine whether the comparison result meets the preset wake-up judgment conditions to trigger the terminal wake-up function. The parameters of the pre-trained feature extraction model remain fixed after it is deployed on the terminal. Adding new keywords does not require retraining the model; incremental registration is completed simply by adding and storing the corresponding prototype features.

[0015] Thirdly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, it performs the following steps: The process involves executing a keyword registration process, acquiring multiple speech samples corresponding to the new keyword, encoding each speech sample into sample features using a pre-trained feature extraction model, generating and locally storing baseline prototype features corresponding to the new keyword based on the multiple sample features, and executing a real-time wake-up recognition process. This process involves continuously collecting audio input information, encoding the audio input information using the pre-trained feature extraction model to obtain real-time input features, comparing the real-time input features with all locally stored baseline prototype features, and determining if the comparison result meets the preset wake-up judgment conditions to trigger the terminal wake-up function. The parameters of the pre-trained feature extraction model remain fixed after deployment on the terminal, and no retraining is required for the model after adding a new keyword; incremental registration is completed simply by adding and storing the corresponding prototype features.

[0016] The technical solution provided by the embodiments of the present invention has the following advantages compared with the prior art: First, this invention achieves incremental self-learning of keywords on low-resource terminal devices. By employing a strategy that combines a fixed pre-trained feature extraction model with incremental registration baseline prototype features, users can directly add custom wake words on the device without the need for cloud server involvement or time-consuming retraining of the neural network model, greatly improving the flexibility and response speed of the function. Second, the registration process is efficient and convenient, completed within seconds, providing a good user experience. Third, it has low resource consumption and excellent performance, for example, it uses a lightweight acoustic feature extraction network with fewer than 50K network parameters. This method can be successfully deployed on microcontrollers with less than 1MB of SRAM (Static Random Access Memory), and supports the registration of multiple custom wake words without increasing the inference and recognition time, while maintaining a low false wake-up rate. Finally, the technical solution of this invention has wide applicability and is not limited to specific types of smart terminal devices. It can be widely applied to all scenarios requiring low power consumption and customizable keyword wake-up functions, such as smart home central control, in-vehicle devices, and wearable devices. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating a keyword wake-up method for a smart terminal device provided by the present invention; Figure 2 This is a schematic diagram of the structure of a smart terminal device provided by the present invention. Detailed Implementation

[0020] To better understand the above-mentioned objectives, features, and advantages of the present invention, the solutions of the present invention will be further described below. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0021] Many specific details are set forth in the following description in order to provide a full understanding of the invention, but the invention may also be practiced in other ways different from those described herein; obviously, the embodiments in the specification are only some embodiments of the invention, and not all embodiments.

[0022] In the current technological context, keyword wake-up technology applied to smart terminal devices, especially for implementing user-defined keywords on resource-constrained microcontrollers, faces severe challenges. On the one hand, the computing resources and storage space of microcontrollers, such as on-chip static random access memory (SRAM) less than 1MB, are extremely limited and cannot support complex model training or large model libraries. On the other hand, traditional keyword customization processes require uploading user voice samples to the cloud for retraining over several days or even weeks, which is complex, slow, and cannot meet users' needs for adding new wake-up words instantly. This invention aims to solve these problems by proposing an innovative keyword wake-up method. Its core idea lies in adopting a "one-time offline training, incremental deployment everywhere" model. This model combines a pre-trained feature extraction model with fixed parameters after deployment with a baseline prototype feature registration mechanism that only requires adding and storing a small amount of data. This allows for the rapid and lightweight incremental addition of keywords on low-resource terminal devices without the need for model retraining.

[0023] To facilitate understanding, the core terms involved in this invention are first defined. A pre-trained feature extraction model refers to a deep neural network model pre-trained on a large-scale dataset in the cloud or on a server. Its function is to map the input speech signal (or its acoustic features) to a low-dimensional feature space with good metric properties. Once deployed on a smart terminal device, the network weights and other parameters of this model remain fixed and are not adjusted or updated. Sample features refer to the output obtained after inputting a single speech sample, such as a user-recorded keyword speech, into the aforementioned pre-trained feature extraction model. This output is typically represented as a fixed-dimensional feature vector. Baseline prototype features are the "standard" feature vectors representing a specific keyword. They are generated by statistically calculating (e.g., averaging) multiple sample features corresponding to multiple speech samples of the keyword, and are used to uniquely identify the keyword in the feature space.

[0024] This invention provides a keyword wake-up method for smart terminal devices, the overall process of which can be divided into two main stages: S110 keyword registration process and S120 real-time wake-up recognition process. These two processes share the same pre-trained feature extraction model, whose parameters remain constant after deployment, which is key to achieving retraining-free incremental registration. By fixing the model parameters, the consistency and stability of the feature space are guaranteed, allowing meaningful comparisons of the feature vectors of new and old keywords in the same metric space. This design avoids the costly retraining of the entire model for each new keyword, greatly improving the efficiency and flexibility of user-defined wake-up words.

[0025] Specifically, when a user needs to add a new wake word, the system first executes a keyword registration process. The first step of this process is to acquire multiple voice samples corresponding to the new keyword. For example, when the user pronounces the keyword several times, the system sequentially feeds each voice sample into a fixed pre-trained feature extraction model for encoding, thus converting each voice sample into a high-dimensional sample feature vector. After obtaining multiple sample feature vectors, the system calculates a baseline prototype feature based on these vectors, representing the core features of the keyword. Finally, the system associates this newly generated baseline prototype feature with the corresponding new keyword text or identifier and stores it in the device's local non-volatile memory. Thus, incremental registration of new keywords is completed by simply adding the storage of a baseline prototype feature; the entire process is fast and consumes very few device resources.

[0026] After registering one or more keywords, the device enters a real-time wake-up recognition process, continuously waiting for a wake-up command. During this process, the device's audio acquisition unit (such as a microphone array) continuously collects audio input information from the environment. The system uses the same pre-trained feature extraction model as in the registration process to frame and encode the real-time audio input information, thus obtaining a real-time input feature vector. Next, the system compares this real-time input feature vector with all locally stored baseline prototype features (including preset ones and those subsequently registered incrementally by the user). This comparison process aims to measure the similarity between the real-time input audio and each registered keyword in the feature space. When the comparison result meets the preset wake-up criteria, the system triggers the terminal wake-up function, such as turning on the screen, launching a specific application, or performing subsequent voice interaction. This wake-up criteria is key to achieving high-precision wake-up and a low false wake-up rate. By setting a reasonable judgment standard, it can be ensured that the wake-up function is triggered only when the real-time input audio is highly similar to a certain keyword, while effectively filtering out environmental noise, irrelevant human voices, and other interference. Since the entire real-time recognition process only involves feature extraction and distance calculation, the computational load is much smaller than that of model inference. Therefore, it can achieve extremely low power consumption and extremely fast response speed, making it very suitable for continuous operation on low-power microcontrollers.

[0027] Based on the above scheme, as a preferred implementation, a baseline prototype feature is calculated based on the features of multiple samples. This includes calculating the average feature of the multiple sample features and using this average feature as the baseline prototype feature. For example, if N speech samples are collected, resulting in N sample feature vectors {v1, v2, ..., vN}, then the baseline prototype feature P can be calculated using the formula P = (1 / N) * Σ(vi). Using the average value as the prototype can, on the one hand, smooth out noise and pronunciation differences in individual samples to a certain extent, extracting more common features; on the other hand, the calculation process is very simple, adding almost no additional computational overhead, and fully meeting the requirements for operation on resource-constrained devices.

[0028] Furthermore, the step of comparing the real-time input features with all locally stored baseline prototype features can be specified as calculating the cosine distance between the real-time input features and each of the baseline prototype features. The cosine distance measures the difference in direction between two vectors, with a value ranging from -1 to 1 (or 0 to 2 after transformation). Compared to Euclidean distance, the cosine distance is insensitive to the absolute magnitude of the vectors, focusing more on their directional similarity. This often yields better results in speech feature matching because it better handles the impact of volume variations. By calculating the cosine distance, the complex feature vector similarity problem can be transformed into an intuitive scalar value comparison problem, simplifying the subsequent decision logic. It should be noted that the cosine distance provided in this invention is only one optional implementation method; other similar comparison methods can also be used, and no specific limitation is made.

[0029] As a further refinement of the above comparison method, the terminal wake-up function is triggered when the comparison result meets a preset wake-up judgment condition. Specifically, this can be set as follows: the terminal wake-up function is triggered when the cosine distance between the real-time input feature and any locally stored benchmark prototype feature is less than a preset distance threshold. This distance threshold is a key parameter that needs to be preset or adaptively adjusted based on the actual application scenario and performance requirements, such as the trade-off between recall and precision. For example, a small threshold such as 0.25 can be set, meaning that the system will only determine a successful match when the angle between the real-time input feature vector and a benchmark prototype feature vector is very small, i.e., their directions are very close. This design achieves the quantification and standardization of wake-up judgment through a clear numerical threshold, making the sensitivity and accuracy of wake-up controllable.

[0030] For example, to obtain a high-performance pre-trained feature extraction model, the method of this invention also includes an important offline training stage. In this stage, an acoustic feature extraction network needs to be pre-constructed and trained offline using a triplet loss function to finally obtain the pre-trained feature extraction model. The triplet loss function is an effective method for metric-oriented learning. Its training data consists of triplets of (anchor samples, positive samples, and negative samples), where the anchor and positive samples belong to the same category (such as the same keyword), and the negative samples belong to different categories. The training objective is to minimize the distance between the anchor and positive samples and maximize the distance between the anchor and negative samples in the feature space output by the model, while maintaining a sufficiently large interval between them. The model trained in this way can learn a highly discriminative feature space, causing similar samples to cluster in the space and dissimilar samples to stay away from each other, which lays a solid foundation for subsequent wake-up recognition based on distance comparison.

[0031] To enable successful deployment of this method on resource-constrained microcontrollers, a specific implementation approach is to use an acoustic feature extraction network with fewer than 50K parameters, thus achieving a lightweight acoustic feature extraction network. Under this constraint, the network structure must be highly optimized and compact. For example, techniques such as depthwise separable convolution, grouped convolution, and parameter sharing can be used to reduce the number of parameters and computational cost, or a compact structure based on recurrent neural networks (RNNs) or convolutional recurrent neural networks (CRNNs) can be employed. The resulting model, for example, one with only 48K parameters, has minimal storage requirements and sufficiently low inference computation, enabling it to run smoothly on typical low-power microcontrollers and meet real-time processing requirements.

[0032] In practical applications, to ensure registration accuracy and improve user experience, when acquiring multiple voice samples corresponding to new keywords, the number can be limited to 2 to 10 voice samples. This range is based on the results of balancing experimental data. Too few samples, such as only one, may lead to overly randomness in the generated baseline prototype features, resulting in poor generalization ability and susceptibility to single pronunciation abnormalities. Conversely, too many samples increase the user's operational burden and reduce the convenience of the registration process. Experiments show that sampling 2-10 voice samples is suitable, while 3 to 5 voice samples can achieve better results. This usually strikes a good balance between convenience and accuracy, generating a stable and representative baseline prototype feature. Therefore, this invention enables registration with only a few samples.

[0033] The method proposed in this invention is targeted at intelligent terminal devices equipped with microcontrollers, especially in scenarios with strict requirements on power consumption and cost. For example, a typical operating environment is one where the microcontroller's on-chip static random access memory (SRAM) is less than 1MB, such as the common 512KB or less. Under such hardware conditions, traditional methods requiring the storage of multiple complete models or online training are completely infeasible. The advantage of this invention lies in the fact that its main memory consumption comes from storing baseline prototype features. Assuming each feature vector is a 128-dimensional floating-point number (i.e., 512 bytes), the resource consumption for storing a single baseline prototype feature is minimal. Even with support for up to 20 custom wake words, the total storage footprint is only about 10KB, well within the capacity limits of the microcontroller's SRAM. This design fully demonstrates a deep understanding and collaborative optimization of the resource constraints of embedded platforms.

[0034] To more clearly illustrate the technical solution of the present invention, a detailed description is provided below with reference to specific embodiments. The example of adding the new wake-up word "Hello Big Screen" to an interactive flat panel as a smart terminal device is used for illustration. (See reference...) Figure 1 The entire process consists of two core steps: executing the keyword registration process S110 and executing the real-time wake-up recognition process S120. When the judgment conditions of the recognition process are met, the terminal wake-up function will be triggered.

[0035] In the keyword registration process, the user is first guided to say "Hello Big Screen" three times. The system collects these three pronunciations through a microphone array, forming three independent speech samples. Subsequently, a pre-trained feature extraction model deployed on the microcontroller of the smart terminal device processes these three speech samples sequentially. The model encodes each high-dimensional raw speech signal into a low-dimensional, dense sample feature vector, such as a 128-dimensional floating-point vector. In this way, three sample feature vectors are generated for each of the three speech samples. Next, the system processes these three sample feature vectors to generate a baseline prototype feature representing the keyword "Hello Big Screen". In this embodiment, the system calculates the arithmetic mean of these three sample feature vectors to obtain a unique 128-dimensional average vector. This average vector is used as the baseline prototype feature of "Hello Big Screen". Finally, the system stores this baseline prototype feature and the identifier of the keyword "Hello Big Screen" together in the non-volatile storage area (such as Flash) of the microcontroller for subsequent recognition. At this point, the registration process for new keywords is complete, without requiring an internet connection or any modifications to the pre-trained model.

[0036] In the real-time wake-up recognition process, the microphone array of the smart terminal device continuously collects ambient sound, forming a continuous audio stream. The smart terminal device processes this audio stream in real time, performing preprocessing operations such as frame segmentation, pre-emphasis, and windowing. Further, the smart terminal device uses the same pre-trained feature extraction model as in the registration process to encode the audio data within each frame or a sliding window, obtaining a real-time input feature vector. This process is continuous, ensuring the device can respond to wake-up words that may appear at any time. Wake-up decision is the core of the recognition process. The smart terminal device compares the real-time generated input feature vector with all baseline prototype features stored in the non-volatile storage area (here, the baseline prototype feature of "Hello Big Screen"). In this embodiment, the comparison method is to calculate the cosine distance between them. If the calculated cosine distance value is less than a preset distance threshold (e.g., 0.2), the system determines that the real-time input audio highly matches the keyword "Hello Big Screen." At this time, the comparison result meets the preset wake-up judgment condition, and the terminal wake-up function is triggered, such as turning on the smart terminal device's screen and preparing to receive the user's next command.

[0037] The present invention also provides a smart terminal device capable of performing the above-described method. Figure 2 This invention provides a structural schematic diagram of a smart terminal device, as shown below. Figure 2 As shown, the smart terminal device includes a registration module 21 and a wake-up module 22. The registration module 21 is responsible for executing the aforementioned keyword registration process. When a user requests to add a new keyword, this module is activated to acquire multiple voice samples corresponding to the new keyword, call a pre-trained feature extraction model to encode each voice sample into sample features, and generate and locally store baseline prototype features corresponding to the new keyword based on these sample features. The wake-up module 22 is responsible for executing the real-time wake-up recognition process. This module runs continuously in the background whether the device is in standby or running, continuously collecting audio input information and encoding the audio information using the same pre-trained feature extraction model to obtain real-time input features. Subsequently, the wake-up module 22 compares the real-time input features with all the baseline prototype features stored in the memory, and determines whether to trigger the terminal wake-up function based on whether the comparison result meets the preset wake-up judgment conditions. The parameters of the pre-trained feature extraction model remain fixed after deployment on the terminal; adding a new keyword does not require retraining the model, but only incremental registration is completed by adding and storing the corresponding prototype features. These two modules work together to constitute the incremental keyword self-learning wake-up system proposed in this invention.

[0038] The internal logical architecture of the smart terminal device also includes a processor and a memory. The memory stores program instructions and data, including the operating system, applications, and the model and benchmark prototype features required by the method of this invention. The processor is the computing core of the entire device, responsible for executing the instructions in the memory. Software modules that implement the core functions of this invention run on the processor. Additionally, audio input information can be acquired via a microphone or microphone array.

[0039] Exemplarily, the present invention also provides a deployment method on an embedded processor. Modern smart terminal devices employ a heterogeneous multi-core System on a Chip (SoC) as their main processor, such as an architecture comprising a high-performance application processor (AP) and a low-power microcontroller. For example, this architecture includes a microcontroller, an application processor, shared memory (Dynamic Random-Access Memory (DRAM) / Flash Memory), and a microphone array. The relatively low-computational-intensive but continuously running wake-up task is deployed on the extremely low-power microcontroller. Specifically, a pre-trained feature extraction model, trained and optimized offline, is stored in the microcontroller's on-chip Flash memory. Baseline prototype features generated by the user through the registration process are also stored in the microcontroller's SRAM or Flash memory. In standby mode, the high-performance application processor can enter a deep sleep mode, leaving only the microcontroller active. The microcontroller continuously listens for ambient sounds through the microphone array and executes a real-time wake-up recognition process. This deployment method significantly reduces the standby power consumption of the device, enabling ultra-low power standby wake-up functionality, which is crucial for battery-powered mobile devices or devices that need to meet high energy efficiency standards.

[0040] Furthermore, as a preferred implementation of this smart terminal device, the pre-trained feature extraction model used is an acoustic feature extraction network trained offline using a triplet loss function. The network has fewer than 50K parameters, making it a lightweight acoustic feature extraction network. Therefore, this device not only optimizes power consumption in its deployment architecture, but its core algorithm model also boasts advantages of high accuracy and low resource consumption. Training with the triplet loss function ensures that the feature space output by the model has good metric properties, enabling simple cosine distance calculation to efficiently and accurately complete the wake-word recognition task. This achieves the technical effects of the invention synergistically at both the hardware and software levels.

[0041] To more comprehensively demonstrate the preferred embodiments of the present invention, a complete embodiment incorporating the aforementioned technical features is described below. This embodiment aims to implement a complete system that runs on a low-resource microcontroller, supports users in quickly customizing multiple wake words, and maintains a high recognition rate and a low false wake-up rate.

[0042] For example, during the offline training phase, a large-scale speech dataset containing thousands of hours of data, covering various accents, speech rates, and environmental noise, is used. Based on this dataset, hundreds of millions of (anchor, positive, negative) sample triples are constructed. A lightweight convolutional recurrent neural network (CRNN) is designed as a lightweight acoustic feature extraction network, and through meticulous structural design and parameter optimization, its total number of parameters is controlled to 48K. Furthermore, using the constructed triple dataset, the CRNN network is fully trained with the triple loss function as the optimization objective until the model converges. The trained model is our pre-trained feature extraction model, which is quantized and embedded into the microcontroller of the target device.

[0043] Secondly, during the registration phase of the terminal device, when a user wants to add a new wake word (e.g., "start meeting"), the device prompts the user to pronounce the word clearly and at a steady pace 3 to 5 times. The system collects these 3 to 5 voice samples and encodes them into 128-dimensional sample feature vectors using a pre-trained feature extraction model embedded in the microcontroller. Subsequently, the average vector of these 3 to 5 feature vectors is calculated, and this average vector is used as the baseline prototype feature for the keyword "start meeting" and stored in the microcontroller's Flash memory. Since each baseline prototype feature only occupies about 512 bytes, even if the user registers 20 different wake words, the total space occupied is only about 10KB, which is well within the capacity of a microcontroller with 512KB of SRAM.

[0044] Furthermore, in the real-time recognition phase, the method runs on a microcontroller configured with 512KB of SRAM. The microcontroller continuously processes the audio stream captured by the microphone, extracting a real-time input feature every frame or at short intervals. Then, the microcontroller sequentially calculates the cosine distance between this real-time input feature and all 20 registered baseline prototype features. This calculation process is highly optimized and can be completed in a very short time. For example, with a global distance threshold set to 0.25, if in a calculation the cosine distance between the real-time input feature and any of the baseline prototype features is found to be less than 0.25, a valid wake-up event can be considered to have occurred, and the main application processor is immediately awakened, simultaneously transmitting the wake-up keyword information.

[0045] The method provided by this invention supports incremental registration of at least 20 custom wake words on the microcontroller side, and the process of registering new words does not increase inference time. This is because each recognition only increases the number of distance calculations, while the computational load of feature extraction, the main time-consuming step, remains constant. Furthermore, adding a new wake word requires only 3 to 5 valid voice samples from the user on average, making the registration process extremely convenient. In a typical scenario where three wake words are used concurrently, after 24 hours of continuous testing, the system's false wake-up rate was less than 1, demonstrating extremely high accuracy and robustness.

[0046] This invention also provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it can implement the steps of the aforementioned keyword wake-up method. Specifically, the program instructions guide the processor to execute a keyword registration process, including acquiring multiple speech samples corresponding to the new keyword, encoding each speech sample into sample features using a pre-trained feature extraction model, and generating and locally storing baseline prototype features corresponding to the new keyword based on the multiple sample features. Simultaneously, the program instructions also guide the processor to execute a real-time wake-up recognition process, including continuously collecting audio input information, encoding it using the same pre-trained feature extraction model to obtain real-time input features, comparing the real-time input features with all locally stored baseline prototype features, and finally triggering the terminal wake-up function when the preset wake-up judgment conditions are met. This storage medium can be any form of non-volatile or volatile memory, such as read-only memory (ROM), random-access memory (RAM), flash memory, hard disk, etc.

[0047] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0048] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0049] The above are merely specific embodiments of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A keyword wake-up method for a smart terminal device, characterized in that, include: The keyword registration process is executed to obtain multiple speech samples corresponding to the new keyword. Each speech sample is encoded into a sample feature by a pre-trained feature extraction model. A baseline prototype feature is calculated based on the multiple sample features. The baseline prototype feature is then associated with the new keyword and stored locally. The real-time wake-up recognition process is executed, audio input information is continuously collected, the audio input information is encoded using the pre-trained feature extraction model to obtain real-time input features, the real-time input features are compared with all the benchmark prototype features stored locally, and the comparison result is determined to meet the preset wake-up judgment conditions to trigger the terminal wake-up function. The parameters of the pre-trained feature extraction model remain fixed after it is deployed on the terminal. Adding new keywords does not require retraining the model; incremental registration is completed simply by adding and storing the corresponding prototype features.

2. The method according to claim 1, characterized in that, The baseline prototype features calculated based on the features of the multiple samples include: Calculate the average feature of the multiple sample features, and use the average feature as the baseline prototype feature.

3. The method according to claim 1, characterized in that, The step of comparing the real-time input features with all locally stored benchmark prototype features includes: Calculate the cosine distance between the real-time input feature and each of the benchmark prototype features in the set of all benchmark prototype features.

4. The method according to claim 3, characterized in that, The determination that the comparison result meets the preset wake-up judgment condition to trigger the terminal wake-up function includes: When the cosine distance between the real-time input feature and any locally stored benchmark prototype feature is less than a preset distance threshold, the terminal wake-up function is triggered.

5. The method according to claim 1, characterized in that, The method further includes: An acoustic feature extraction network is pre-constructed and trained offline using a triplet loss function to obtain the pre-trained feature extraction model.

6. The method according to claim 5, characterized in that, The number of network parameters of the acoustic feature extraction network is less than 50K.

7. The method according to claim 1, characterized in that, The process of obtaining multiple voice samples corresponding to the newly added keywords includes: At least 2 to 10 speech samples are collected as the multiple speech samples.

8. The method according to claim 1, characterized in that, The method operates on a smart terminal device equipped with a microcontroller, wherein the on-chip static random access memory of the microcontroller is less than 1MB.

9. A smart terminal device, characterized in that, include: The registration module is used to obtain multiple speech samples corresponding to the new keywords, encode each speech sample into sample features through a pre-trained feature extraction model, and generate and store the baseline prototype features corresponding to the new keywords locally based on the multiple sample features. The wake-up module is used to continuously collect audio input information, encode the audio input information using the pre-trained feature extraction model to obtain real-time input features, compare the real-time input features with all the benchmark prototype features stored locally, and determine whether the comparison result meets the preset wake-up judgment conditions to trigger the terminal wake-up function. The parameters of the pre-trained feature extraction model remain fixed after it is deployed on the terminal. Adding new keywords does not require retraining the model; incremental registration is completed simply by adding and storing the corresponding prototype features.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it performs the following steps: The keyword registration process is executed to obtain multiple speech samples corresponding to the new keyword. Each speech sample is encoded into a sample feature by a pre-trained feature extraction model. Based on the multiple sample features, a baseline prototype feature corresponding to the new keyword is generated and stored locally. The system executes a real-time wake-up recognition process, continuously collects audio input information, encodes the audio input information using the pre-trained feature extraction model to obtain real-time input features, compares the real-time input features with all locally stored baseline prototype features, and determines whether the comparison result meets the preset wake-up judgment conditions to trigger the terminal wake-up function; wherein, the parameters of the pre-trained feature extraction model remain fixed after deployment on the terminal, and adding new keywords does not require retraining the model, but only completing incremental registration by adding and storing the corresponding prototype features.

Images