Custom wake-word recognition method, apparatus and device based on dual-mode centroid

Abstract

The present application relates to the technical field of speech recognition, and particularly relates to a self-defined wake-up word recognition method, device and equipment based on a dual-mode centroid. The method is based on a dual-mode centroid mechanism, similarity calculation is performed on a to-be-recognized vector and a dynamic drift centroid by obtaining the to-be-recognized vector, a wake-up similarity is obtained, and it is determined whether the wake-up similarity is greater than a preset threshold, to determine whether the to-be-recognized vector can be used to update the dynamic drift centroid. The dynamic drift centroid is initially a static anchor centroid, supports adaptive updating based on effective wake-up speech, and introduces a callback mechanism to prevent the dynamic drift centroid from deviating too much. The technical solution combines the safety of the static anchor and the adaptability of the dynamic drift centroid, and realizes identity-aware personalized recognition. The present application effectively improves the accuracy and robustness of wake-up recognition, reduces the false wake-up rate, enhances the adaptability of the system to user pronunciation changes and environmental noise, improves the user experience, and is suitable for scenarios such as smart speakers, wearable devices and other voice-triggered scenarios.

Description

Technical Field

[0001] This invention relates to the field of speech recognition, and in particular to a method, apparatus, and device for recognizing custom wake words based on dual-mode centroids. Background Technology

[0002] With the development of artificial intelligence technology, the demand for Custom Keyword Spotting (KWS) technology is growing. Users expect to be able to activate their devices with any phrase they set themselves, rather than using fixed wake words preset by the manufacturer. The key to achieving this goal is zero-shot or few-shot learning capability, meaning the system does not need to retrain the model for each new keyword. Currently, the mainstream custom KWS technology approach is to convert audio segments into fixed-dimensional embeddings using deep learning models, and then determine whether to activate the device by comparing the similarity between the embedding vector of the current input audio and the embedding vector of the user's registered keywords (i.e., the "registered centroid").

[0003] User registration is a one-time process; however, a user's pronunciation habits, tone of voice, speaking speed, and even their environment change dynamically over time. A static registration centroid generated under ideal conditions will have diminishing representativeness over time, leading to lower wake-up rates or higher false wake-up rates. Existing solutions lack a mechanism that allows the model to self-evolve on the device and dynamically adapt to changes in users and the environment. Summary of the Invention

[0004] The main objective of this invention is to provide a custom wake word recognition method based on dual-mode centroids, which aims to solve the technical problem that the representativeness of the registered centroids generated in a static environment decreases over time, leading to a decrease in wake-up rate or an increase in false wake-up rate.

[0005] To achieve the aforementioned objectives, the first aspect of this invention proposes a custom wake-word recognition method based on a dual-mode centroid, the method comprising:

[0006] The received speech is encoded using an encoder to obtain the vector to be recognized;

[0007] The similarity between the vector to be identified and the dynamic drift centroid is calculated to obtain the wake-up similarity, wherein the initial value of the dynamic drift centroid is a preset static anchor centroid;

[0008] Determine whether the wake-up similarity is greater than a first similarity threshold;

[0009] If the wake-up similarity is greater than the first similarity threshold, then the dynamic drift centroid is updated based on the vector to be identified;

[0010] Determine whether the current state triggers a drift constraint operation;

[0011] If the current state triggers a drift constraint operation, then calculate the drift distance between the updated dynamic drift centroid and the static anchor centroid.

[0012] If the drift distance is greater than the distance threshold, the updated dynamic drift centroid is called back.

[0013] Further, the process of encoding the received speech based on the encoder to obtain the vector to be recognized includes:

[0014] The received speech is globally attention encoded based on the global attention head in the encoder to obtain a global vector to be recognized;

[0015] The received speech is locally attention encoded based on the K local attention heads in the encoder to obtain K local vectors to be recognized, where K is an integer greater than 1;

[0016] The global vector to be identified and the K local vectors to be identified are combined to form the vector to be identified.

[0017] Further, the step of calculating the similarity between the vector to be identified and the dynamically drifting centroid to obtain the wake-up similarity includes:

[0018] The global wake-up similarity is obtained by calculating the similarity between the global vector to be identified and the global dynamic drift centroid in the dynamic drift centroid.

[0019] The similarity between the K local vectors to be identified and the K local dynamic drift centroids in the dynamic drift centroid is calculated to obtain the K local wake-up similarity.

[0020] The wake-up similarity is obtained by weighting the global wake-up similarity and the K local wake-up similarities.

[0021] Further, updating the dynamically drifting centroid based on the vector to be identified includes:

[0022] Update the dynamically drifting centroid using the following formula:

[0023]

[0024] in:

[0025] The updated dynamic drift centroid;

[0026] The dynamic drift centroid;

[0027] The vector to be identified;

[0028] This is the learning rate.

[0029] Furthermore, the encoder is a discriminator trained on an adversarial network, wherein the adversarial network includes a generator and a discriminator, the generator is used to generate obfuscated audio as difficult negative samples, and the discriminator is used to distinguish between real audio and obfuscated audio.

[0030] Furthermore, the encoder obtained based on adversarial network training includes:

[0031] Construct a discriminator consisting of a global attention head, K local attention heads, and an output layer; and construct a generator to generate confusing audio samples that are similar to real audio as difficult negative samples;

[0032] The generator is fixed and the initial encoder is trained, and the discriminator is fixed and the generator is trained; wherein the number of iterations and the termination condition are determined based on experimental data.

[0033] After training, the trained discriminator is retained as the encoder.

[0034] Furthermore, the loss function of the discriminator is the GE2E loss, which includes:

[0035] The weighted sum of the global GE2E loss and each of the aforementioned local GE2E losses.

[0036] A second aspect of this invention provides a custom wake-word recognition device based on a dual-mode centroid, comprising:

[0037] The encoding unit is used to encode the received speech based on the encoder to obtain the vector to be recognized;

[0038] The first calculation unit is used to calculate the similarity between the vector to be identified and the dynamic drift centroid to obtain the wake-up similarity, wherein the initial value of the dynamic drift centroid is a preset static anchor centroid;

[0039] The first judgment unit is used to determine whether the wake-up similarity is greater than a first similarity threshold;

[0040] An update unit is used to update the dynamic drift centroid based on the vector to be identified if the wake-up similarity is greater than a first similarity threshold.

[0041] The second judgment unit is used to determine whether the current state triggers the drift constraint operation;

[0042] The second calculation unit is used to calculate the drift distance between the updated dynamic drift centroid and the static anchor centroid if the current state triggers a drift constraint operation.

[0043] A callback unit is used to call back the updated dynamic drift centroid if the drift distance is greater than a distance threshold.

[0044] A third aspect of the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the custom wake word recognition method based on dual-mode centroid described above.

[0045] Beneficial effects:

[0046] This invention presents a custom wake-word recognition method based on a dual-mode centroid. By setting the initial value of the dynamically drifting centroid to a preset static anchor centroid, this method can dynamically adjust the centroid position when the voice input changes, while maintaining anchoring to the original wake-word features. This design effectively addresses changes in user pronunciation habits and the influence of environmental noise, thereby significantly improving the accuracy of wake-word recognition. When the wake-word similarity exceeds a first similarity threshold, the system updates the dynamically drifting centroid based on the vector to be recognized. This mechanism enables the system to continuously learn and adapt to changes in user voice features, enhancing its adaptability and robustness, and maintaining high recognition performance even in complex usage environments. By introducing the concepts of drift distance and distance threshold, the system can automatically revert to its centroid position when the dynamically drifting centroid deviates too far from the static anchor point. This design effectively prevents recognition errors caused by excessive drift of the dynamically drifting centroid, ensuring the long-term stability and reliability of the system. The system's built-in dynamic update and drift constraint mechanism can automatically adjust and correct the centroid position, reducing the need for manual intervention. This not only reduces the system's maintenance costs but also allows the system to maintain good recognition performance over a long period without frequent manual calibration and optimization. This invention achieves high-precision and robust wake word recognition, improving user experience and simplifying system maintenance. It has broad application prospects and significant technical value, such as being applicable to scenarios requiring voice triggering, like smart speakers and wearable devices. Attached Figure Description

[0047] Figure 1 A flowchart illustrating a custom wake word recognition method based on a dual-mode centroid according to an embodiment of the invention;

[0048] Figure 2 This is a schematic diagram of the structure of a custom wake word recognition device based on a dual-mode centroid according to an embodiment of the invention;

[0049] Figure 3 This is a schematic diagram of the structure of a computer device according to an embodiment of the invention.

[0050] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0052] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of features, integers, steps, operations, elements, modules, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, modules, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any modules and all combinations of one or more associated listed items.

[0053] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0054] Reference Figure 1 This invention provides a custom wake word recognition method based on a dual-mode centroid, comprising the following steps S1-S7:

[0055] S1: The received speech is encoded based on the encoder to obtain the vector to be recognized.

[0056] The encoder mentioned above refers to a deep neural network model (such as Conformer or Transformer) used to convert the input speech signal into a high-dimensional embedding vector, preserving the semantic and acoustic features of the speech. The vector to be recognized is the speech embedding, which is the numerical representation of speech in the feature space and is used for subsequent similarity comparison.

[0057] In this step, when the user utters a wake word (such as "Hey, Lingsheng"), the device's microphone captures the audio signal, which, after preprocessing (such as windowing and FFT), is input into the encoder, outputting a fixed-dimensional vector. This serves as the "digital fingerprint" of current voice. For example, in the "LingSheng" smart speaker, when a user says "Hi, Xiaoling," the system uses a Conformer encoder to convert it into a 256-dimensional vector. This step converts unstructured speech into computable vectors, facilitating subsequent mathematical comparisons and improving recognition efficiency and accuracy.

[0058] S2: Calculate the similarity between the vector to be identified and the dynamic drift centroid to obtain the wake-up similarity, wherein the initial value of the dynamic drift centroid is a preset static anchor centroid.

[0059] The aforementioned dynamic drift centroid refers to the continuously updated user wake-up word embedding center point during device use, reflecting the user's current pronunciation habits (such as speech rate and accent changes). The static anchor centroid is the original wake-up word embedding center generated during user registration, never updated, and serves as a "trust anchor" to constrain the dynamic drift centroid, preventing excessive shifts after updates. Wake-up similarity is the mapping value of the cosine similarity or Euclidean distance between the vector to be identified and the dynamic drift centroid, used to determine if a match exists. In this step, sim( , If the value is high, it indicates that the current voice matches the user's habits. For example, if a user frequently says "Hi, Xiaoling," the system records their pronunciation characteristics and dynamically shifts the centroid. Gradually converging towards this pronunciation. New pronunciation. The similarity is 0.92, etc.

[0060] S3: Determine whether the wake-up similarity is greater than the first similarity threshold.

[0061] The aforementioned first similarity threshold is a preset threshold for determining whether to update the dynamic drift centroid (e.g., 0.90), used to decide whether to trigger a wake-up action. When the wake-up similarity is greater than the first similarity threshold, it indicates that the received speech is high-quality speech with high confidence, and therefore it can be optimized based on the vector to be identified. Furthermore, this application also sets a second similarity threshold, which is less than the first similarity threshold (e.g., 0.85), and is the minimum threshold for waking up the device. When the wake-up similarity is greater than the second similarity threshold, it is determined to be a valid wake-up and enters the wake-up process; otherwise, it is ignored. When the wake-up similarity is greater than the second similarity threshold but less than the first similarity threshold, it indicates that the wake-up process can be entered, but the process of updating the dynamic drift centroid cannot be entered.

[0062] S4: If the wake-up similarity is greater than the first similarity threshold, then update the dynamic drift centroid based on the vector to be identified.

[0063] In this step, an exponential moving average (EMA) update strategy can be used to gradually bring the centroid closer to the new sample, avoiding sudden changes. This enables personalized user adaptation, allowing the system to "understand you better the more you use it," thus avoiding frequent re-registration.

[0064] S5: Determine whether the current state triggers the drift constraint operation.

[0065] In this step, the drift constraint operation is a protective mechanism to prevent the dynamic drift centroid from deviating too far from the original registered features (static anchor centroid), thus avoiding mislearning. It can be set to trigger after every 5 wake-ups or once per hour. For example, the system can be set to "check for drift after every 5 wake-ups," and constraint judgment will be initiated after the 5th wake-up. This prevents severe centroid shifts caused by accidental erroneous speech (such as imitation by others), ensuring system stability.

[0066] S6: If the current state triggers a drift constraint operation, calculate the drift distance between the updated dynamic drift centroid and the static anchor centroid.

[0067] The drift distance mentioned above is the Euclidean or cosine distance between the dynamic drift centroid and the static anchor centroid. This provides an objective indicator to measure the degree of drift and to inform corrective actions.

[0068] S7: If the drift distance is greater than the distance threshold, the updated dynamic drift centroid is called back.

[0069] The aforementioned callback refers to pulling the dynamically drifting centroid closer to the static anchor point to prevent excessive drift. The distance threshold is a preset maximum allowable drift distance (e.g., 0.5). If the drift distance exceeds 0.5, the callback operation is executed. This solves the industry problem of "uncontrolled centroid drift," balancing personalization and security.

[0070] The dual-centroid dynamic adaptive wake-up mechanism proposed in this embodiment is based on the following core idea: using a dynamically drifting centroid to adapt to changes in the user's pronunciation; introducing a static anchor centroid as a trust benchmark; and preventing false learning through periodic drift detection and callbacks. This solution resolves the contradiction between "user pronunciation changes leading to rejection" and "centroid drift leading to false wake-up" in existing technologies. It achieves a balance between personalized adaptation and stability in the wake-up system; significantly improves wake-up accuracy under long-term use; reduces the frequency of user re-registration; and enhances the user experience.

[0071] In one implementation, step S1, which encodes the received speech using an encoder to obtain a vector to be recognized, includes:

[0072] S11: Perform global attention encoding on the received speech based on the global attention heads in the encoder to obtain a globally待 - recognized vector.

[0073] The above - mentioned global attention heads refer to paying attention to the semantic integrity of the entire wake - up word sequence and capturing the complete semantic structure of the keyword. The global attention mechanism performs weighted aggregation on the entire speech frame sequence to generate a vector representing the overall semantics. . For example, when the user says "Hi, Xiaoling", the global attention head pays attention to the complete semantic flow from "Hi" to "Ling" and generates . Prevent misjudgment due to local pronunciation ambiguity and improve the semantic integrity recognition ability.

[0074] S12: Perform local attention encoding on the received speech based on K local attention heads in the encoder to obtain K locally待 - recognized vectors, where K is an integer greater than 1.

[0075] The above - mentioned local attention heads refer to each head paying attention to a specific sub - structure (such as syllables, phonemes) of the wake - up word to achieve structured parsing. For example, when the user says "Hi, Xiaoling" and K = 3, it pays attention to the three syllables "Hi", "Xiao", and "Ling" respectively. Each local attention head focuses on a local area of the speech sequence and extracts local features. . For example, head 1 pays attention to "Hi", head 2 pays attention to "Xiao", and head 3 pays attention to "Ling", and outputs locally待 - recognized vectors respectively , , . In this step, even if a certain part is not clearly pronounced (such as "Xiao" is pronounced softly), other parts can still assist in recognition, improving robustness.

[0076] S13: Combine the globally待 - recognized vector and the K locally待 - recognized vectors to form the待 - recognized vector.

[0077] The above vector combination can fuse multi - scale features by using concatenation or weighted summation. For example, by concatenating , , , into = , , , . The final vector contains the overall semantics + three syllable details, with richer information. It realizes "global - local" double verification and significantly reduces the false wake - up rate.

[0078] This embodiment refines the encoder structure design, proposing a hybrid attention architecture of "global + K local attention heads". This design mimics the "holistic perception + detail discrimination" mechanism of human hearing, enabling the system to grasp both the overall meaning of keywords and analyze local pronunciation variations. This enhances the structured understanding capability of wake-up word recognition; effectively addresses challenges such as unclear pronunciation and accent variations; and forms the core foundation for achieving high-precision wake-up.

[0079] In one embodiment, step S2, which calculates the similarity between the vector to be identified and the dynamically drifting centroid to obtain the wake-up similarity, includes:

[0080] S21: Calculate the similarity between the global vector to be identified and the global dynamic drift centroid in the dynamic drift centroid to obtain the global wake-up similarity.

[0081] This step calculates global wake-up similarity. ,Should Specifically refers to the global vector to be identified With global dynamic drift centroid Cosine similarity between them This is used to determine whether the current voice matches the user's current pronunciation habits. For example, when the user says "Hi, Xiaoling". =[0.85,-0.12,…], =[0.82,-0.10,…], calculated as follows =0.93, which reflects the overall pronunciation matching degree and is the core basis for awakening judgment.

[0082] S22: Calculate the similarity between the K local vectors to be identified and the K local dynamic drift centroids in the dynamic drift centroids to obtain the K local wake-up similarities.

[0083] This step calculates the similarity of K local wake-ups. , It refers to the k-th local vector to be identified. With the corresponding local dynamic drift centroid The similarity. This can be based on a formula. Calculation. For example, let K=3, corresponding to "Hi", "Small", and "Spirit": , , It captures local pronunciation changes, supporting refined recognition.

[0084] S23: The global wake-up similarity and the K local wake-up similarities are weighted and calculated to obtain the wake-up similarity.

[0085] This step involves weighted calculation of the total wake-up similarity. ,for example ,in =0.6, =0.4 / K. The above weighted fusion refers to combining global and local information to output the final wake-up confidence. For example... =0.6×0.93+0.4 / 3×(0.88+0.75+0.91)=0.8967. This step can improve the robustness of recognition and reduce the false wake-up rate.

[0086] This embodiment defines the complete calculation process for "wake-up similarity," where all similarities are based on dynamically drifting centroids for real-time wake-up judgment. It is a specific implementation of the aforementioned "determining whether the wake-up similarity is greater than the first similarity threshold." This achieves dual "global-local" verification; improves the system's adaptability to pronunciation changes; and provides reliable input for subsequent update mechanisms.

[0087] In one embodiment, updating the dynamically drifting centroid based on the vector to be identified includes:

[0088] Update the dynamically drifting centroid using the following formula:

[0089]

[0090] in:

[0091] The updated dynamic drift centroid;

[0092] The dynamic drift centroid;

[0093] The vector to be identified;

[0094] This is the learning rate.

[0095] In this embodiment, an exponential moving average (EMA) update strategy is used to allow the centroid to slowly converge towards the new sample. For example, =0.01, =[0.8,-0.1,…], =[0.85,-0.12,…], then: =0.99×[0.8,-0.1,…]+0.01×[0.85,-0.12,…]. This embodiment defines the basic update mechanism for the dynamically drifting centroid, which is the core of realizing personalized learning for users. It supports online learning; the update process is smooth.

[0096] In one embodiment, the encoder described above is a discriminator trained on an adversarial network, wherein the adversarial network includes a generator and a discriminator, the generator being used to generate obfuscated audio as hard negative samples, and the discriminator being used to distinguish between real audio and obfuscated audio.

[0097] The generator G described above is used to generate highly realistic interference speech (such as speed variation, noise addition, and accent simulation). The discriminator D is used to distinguish between real and generated speech; the encoder used in this invention is the trained discriminator. Through adversarial training, the robustness of the encoder to similar words is improved. For example, generator G generates "Hey, Xiaoling" and "Hi, Xiaoling," and discriminator D learns to distinguish between them. Adversarial training improves the encoder's discriminative ability and robustness to difficult samples. This reduces false wake-ups in the voice wake-up system and supports high-quality feature extraction.

[0098] In one embodiment, the encoder obtained by training an adversarial network includes:

[0099] A discriminator is constructed, consisting of a global attention head, K local attention heads, and an output layer; and a generator is constructed to generate obfuscated audio similar to real audio as difficult negative samples.

[0100] The discriminator D mentioned above is the encoder structure used in this invention, which maps the input audio into a high-dimensional embedding vector and determines whether the vector comes from real user speech. The generator G is a generative model (such as one based on the WGAN-GP architecture) used to generate highly realistic obfuscated audio from noise or real speech as hard negative samples.

[0101] The discriminator D employs a dual-branch attention encoder structure (global attention + local attention), consistent with the encoder described above. The generator G receives random noise z and real speech segments x. real As input, the output is obfuscated speech x fake =G(z,x real The generator simulates accent variations, speech rate stretching, and background noise superposition. For example, given the real speech "Hi, Xiaoling," the generator outputs confused speech such as "Hey, Xiaoling," "Hi, Xiaoling," "Hi...Xiaoling..." (with a pause), and "Hi—Xiaoling" (with a drawn-out sound). The generator actively creates boundary samples to improve the robustness of the discriminator to similar words and variant pronunciations.

[0102] The generator G is fixed and the discriminator D is trained, and the discriminator D and the generator G are trained in a fixed loop; wherein the number of loops and the termination condition are determined based on experimental data.

[0103] In adversarial training, the discriminator D attempts to correctly distinguish between real and generated speech, while the generator G attempts to "deceive" the discriminator D; the two co-evolve in the game. The discriminator D uses GE2E loss (see the example below), and the generator G uses adversarial loss plus multi-scale reconstruction loss. Specifically:

[0104] Fix generator G, update discriminator D:

[0105] Fixed discriminator D, updated generator G:

[0106] in, λ represents the spectrum reconstruction loss, and λ is the weighting coefficient.

[0107] The training convergence condition can be that the GE2E loss of the discriminator D on the validation set decreases by less than 0.001 for 5 consecutive epochs; or the number of training epochs reaches 200 epochs, etc.

[0108] After training, the trained discriminator D is retained as the encoder.

[0109] Retaining the discriminator D means that after adversarial training, the generator G is discarded, and the trained discriminator D is used as the final encoder for subsequent wake-up recognition tasks. The discriminator D has learned a high-quality, robust embedding space, effectively distinguishing user speech from interference speech. This encoder is directly used to extract... , , Equivalent vectors. For example, during user registration, this encoder can be used to extract the static anchor centroid of "Hi, Xiaoling". Extraction upon wake-up A comparison was performed. The encoder obtained in this embodiment has strong anti-interference capabilities, providing a reliable feature basis for subsequent dynamic drift centroid updates and wake-up judgments.

[0110] This embodiment introduces the complete adversarial training process of the encoder, which is a prerequisite for achieving highly robust wake-up recognition. This process generates difficult negative samples through a generator, forcing the discriminator to learn more refined speech representation boundaries. The final discriminator D is the encoder used in this invention. This significantly reduces the false wake-up rate (especially in scenarios with similar words such as "hey / hi" and "xiaoling / xiaoling"); improves the system's stability under noise, accent, and speech rate variations; and allows for lightweight deployment by retaining only the discriminator after training.

[0111] Furthermore, the loss function of the discriminator described above is the GE2E loss, which includes:

[0112] The weighted sum of the global GE2E loss and each of the aforementioned local GE2E losses.

[0113] The above GE2E (Generalized End-to-End) loss is a metric learning loss function, aiming to make the embedding vectors of the same speaker as close as possible and those of different speakers as far apart as possible. The formula is , where e is the query embedding, is the reference centroid of the same class, that is, the central point of the reference features that belong to the "same speaker" as the query embedding e, is all reference centroids, enhancing the intra-class compactness and inter-class separability of the embedding space.

[0114] The global GE2E loss refers to the GE2E loss calculated based on the global vector to be recognized and the global centroid c g , capturing the overall speech features , where is the global centroid of all samples in the batch. There are 10 users in a batch, and each user has 5 speech segments. Calculate the similarity with 10 global centroids to optimize the correct user matching. Learn the overall pronunciation style features.

[0115] The local GE2E loss calculates the GE2E loss for each local segment k to capture local pronunciation details. The formula is: where is the local centroid of the sample. For example, calculate the loss for the three segments of "hi", "xiao", and "ling" respectively to ensure that each syllable can be accurately recognized. Enhance the robustness to local pronunciation variations.

[0116] The weighted sum loss function is to fuse the global and local losses according to the weights to form the final discriminator loss, , where λ g = 0.6, = 0.4 / K. This step can balance the global and local information and improve the embedding quality.

[0117] This embodiment defines the core loss function of the discriminator (i.e., the encoder), adopts the weighted GE2E loss, combines the global and local information, ensures that the encoder learns multi-level and highly discriminative speech embeddings, can improve the discriminative ability of feature embeddings; supports subsequent dynamic drift centroid update and wake-up judgment; forms a closed-loop optimization with the above adversarial training.

[0118] In one embodiment, the updated dynamic drift centroid described in the above callback includes:

[0119] Use the following callback formula to constrain the updated dynamic drift centroid:

[0120]

[0121] in:

[0122] The dynamic drift centroid after the callback;

[0123] The updated dynamic drift centroid;

[0124] The centroid of the static anchor point;

[0125] To measure the degree of drift, such as drift distance. =[0.81,-0.11,…], If the range is [0.80, -0.10, ...], then d = 0.014;

[0126] The non-linear callback strength function controls the pullback strength, and is defined as follows: ,in: The distance threshold is γ; γ>0 is the growth rate parameter, such as γ=10, etc.; when Less than hour, ≈0, almost no callback; when → hour, →0.5; when Much larger hour, →1, strong callback; smooth transition, avoid abrupt changes. For example, =0.06, greater than =0.05, ≈0.73, the centroid is pulled back by 73% of the distance, achieving a gradual constraint.

[0127] This embodiment defines a callback mechanism for dynamically drifting centroids. After each update, the drift distance is checked. If it exceeds a threshold, the centroid is pulled back to the original static anchor point using a nonlinear function, achieving a balance between personalization and security. This prevents the centroid from deviating from its original characteristics due to long-term use, resists continuous imitation attacks, and improves the long-term stability of the system.

[0128] Reference Figure 2 This application also provides a custom wake word recognition device based on a dual-mode centroid, used to implement the custom wake word recognition method based on a dual-mode centroid in any of the above embodiments, including:

[0129] The encoding unit 10 is used to encode the received speech based on the encoder to obtain the vector to be recognized;

[0130] The first calculation unit 20 is used to calculate the similarity between the vector to be identified and the dynamic drift centroid to obtain the wake-up similarity, wherein the initial value of the dynamic drift centroid is a preset static anchor centroid.

[0131] The first judgment unit 30 is used to determine whether the wake-up similarity is greater than a first similarity threshold;

[0132] The update unit 40 is used to update the dynamic drift centroid based on the vector to be identified if the wake-up similarity is greater than the first similarity threshold.

[0133] The second judgment unit 50 is used to determine whether the current state triggers the drift constraint operation;

[0134] The second calculation unit 60 is used to calculate the drift distance between the updated dynamic drift centroid and the static anchor centroid if the current state triggers a drift constraint operation.

[0135] Callback unit 70 is used to call back the updated dynamic drift centroid if the drift distance is greater than a distance threshold.

[0136] Reference Figure 3 The present invention also provides a computer device, the internal structure of which can be as follows: Figure 3As shown, this computer device includes a processor, memory, network interface, and database connected via a system bus. The processor is designed to provide computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores operating devices, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores encoders, etc. The network interface is used for communication with external terminals via a network connection. Furthermore, the computer device may also include input devices and a display screen, etc. When the aforementioned computer program is executed by a processor, it implements a custom wake-word recognition method based on a dual-mode centroid. The method includes: encoding received speech using an encoder to obtain a vector to be recognized; calculating the similarity between the vector to be recognized and a dynamically drifting centroid to obtain a wake-word similarity, wherein the initial value of the dynamically drifting centroid is a preset static anchor centroid; determining whether the wake-word similarity is greater than a first similarity threshold; if the wake-word similarity is greater than the first similarity threshold, updating the dynamically drifting centroid based on the vector to be recognized; determining whether the current state triggers a drift constraint operation; if the current state triggers a drift constraint operation, calculating the drift distance between the updated dynamically drifting centroid and the static anchor centroid; if the drift distance is greater than a distance threshold, calling back the updated dynamically drifting centroid. Those skilled in the art will understand that... Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer equipment on which the present application is applied.

[0137] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A custom wake word recognition method based on dual-mode centroid, characterized in that, The method includes: The received speech is encoded using an encoder to obtain the vector to be recognized; The similarity between the vector to be identified and the dynamic drift centroid is calculated to obtain the wake-up similarity. The initial value of the dynamic drift centroid is a preset static anchor centroid. The dynamic drift centroid refers to the user wake-up word embedding center point that is continuously updated during device use. The static anchor centroid is the original wake-up word embedding center generated when the user registers. Determine whether the wake-up similarity is greater than a first similarity threshold; If the wake-up similarity is greater than the first similarity threshold, then the dynamic drift centroid is updated based on the vector to be identified; Determine whether the current state triggers a drift constraint operation; If the current state triggers a drift constraint operation, then calculate the drift distance between the updated dynamic drift centroid and the static anchor centroid. If the drift distance is greater than the distance threshold, the updated dynamic drift centroid is called back; The process of encoding the received speech using an encoder to obtain a vector to be recognized includes: The received speech is globally attention encoded based on the global attention head in the encoder to obtain a global vector to be recognized; The received speech is locally attention encoded based on the K local attention heads in the encoder to obtain K local vectors to be recognized, where K is an integer greater than 1; The global vector to be identified and the K local vectors to be identified are combined to form the vector to be identified; The step of calculating the similarity between the vector to be identified and the dynamically drifting centroid to obtain the wake-up similarity includes: The global wake-up similarity is obtained by calculating the similarity between the global vector to be identified and the global dynamic drift centroid in the dynamic drift centroid. The similarity between the K local vectors to be identified and the K local dynamic drift centroids in the dynamic drift centroid is calculated to obtain the K local wake-up similarity. The wake-up similarity is obtained by weighting the global wake-up similarity and the K local wake-up similarities. The step of updating the dynamically drifting centroid based on the vector to be identified includes: Update the dynamically drifting centroid using the following formula: in: The updated dynamic drift centroid; The dynamic drift centroid; The vector to be identified; This is the learning rate.

2. The custom wake word recognition method based on dual-mode centroid according to claim 1, characterized in that, The encoder is a discriminator trained on an adversarial network, wherein the adversarial network includes a generator and a discriminator. The generator is used to generate obfuscated audio as difficult negative samples, and the discriminator is used to distinguish between real audio and obfuscated audio.

3. The custom wake-word recognition method based on dual-mode centroid according to claim 2, characterized in that, The encoder obtained by training based on adversarial networks includes: Construct a discriminator consisting of a global attention head, K local attention heads, and an output layer; and construct a generator to generate confusing audio samples that are similar to real audio as difficult negative samples; The generator is fixed and the initial encoder is trained, and the discriminator is fixed and the generator is trained; wherein the number of iterations and the termination condition are determined based on experimental data. After training, the trained discriminator is retained as the encoder.

4. The custom wake-up word recognition method based on dual-mode centroid according to claim 3, characterized in that, The loss function of the discriminator is the GE2E loss, which includes: The weighted sum of the global GE2E loss and the local GE2E losses.

5. A custom wake word recognition device based on bimodal centroid, used to implement the custom wake word recognition method based on bimodal centroid as described in any one of claims 1-4, characterized in that, include: The encoding unit is used to encode the received speech based on the encoder to obtain the vector to be recognized; The first calculation unit is used to calculate the similarity between the vector to be identified and the dynamic drift centroid to obtain the wake-up similarity, wherein the initial value of the dynamic drift centroid is a preset static anchor centroid; The first judgment unit is used to determine whether the wake-up similarity is greater than a first similarity threshold; An update unit is used to update the dynamic drift centroid based on the vector to be identified if the wake-up similarity is greater than a first similarity threshold. The second judgment unit is used to determine whether the current state triggers the drift constraint operation; The second calculation unit is used to calculate the drift distance between the updated dynamic drift centroid and the static anchor centroid if the current state triggers a drift constraint operation. A callback unit is used to call back the updated dynamic drift centroid if the drift distance is greater than a distance threshold.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the custom wake word recognition method based on dual-mode centroid as described in any one of claims 1 to 4.

Images