Custom wake-up word recognition method, device and equipment based on dual-mode centroid

Abstract

The invention relates to the technical field of voice recognition, in particular to a self-defined wake-up word recognition method, device and equipment based on a dual-mode mass center. The method is based on a dual-mode centroid mechanism and comprises the steps of obtaining a to-be-recognized vector, performing similarity calculation on the to-be-recognized vector and a dynamic drift centroid to obtain wake-up similarity, judging whether the wake-up similarity is greater than a preset threshold value or not, and determining whether the to-be-recognized vector can be used for updating the dynamic drift centroid or not. The dynamic drift centroid is initially a static anchor centroid and supports self-adaptive updating based on effective wake-up voice, and a callback mechanism is introduced to prevent excessive deviation of the dynamic drift centroid. According to the technical scheme, the safety of the static anchor point and the adaptability of the dynamic drift mass center are fused, and personalized identification of identity perception is achieved. The method effectively improves the accuracy and robustness of wake-up recognition, reduces the false wake-up rate, enhances the adaptability of the system to the pronunciation change of the user and the environmental noise, improves the user experience, and is suitable for scenes needing voice triggering, such as an intelligent loudspeaker box and wearable equipment.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of speech recognition, in particular to a self-defined wake-up word recognition method, device and equipment based on a dual-mode centroid. BACKGROUND

[0002] With the development of artificial intelligence technology, the demand for custom keyword spotting (KWS) technology is growing. Users expect to be able to activate the device with their own set of arbitrary phrases, rather than using the manufacturer's pre-set fixed wake-up word. The key to achieving this goal is the zero-shot or few-shot learning ability, that is, the system does not need to retrain the model for each new keyword. Currently, the mainstream custom KWS technology route is to convert the audio segment into a fixed-dimensional embedding vector through a deep learning model, and to determine whether to wake up by comparing the embedding vector of the current input audio with the embedding vector of the user-registered keyword (i.e. "registered centroid").

[0003] The user's registration process is one-time, however, the user's pronunciation habits, tone, speed, and even the environment they are in will change dynamically over time. A static, registered centroid generated in an ideal environment will lose its representativeness over time, resulting in a decrease in wake-up rate or an increase in false wake-up rate. The existing solution lacks a mechanism that allows the model to evolve and dynamically adapt to user and environmental changes at the device end. SUMMARY

[0004] The main purpose of the present application is to provide a self-defined wake-up word recognition method based on a dual-mode centroid, which aims to solve the technical problem that the representativeness of a registered centroid generated in a static environment will decrease over time, resulting in a decrease in wake-up rate or an increase in false wake-up rate.

[0005] In order to achieve the above-mentioned application purpose, the first aspect of the present application provides a self-defined wake-up word recognition method based on a dual-mode centroid, the method comprising: encoding the received voice based on an encoder to obtain a to-be-identified vector; calculating the similarity between the to-be-identified vector and a dynamic drift centroid to obtain a wake-up similarity, wherein the initial value of the dynamic drift centroid is a pre-set static anchor centroid; determining whether the wake-up similarity is greater than a first similarity threshold; if the wake-up similarity is greater than the first similarity threshold, updating the dynamic drift centroid based on the to-be-identified vector; determining whether the current state triggers a drift constraint operation; If the current state triggers a drift constraint operation, a distance of drift between an updated dynamic drift centroid and the static anchor centroid is calculated; If the distance of drift is greater than a distance threshold, the updated dynamic drift centroid is recalled.

[0006] Further, the received voice is encoded based on the encoder to obtain a to-be-recognized vector, comprising: The received voice is globally attention encoded based on a global attention head in the encoder to obtain a global to-be-recognized vector; The received voice is locally attention encoded based on K local attention heads in the encoder to obtain K local to-be-recognized vectors, wherein K is an integer greater than 1; The global to-be-recognized vector and the K local to-be-recognized vectors are combined to form the to-be-recognized vector.

[0007] Further, the similarity between the to-be-recognized vector and the dynamic drift centroid is calculated to obtain a wake-up similarity, comprising: The global to-be-recognized vector and a global dynamic drift centroid in the dynamic drift centroid are similarity calculated to obtain a global wake-up similarity; The K local to-be-recognized vectors are respectively similarity calculated with K local dynamic drift centroids in the dynamic drift centroid to obtain K local wake-up similarities; The global wake-up similarity and the K local wake-up similarities are weighted calculated to obtain the wake-up similarity.

[0008] Further, the dynamic drift centroid is updated based on the to-be-recognized vector, comprising: The dynamic drift centroid is updated by using the following formula:

[0009] Wherein: is the updated dynamic drift centroid; is the dynamic drift centroid; is the to-be-recognized vector; is a learning rate.

[0010] Further, the encoder is a discriminator trained based on a generative adversarial network, wherein the generative adversarial network comprises a generator and a discriminator, the generator is used to generate an ambiguous audio as a difficult negative sample, and the discriminator is used to distinguish between real audio and ambiguous audio.

[0011] Further, the encoder is trained based on the adversarial network, and the encoder comprises: constructing a discriminator comprising a global attention head, K local attention heads and an output layer, and constructing a generator for generating confusing audio similar to the real audio as a difficult negative sample; cyclically executing the fixed generator, training the initial encoder, and fixing the discriminator, and training the generator, wherein the number of times of cyclic execution and the termination condition are determined according to experimental data; After the training is completed, the trained discriminator is retained as the encoder.

[0012] Further, the loss function of the discriminator is GE2E loss, and the GE2E loss comprises: a weighted sum of the global GE2E loss and each local GE2E loss.

[0013] The second aspect of the present application provides a self-defined wake-up word recognition device based on a dual-mode centroid, comprising: An encoding unit is configured to encode the received voice based on an encoder to obtain a to-be-identified vector; A first calculation unit is configured to calculate the similarity between the to-be-identified vector and a dynamic drift centroid to obtain a wake-up similarity, wherein the initial value of the dynamic drift centroid is a preset static anchor centroid. A first judgment unit is configured to judge whether the wake-up similarity is greater than a first similarity threshold. An updating unit is configured to update the dynamic drift centroid based on the to-be-identified vector if the wake-up similarity is greater than the first similarity threshold. A second judgment unit is configured to judge whether the current state triggers a drift constraint operation. A second calculation unit is configured to calculate the drift distance between the updated dynamic drift centroid and the static anchor centroid if the current state triggers the drift constraint operation. A callback unit is configured to callback the updated dynamic drift centroid if the drift distance is greater than a distance threshold.

[0014] The third aspect of the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to realize the steps of the self-defined wake-up word recognition method based on the dual-mode centroid.

[0015] Advantages: The self-defined wake-up word recognition method based on the dual-mode centroid of the application can dynamically adjust the centroid position when the voice input changes while maintaining the anchoring of the original wake-up word features by setting the initial value of the dynamic drifting centroid as the preset static anchor centroid. This design can effectively cope with the changes in user pronunciation habits and the influence of environmental noise, thereby significantly improving the accuracy of wake-up word recognition. When the wake-up similarity is greater than the first similarity threshold, the system will update the dynamic drifting centroid based on the to-be-recognized vector. This mechanism enables the system to continuously learn and adapt to changes in user voice features, enhancing the system's adaptive ability and robustness, and maintaining high recognition performance even in complex use environments. By introducing the concepts of drift distance and distance threshold, the system can automatically adjust the centroid position when the dynamic drifting centroid deviates too far from the static anchor. This design effectively prevents recognition errors caused by excessive drifting of the dynamic drifting centroid, ensuring the stability and reliability of the system during long-term operation. The system's built-in dynamic updating and drift constraint mechanism can automatically adjust and correct the centroid position, reducing the need for manual intervention. This not only reduces the maintenance cost of the system, but also enables the system to maintain good recognition performance for a long time without the need for frequent manual calibration and optimization. The application realizes high-precision and high-robustness wake-up word recognition, improves user experience and simplifies system maintenance, and has wide application prospects and important technical value. It is suitable for scenarios such as smart speakers, wearable devices, etc. that require voice triggering. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 Flowchart of the self-defined wake-up word recognition method based on the dual-mode centroid of an embodiment of the application; Figure 2 Structure diagram of the self-defined wake-up word recognition device based on the dual-mode centroid of an embodiment of the application; Figure 3 Structure diagram of the computer device of an embodiment of the application.

[0017] The implementation, functional features and advantages of the application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0019] Those skilled in the art can understand that the singular forms "a," "an," and "the" as used herein, unless specifically stated otherwise, also include plural forms. It should be further understood that the use of the term "includes" in the specification of the present application means that a feature, integer, step, operation, element, module and / or assembly is present, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, modules, assemblies and / or groups thereof. It should be understood that when we say that an element is "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or there can be intermediate elements. In addition, "connected" or "coupled" as used herein can include wireless connection or wireless coupling. The phrase "and / or" as used herein includes all or any combination of the associated listed items.

[0020] Those skilled in the art can understand that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as that generally understood by those skilled in the art to which the present application belongs. It should also be understood that terms such as those defined in a general dictionary should be understood to have meanings consistent with those in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as such.

[0021] Reference Figure 1 The embodiment of the present application provides a self-defined wake-up word recognition method based on a dual-mode centroid, comprising the following steps S1-S7: S1: encoding the received voice based on an encoder to obtain a to-be-recognized vector.

[0022] The above-mentioned encoder refers to a deep neural network model (such as Conformer, Transformer), which is used to convert the input voice signal into a high-dimensional embedding vector (embedding) to preserve the semantic and acoustic features of the voice. The above-mentioned to-be-recognized vector, i.e. the voice embedding (utterance embedding), is a numerical representation of the voice in the feature space, which is used for subsequent similarity comparison.

[0023] In this step, when the user says the wake-up word (such as "Hey, Ling Voice"), the device microphone collects the audio signal, which is input into the encoder after pre-processing (such as windowing, FFT), and outputs a fixed-dimensional vector as the "digital fingerprint" of the current voice. For example, in the "Ling Voice" smart speaker, the user says "Hi, Little Ling", and the system uses the Conformer encoder to convert it into a 256-dimensional vector This step converts unstructured voice into a computable vector, which facilitates subsequent mathematical comparison and improves recognition efficiency and accuracy.

[0024] S2: similarity calculation is performed between the to-be-identified vector and a dynamic drift centroid to obtain a wake-up similarity, wherein an initial value of the dynamic drift centroid is a preset static anchor centroid.

[0025] The dynamic drift centroid refers to a user wake-up word embedding center point that is continuously updated during use of the device, reflecting the current pronunciation habit (such as speech speed, accent change) of the user. The static anchor centroid is an original wake-up word embedding center generated when the user registers, and is never updated. It is used as a "trust anchor" to constrain the dynamic drift centroid and prevent the dynamic drift centroid from deviating too much after being updated. The wake-up similarity is a mapping value of the cosine similarity or Euclidean distance between the to-be-identified vector and the dynamic drift centroid, and is used to determine whether there is a match. In this step, sim( , ) is calculated. If the value is high, it means that the current speech matches the user's habit. For example, if a user has been saying "Hi, little spirit" for a long time, the system records the pronunciation features, and the dynamic drift centroid gradually converges to the pronunciation. The similarity between the new speech and it is 0.92 or the like.

[0026] S3: Determine whether the wake-up similarity is greater than a first similarity threshold.

[0027] The first similarity threshold is a preset determination threshold for updating the dynamic drift centroid (such as 0.90), and is used to determine whether to trigger a wake-up action. When the wake-up similarity is greater than the first similarity threshold, it means that the received speech is a high-quality speech with high confidence, and therefore the to-be-identified vector can be optimized based on it. Further, a second similarity threshold is also provided, which is less than the first similarity threshold (such as 0.85), and is the minimum threshold for waking up the device. When the wake-up similarity is greater than the second similarity threshold, it is determined to be an effective wake-up, and the wake-up process is entered. Otherwise, it is ignored. When the wake-up similarity is greater than the second similarity threshold and less than the first similarity threshold, it means that the wake-up process can be entered, but the process of updating the dynamic drift centroid cannot be entered.

[0028] S4: If the wake-up similarity is greater than the first similarity threshold, update the dynamic drift centroid based on the to-be-identified vector.

[0029] In this step, an exponential moving average (EMA) update strategy can be used to gradually converge the centroid to the new sample, avoiding sudden changes. This achieves user personalization and adaptation, and the system "understands you better the more you use it", avoiding frequent re-registration.

[0030] S5: Determine whether the current state triggers a drift constraint operation.

[0031] In this step, the drift constraint operation is a protection mechanism to prevent the dynamic drift centroid from deviating too far from the original registration feature (static anchor centroid), avoiding mislearning. It can be set to trigger after completing 5 wake-ups or once an hour. For example, the system is set to "check the drift once every 5 wake-ups", and the constraint judgment is started after the 5th wake-up. It prevents the centroid from being severely shifted due to accidental error speech (such as imitations by others), ensuring system stability.

[0032] S6: If the current state triggers the drift constraint operation, calculate the drift distance between the updated dynamic drift centroid and the static anchor centroid.

[0033] The above drift distance is the Euclidean distance or cosine distance between the dynamic drift centroid and the static anchor centroid. An objective indicator is provided to measure the degree of drift, providing a basis for recall.

[0034] S7: If the drift distance is greater than the distance threshold, recall the updated dynamic drift centroid.

[0035] The above recall refers to pulling the dynamic drift centroid closer to the static anchor point to prevent excessive drift. The distance threshold is the maximum allowed drift distance (e.g. 0.5). If the drift distance is greater than 0.5, the recall operation is performed. This solves the industry problem of "centroid drift out of control", balancing personalization and security.

[0036] The dual centroid dynamic adaptive wake-up mechanism proposed in this embodiment has the core idea of using a dynamic drift centroid to adapt to user pronunciation changes, introducing a static anchor centroid as a trust reference, and preventing mislearning through regular drift detection and recall. This solution solves the contradiction between "user pronunciation changes leading to rejection" and "centroid drift leading to false wake-up" in the prior art. It achieves the unity of wake-up system personalization and stability, significantly improves the wake-up accuracy rate over a long period of use, reduces the frequency of user re-registration, and improves user experience.

[0037] In one implementation, the above step S1 of encoding the received speech based on the encoder to obtain the to-be-identified vector includes: S11: Perform global attention encoding on the received speech based on the global attention head in the encoder to obtain a global to-be-identified vector.

[0038] The above global attention head refers to focusing on the semantic integrity of the entire wake-up word sequence to capture the complete semantic structure of the keyword. The global attention mechanism weights and aggregates the entire speech frame sequence to generate a vector representing the overall semantics . For example, if the user says "Hi, little spirit", the global attention head focuses on the complete semantic flow from "Hi" to "spirit" to generate . This prevents overall misjudgment due to local pronunciation ambiguity and improves semantic integrity recognition ability.

[0039] S12: encode the received voice based on K local attention heads in the encoder to obtain K local to-be-recognized vectors, wherein K is an integer greater than 1.

[0040] The above local attention head refers to each head paying attention to a specific substructure (such as a syllable, a phoneme) of the wake-up word, realizing structured analysis. For example, the user says "Hey, little spirit", K = 3, and pays attention to "Hey", "little", and "spirit" respectively. Each local attention head focuses on a local area of the voice sequence and extracts local features . For example, head 1 pays attention to "Hey", head 2 pays attention to "little", and head 3 pays attention to "spirit", and outputs local to-be-recognized vectors , , In this step, even if part of the pronunciation is unclear (such as "little" pronunciation light), other parts can still assist in recognition, improving robustness.

[0041] S13: combining the global to-be-recognized vector and the K local to-be-recognized vectors to form the to-be-recognized vector.

[0042] The above vector combination can fuse multi-scale features in a concatenation or weighted summation manner. For example, the global to-be-recognized vector , , , is concatenated as [ , , , ]. The final vector contains the overall semantics + three syllable details, and the information is more abundant. Realize "overall-local" double verification, significantly reduce the false wake-up rate.

[0043] This embodiment deepens the encoder structure design and proposes a hybrid attention architecture of "global + K local attention heads". This design simulates the "overall perception + detail resolution" mechanism of human hearing, enabling the system to grasp the overall keyword and analyze local pronunciation changes. Improve the structured understanding ability of wake-up word recognition; effectively deal with challenges such as user pronunciation ambiguity and accent variation; is the core foundation to realize high-precision wake-up.

[0044] In one embodiment, the step S2 of calculating the similarity between the to-be-recognized vector and the dynamic drift centroid to obtain the wake-up similarity includes: S21: calculating the similarity between the global to-be-recognized vector and the global dynamic drift centroid in the dynamic drift centroid to obtain the global wake-up similarity.

[0045] This step calculates the global wake-up similarity , the global to-be-recognized vector and the global dynamic shift centroid , the cosine similarity between them, is used to determine whether the current voice matches the user's current pronunciation habit. For example, the user says "Hi, little spirit", = [0.85, -0.12,...], = [0.82, -0.10,...], and the calculation result is = 0.93, which reflects the overall pronunciation matching degree and is the core basis for the wake-up judgment.

[0046] S22: Calculate the similarity between the K local to-be-recognized vectors and the K local dynamic shift centroids in the dynamic shift centroid, to obtain K local wake-up similarities.

[0047] This step calculates K local wake-up similarities , , which refers to the similarity between the k-th local to-be-recognized vector and the corresponding local dynamic shift centroid . It can be calculated based on the formula . For example, set K = 3, corresponding to "Hi", "little", and "spirit" respectively: , , . Capture local pronunciation changes and support fine-grained recognition.

[0048] S23: Perform weighted calculation on the global wake-up similarity and the K local wake-up similarities to obtain the wake-up similarity.

[0049] This step calculates the total wake-up similarity , for example , where = 0.6, = 0.4 / K. The above weighted fusion refers to combining global and local information to output the final wake-up confidence. For example, = 0.6 x 0.93 + 0.4 / 3 x (0.88 + 0.75 + 0.91) = 0.8967. This step can improve the recognition robustness and reduce the false wake-up rate.

[0050] This embodiment defines the complete calculation process of "wake-up similarity", all similarities are based on dynamic shift centroid, which is used for real-time wake-up judgment. It is the specific implementation of "determining whether the wake-up similarity is greater than the first similarity threshold". It realizes "global-local" double verification; improves the adaptability of the system to pronunciation changes; and provides reliable input for the subsequent update mechanism.

[0051] In one embodiment, the updating the dynamic shift centroid based on the to-be-identified vector comprises: The dynamic shift centroid is updated by using the following formula:

[0052] Wherein: is the updated dynamic shift centroid; is the dynamic shift centroid; is the to-be-identified vector; is the learning rate.

[0053] In this embodiment, the exponential moving average (EMA) update strategy is used to make the centroid slowly approach the new sample. For example, = 0.01, = [0.8, -0.1,...], = [0.85, -0.12,...], then: = 0.99 * [0.8, -0.1,...] + 0.01 * [0.85, -0.12,...]. This embodiment defines the basic update mechanism of the dynamic shift centroid, which is the core of realizing user personalized learning. It supports online learning; the update process is smooth.

[0054] In one embodiment, the encoder is a discriminator trained based on an adversarial network, wherein the adversarial network includes a generator and a discriminator, the generator is used to generate confused audio as a difficult negative sample, and the discriminator is used to distinguish between real audio and confused audio.

[0055] The generator G is used to generate "highly simulated" interference speech (such as variable speed, noise addition, and accent simulation). The discriminator D is used to distinguish between real and generated speech, and the encoder used in the present application is the trained discriminator. Through adversarial training, the robustness of the encoder to approximate words is improved. For example, the generator G generates "Hey, little spirit" and "Hi, little Ling", and the discriminator D learns to distinguish. Through adversarial training, the discrimination ability of the encoder is improved, and the robustness of the encoder to difficult samples is improved. The voice wake-up system reduces false wake-up and supports high-quality feature extraction.

[0056] In one embodiment, the encoder is trained based on an adversarial network, comprising: constructing a discriminator including a global attention head, K local attention heads, and an output layer; and constructing a generator for generating confused audio similar to real audio as a difficult negative sample.

[0057] The discriminator D described above is an encoder structure adopted by the present application, which is used to map the input audio to a high-dimensional embedding vector and determine whether the vector is from real user speech. The generator G is a generation model (such as a WGAN-GP architecture-based model), which is used to generate high-simulation confusion audio from noise or real speech as difficult negative samples.

[0058] The discriminator D adopts a double-branch attention encoder structure (global attention + local attention), which is consistent with the above-described encoder. The generator G receives random noise z and real speech segment x real As input, output confusion speech x fake =G(z,x real ), simulating changes in accent, speech speed stretching, background noise superposition, etc. For example, real speech: "Hey, little spirit", generator output confusion speech: "Hey, little spirit", "Hey, little spirit", "Hey… little spirit…" (with pause), "Hey—little spirit" (lengthened sound). The generator actively creates boundary samples to improve the robustness of the discriminator to approximate words and variant pronunciations.

[0059] The fixed generator G is cyclically executed, the discriminator D is trained, and the fixed discriminator D is trained, and the fixed discriminator D is trained; wherein the number of times of cyclic execution and the termination condition are determined according to experimental data.

[0060] In the adversarial training, the discriminator D tries to correctly distinguish between real and generated speech, and the generator G tries to "deceive" the discriminator D, and both evolve together in the game. The discriminator D uses the GE2E loss (see the following embodiment), and the generator G uses the adversarial loss + multi-scale reconstruction loss. Specifically: The generator G is fixed, and the discriminator D is updated:

[0061] The discriminator D is fixed, and the generator G is updated:

[0062] wherein, is the spectrum reconstruction loss, and λ is the weight coefficient.

[0063] The training convergence condition can be that the GE2E loss of the discriminator D on the verification set decreases by less than 0.001 for 5 consecutive epochs, or the training number reaches 200 rounds, etc.

[0064] After the training is completed, the trained discriminator D is retained as the encoder.

[0065] The discriminator D is retained, that is, after the adversarial training is completed, the generator G is discarded, and the trained discriminator D is used as the final encoder for subsequent wake-up recognition tasks. The discriminator D has learned a high-quality and robust embedding space, which can effectively distinguish between user speech and interference speech. The encoder is directly used to extract , , The vector. For example, when the user registers, the static anchor point centroid of "Hey, Xiao Ling" is extracted using the encoder ; the vector is extracted when waking up for comparison. The encoder obtained in this embodiment has strong anti-interference capability; and provides a reliable feature basis for subsequent dynamic drift centroid updating and wake-up judgment.

[0066] The complete adversarial training process of the encoder is introduced in this embodiment, which is a prerequisite for realizing high-robustness wake-up recognition. The process generates difficult negative samples through the generator, forces the discriminator to learn more fine speech representation boundaries, and finally retains the discriminator D as the encoder used in the present application. The false wake-up rate is significantly reduced (especially in the context of similar words such as "Hey / Hi" and "Xiao Ling / Xiao Ling"); the stability of the system under noise, accent, and speech rate changes is improved; and only the discriminator is retained after training, which can realize lightweight deployment.

[0067] Further, the loss function of the above-described discriminator is a GE2E loss, including: a weighted sum of the global GE2E loss and each local GE2E loss.

[0068] The above-mentioned GE2E (Generalized End-to-End) loss is a kind of metric learning loss function, the goal is to make the embedding vectors of the same speaker as close as possible, and the embedding vectors of different speakers as far as possible, the formula is where e is the query embedding, is the same class reference centroid, that is, the reference feature center point of "the same speaker" to which the query embedding e belongs, is all reference centroids, which improves the intra-class compactness and inter-class separability of the embedding space.

[0069] The global GE2E loss refers to the GE2E loss calculated based on the global to-be-recognized vector and the global centroid c g , which captures the overall speech feature , where is the global centroid of all samples in a batch. In a batch, there are 10 users, and each user has 5 speeches. The similarity between and the 10 global centroids is calculated, and the correct user is optimized and matched. The overall pronunciation style feature is learned.

[0070] The local GE2E loss is to calculate the GE2E loss for each local segment k, which captures the local pronunciation details, and the formula is: where The local centroid of the sample. For example, calculate the loss of the "Hey", "small", "spirit" three segments respectively, and ensure that each syllable can be accurately identified. Enhance the robustness to local pronunciation variation.

[0071] The weighted sum loss function is to fuse the global and local losses by weight to form the final discriminator loss, where λ g = 0.6, = 0.4 / K. This step can balance the global and local information and improve the embedding quality.

[0072] This embodiment defines the core loss function of the discriminator (i.e. the encoder), adopts the weighted GE2E loss, combines global and local information, ensures that the encoder learns multi-level and high discriminative speech embedding, and can improve the discriminative ability of feature embedding; Support subsequent dynamic drift centroid update and wake-up judgment; Form a closed loop optimization with the above-mentioned adversarial training.

[0073] In one embodiment, the above callback updates the dynamic drift centroid, comprising: The updated dynamic drift centroid is constrained by the following callback formula:

[0074] Wherein: is the callback dynamic drift centroid; is the updated dynamic drift centroid; is the static anchor centroid; is the drift distance, which measures the drift degree, such as =[0.81,-0.11,…], =[0.80,-0.10,…], then d = 0.014; is a nonlinear callback intensity function that controls the pullback intensity, which is defined as: wherein: is the distance threshold; γ>0 is the growth rate parameter, such as γ=10, etc.; When is less than , ≈0, almost no callback; When → , →0.5; When is much larger than , →1, strong callback; Transition smoothing, avoid sudden changes. For example, =0.06, greater than = 0.05, ≈ 0.73, the centroid is pulled back to 73% of the distance, achieving gradual constraint.

[0075] The embodiment defines a callback mechanism of the dynamically drifting centroid, checks the drift distance after each update, and if the threshold is exceeded, the centroid is pulled back to the original static anchor point through a nonlinear function, achieving the balance between "personalization and security". Preventing the centroid from deviating from the original features due to long-term use, resisting continuous imitation attacks, and improving the long-term stability of the system.

[0076] Referring to Figure 2 The embodiment of the application also provides a self-defined wake-up word recognition device based on a dual-mode centroid, which is used to implement the self-defined wake-up word recognition method based on the dual-mode centroid of any of the above embodiments, and includes: The encoding unit 10 is configured to encode the received voice based on an encoder to obtain a to-be-recognized vector; The first calculation unit 20 is configured to perform similarity calculation on the to-be-recognized vector and a dynamically drifting centroid to obtain a wake-up similarity, wherein the initial value of the dynamically drifting centroid is a preset static anchor centroid; The first judgment unit 30 is configured to judge whether the wake-up similarity is greater than a first similarity threshold; The updating unit 40 is configured to update the dynamically drifting centroid based on the to-be-recognized vector if the wake-up similarity is greater than the first similarity threshold; The second judgment unit 50 is configured to judge whether the current state triggers a drift constraint operation; The second calculation unit 60 is configured to calculate the drift distance between the dynamically drifting centroid after updating and the static anchor centroid if the current state triggers the drift constraint operation; The callback unit 70 is configured to callback the dynamically drifting centroid after updating if the drift distance is greater than a distance threshold.

[0077] Referring to Figure 3 The embodiment of the application also provides a computer device, the internal structure of which can be as shown in Figure 3As shown. The computer device includes a processor, a memory, a network interface and a database connected by a system bus. Among them, the processor of the computer device is designed to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium, an internal memory. The non-volatile storage medium stores operating devices, computer programs and databases. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device is used to store encoders and the like. The network interface of the computer device is used to communicate with external terminals through network connection. Further, the above-mentioned computer device can also be provided with input devices and display screens and the like. The above-mentioned computer program is executed by the processor to realize a kind of self-defined wake-up word recognition method based on double mode centroid, the method comprises: based on the encoder, the received voice is encoded, and the to-be-identified vector is obtained;Similarity calculation is carried out between the to-be-identified vector and dynamic drift centroid, and the wake-up similarity is obtained, wherein the initial value of the dynamic drift centroid is the preset static anchor centroid;Judge whether the wake-up similarity is greater than the first similarity threshold value;If the wake-up similarity is greater than the first similarity threshold value, then update the dynamic drift centroid based on the to-be-identified vector;Judge whether the current state triggers drift constraint operation;If the current state triggers drift constraint operation, then calculate the drift distance between the updated dynamic drift centroid and the static anchor centroid;If the drift distance is greater than distance threshold, then call back the updated dynamic drift centroid. Those skilled in the art can understand, Figure 3 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied.

[0078] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation using the content of the specification and drawings of the present application, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A custom wake word recognition method based on dual-mode centroid, characterized in that, The method includes: The received speech is encoded using an encoder to obtain the vector to be recognized; The similarity between the vector to be identified and the dynamic drift centroid is calculated to obtain the wake-up similarity, wherein the initial value of the dynamic drift centroid is a preset static anchor centroid; Determine whether the wake-up similarity is greater than a first similarity threshold; If the wake-up similarity is greater than the first similarity threshold, then the dynamic drift centroid is updated based on the vector to be identified; Determine whether the current state triggers a drift constraint operation; If the current state triggers a drift constraint operation, then calculate the drift distance between the updated dynamic drift centroid and the static anchor centroid. If the drift distance is greater than the distance threshold, the updated dynamic drift centroid is called back.

2. The custom wake word recognition method based on dual-mode centroid according to claim 1, characterized in that, The process of encoding the received speech using an encoder to obtain a vector to be recognized includes: The received speech is globally attention encoded based on the global attention head in the encoder to obtain a global vector to be recognized; The received speech is locally attention encoded based on the K local attention heads in the encoder to obtain K local vectors to be recognized, where K is an integer greater than 1; The global vector to be identified and the K local vectors to be identified are combined to form the vector to be identified.

3. The custom wake-word recognition method based on dual-mode centroid according to claim 2, characterized in that, The step of calculating the similarity between the vector to be identified and the dynamically drifting centroid to obtain the wake-up similarity includes: The global wake-up similarity is obtained by calculating the similarity between the global vector to be identified and the global dynamic drift centroid in the dynamic drift centroid. The similarity between the K local vectors to be identified and the K local dynamic drift centroids in the dynamic drift centroid is calculated to obtain the K local wake-up similarity. The wake-up similarity is obtained by weighting the global wake-up similarity and the K local wake-up similarities.

4. The custom wake-word recognition method based on dual-mode centroid according to claim 1, characterized in that, The step of updating the dynamically drifting centroid based on the vector to be identified includes: Update the dynamically drifting centroid using the following formula: in: The updated dynamic drift centroid; The dynamic drift centroid; The vector to be identified; This is the learning rate.

5. The custom wake-up word recognition method based on dual-mode centroid according to claim 2, characterized in that, The encoder is a discriminator trained on an adversarial network, wherein the adversarial network includes a generator and a discriminator. The generator is used to generate obfuscated audio as difficult negative samples, and the discriminator is used to distinguish between real audio and obfuscated audio.

6. The custom wake-word recognition method based on dual-mode centroid according to claim 5, characterized in that, The encoder obtained by training based on adversarial networks includes: Construct a discriminator consisting of a global attention head, K local attention heads, and an output layer; and construct a generator to generate confusing audio samples that are similar to real audio as difficult negative samples; The generator is fixed and the initial encoder is trained, and the discriminator is fixed and the generator is trained; wherein the number of iterations and the termination condition are determined based on experimental data. After training, the trained discriminator is retained as the encoder.

7. The custom wake-word recognition method based on dual-mode centroid according to claim 6, characterized in that, The loss function of the discriminator is the GE2E loss, which includes: The weighted sum of the global GE2E loss and the local GE2E losses.

8. A custom wake-word recognition device based on dual-mode centroid, characterized in that, include: The encoding unit is used to encode the received speech based on the encoder to obtain the vector to be recognized; The first calculation unit is used to calculate the similarity between the vector to be identified and the dynamic drift centroid to obtain the wake-up similarity, wherein the initial value of the dynamic drift centroid is a preset static anchor centroid; The first judgment unit is used to determine whether the wake-up similarity is greater than a first similarity threshold; An update unit is used to update the dynamic drift centroid based on the vector to be identified if the wake-up similarity is greater than a first similarity threshold. The second judgment unit is used to determine whether the current state triggers the drift constraint operation; The second calculation unit is used to calculate the drift distance between the updated dynamic drift centroid and the static anchor centroid if the current state triggers a drift constraint operation. A callback unit is used to call back the updated dynamic drift centroid if the drift distance is greater than a distance threshold.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the custom wake word recognition method based on dual-mode centroid as described in any one of claims 1 to 7.

Images