A wake-up word recognition method, device and terminal equipment

Abstract

The application provides a wakeword recognition method and device and a terminal device, and is suitable for the technical field of data processing. The method comprises the following steps: generating human voice recognition information according to to-be-recognized voice information and a preset human voice recognition model; generating human voice conversion text information according to the human voice recognition information and a preset human voice text conversion model; and generating wakeword information according to the human voice conversion text information, a wakeword recognition model and a preset wakeword set. The application realizes layer-by-layer analysis from voice data to effective wakeword information, effectively filters the interference of environmental noise on the recognition calculation process, improves the integrity and stability of wakeword recognition, and provides data support for high-precision wakeword trigger judgment.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, and in particular relates to wake word recognition methods, devices and terminal equipment. Background Technology

[0002] Wake word recognition, as a front-end technology for human-computer voice interaction systems such as smart speakers, smart home terminals, in-vehicle voice devices, and wearable smart hardware, plays a core role in triggering device wake-up and enabling interaction permissions, and has now become a standard basic module for consumer-grade smart hardware.

[0003] In existing technologies, GMM-HMM statistical modeling is usually used for wake word recognition. However, using a single standard wake word template for matching is not enough to adapt to multiple forms of wake words, resulting in low wake word recognition accuracy. Summary of the Invention

[0004] In view of this, embodiments of this application provide a wake word recognition method, apparatus, and terminal device, aiming to solve the problems of insufficient adaptability to multiple forms of wake words and low wake word recognition accuracy in the prior art.

[0005] The first aspect of this application provides a wake-up word recognition method, including: Acquire multiple speech information to be recognized and a wake word recognition model; Based on the multiple voice information to be recognized and the preset voice recognition model, multiple voice recognition information is generated; Based on the multiple voice recognition information and the preset voice-to-text conversion model, multiple voice-to-text conversion information is generated; Based on the multiple voice-to-text conversion information, the wake word recognition model, and the preset wake word set, multiple wake word information is generated.

[0006] A second aspect of this application provides a wake-word recognition device, comprising: The module for acquiring speech information to be recognized and wake word recognition model is used to acquire multiple speech information to be recognized and wake word recognition model. The voice recognition information generation module is used to generate multiple voice recognition information based on the multiple voice information to be recognized and the preset voice recognition model; The voice-to-text conversion module is used to generate multiple voice-to-text conversion information based on the multiple voice recognition information and the preset voice-to-text conversion model. The wake-up word information generation module is used to generate multiple wake-up word information based on the multiple human voice-converted text information, the wake-up word recognition model, and the preset wake-up word set.

[0007] A third aspect of this application provides a terminal device, the terminal device including a memory and a processor, the memory storing a computer program executable on the processor, the processor executing the computer program to implement the steps of the wake word recognition method as described in the first aspect above.

[0008] A fourth aspect of this application provides a computer-readable storage medium, comprising: storing a computer program, wherein when executed by a processor, the computer program implements the steps of the wake word recognition method described in the first aspect above.

[0009] Compared with the prior art, the beneficial effects of the embodiments of this application are: this application realizes the layer-by-layer parsing from voice data to effective wake word information, effectively filters the interference of environmental noise on the recognition calculation process, improves the integrity and stability of wake word recognition, and provides data support for high-precision wake-up trigger determination. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram illustrating the implementation process of the wake word recognition method provided in Embodiment 1 of this application; Figure 2 This is a schematic diagram illustrating the implementation process of the wake word recognition method provided in Embodiment 2 of this application; Figure 3 This is a schematic diagram illustrating the implementation process of the wake word recognition method provided in Embodiment 3 of this application; Figure 4 This is a schematic diagram illustrating the implementation process of the wake word recognition method provided in Embodiment 4 of this application; Figure 5 This is a schematic diagram illustrating the implementation process of the wake word recognition method provided in Embodiment 5 of this application; Figure 6 This is a schematic diagram illustrating the implementation process of the wake word recognition method provided in Embodiment Six of this application; Figure 7 This is a schematic diagram illustrating the implementation process of the wake word recognition method provided in Embodiment 7 of this application; Figure 8 This is a schematic diagram of the wake word recognition device provided in the embodiments of this application; Figure 9 This is a schematic diagram of the terminal device provided in the embodiments of this application. Detailed Implementation

[0012] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0013] To illustrate the technical solution described in this application, specific embodiments are provided below.

[0014] Figure 1 The implementation flowchart of the wake word recognition method provided in Embodiment 1 of this application is shown, and is described in detail below: Step S101: Obtain multiple voice information to be recognized and a wake word recognition model.

[0015] In this embodiment, the speech information to be recognized can be user-generated speech data collected in real time by the microphone array of a smart device. This data can include various types of speech segments, such as clear speech at close range, attenuated speech at long distance, and speech superimposed with environmental noise. This data forms the basis for wake-up word recognition and can comprehensively recreate the user's actual speech scenario. Multiple speech information to be recognized can refer to speech data collected at multiple different sampling times. The wake-up word recognition model can be pre-set by the user. It can be a pre-trained, lightweight deep learning model for speech feature recognition, which can adapt to the local computing scenarios of the terminal device and accurately match and analyze subsequent text data to provide algorithmic support for wake-up word recognition.

[0016] In this embodiment, the wake word recognition model can be a lightweight model such as CNN, 3DCNN, LSTM, BiLSTM, Transformer, or Whisper.

[0017] Step S102: Generate multiple voice recognition information based on the multiple voice information to be recognized and the preset voice recognition model.

[0018] In this embodiment, the preset voice recognition model can be manually preset, and a deep learning model for voice activity detection can be used. Then, the preset voice recognition model can be used to analyze multiple voice information to be recognized frame by frame, distinguish between valid voice signals and environmental noise signals, filter out user voice frequency band data, and remove invalid interference signals such as wind noise, environmental noise, and equipment background noise. Then, the filtered voice signals are subjected to signal-to-noise ratio quantization, voice frequency band marking, and interference signal labeling. Finally, the analysis results of multiple frames of voice are integrated to generate multiple voice recognition information, accurately distinguishing between valid voices and invalid noise, and removing interference factors for subsequent speech-to-text processing.

[0019] In this embodiment, the preset voice recognition model can be a VAD voice activity detection network, CNN, LSTM, BiLSTM, ECAPA-TDNN, U-Net, lightweight Transformer, or WebRTC model.

[0020] Step S103: Generate multiple voice-converted text information based on the multiple voice recognition information and the preset voice-to-text conversion model.

[0021] In this embodiment, the preset human voice to text conversion model can be preset by humans. An end-to-end speech-to-text deep learning model can be adopted. Multiple human voice recognition information can be filtered and input into the preset human voice to text conversion model to perform temporal decomposition and semantic analysis on continuous human voice audio signals, thereby achieving accurate mapping of audio signals to text content, generating multiple human voice converted text information, and realizing the effective conversion of human voice speech to standard text data.

[0022] In this embodiment, the preset human voice to text conversion model can be Whisper, Conformer, end-to-end LAS model, CNN+LSTM hybrid speech to text model, RNN-T, and lightweight Transformer model.

[0023] Step S104: Generate multiple wake word information based on the multiple human voice-converted text information, the wake word recognition model, and the preset wake word set.

[0024] In this embodiment, the preset wake word set can be manually preset. The wake word set can include all the standard wake text data preset by the user. Then, multiple voice-converted text information can be used as input data and input into the trained wake word recognition model. The wake word recognition model then performs semantic comparison, keyword retrieval, and content matching analysis on the text content. Finally, it combines the preset wake word set to complete the wake word screening and judgment of the entire text, thereby generating multiple wake word information to accurately identify the effective wake word content contained in the user's voice.

[0025] The wake word recognition method provided in this application realizes the layer-by-layer parsing from voice data to effective wake word information, effectively filters the interference of environmental noise on the recognition calculation process, improves the integrity and stability of wake word recognition, and provides data support for high-precision wake-up trigger determination.

[0026] Figure 2 The flowchart illustrating the implementation of the wake word recognition method provided in Embodiment 2 of this application is shown. Its difference from Embodiment 1 described above lies in: The preset human voice text conversion information includes a preset human voice text conversion generation sub-model, a preset human voice text conversion sentence segmentation sub-model, a preset human voice text conversion dialect correction sub-model, and a preset human voice text conversion tail sound missing information supplementation sub-model; Step S103 specifically includes: Step S201: Generate multiple voice-to-text conversion information to be optimized based on the multiple voice recognition information and the preset voice-to-text conversion sub-model.

[0027] In this embodiment, the preset voice-to-text generation sub-model can be manually preset and can adopt the Whisper lightweight basic transcription deep learning model. The model confidence parameter can be set to 0.85. Then, multiple voice recognition information that has been filtered can be input into the voice-to-text generation sub-model in batches. The voice-to-text generation sub-model then performs semantic transcription, initial character matching and preliminary text generation processing on the effective voice temporal signals. Finally, it outputs the original text content without sentence optimization, dialect correction and end-sound completion, thereby generating multiple voice-to-text information to be optimized, providing the original text data foundation for subsequent multi-layer text fine optimization processing.

[0028] Step S202: Based on the multiple voice-to-text information to be optimized and the preset voice-to-text segmentation model, generate multiple voice-to-text information after segmentation.

[0029] In this embodiment, the preset speech-to-text segmentation model can be manually preset and can adopt the Conformer temporal segmentation deep learning model. The sentence pause judgment threshold parameter can be set to 0.3 seconds. Then, multiple speech-to-text information to be optimized can be input into the speech-to-text segmentation model. The speech-to-text segmentation model then combines the temporal pause features and semantic paragraph features of human speech to intelligently segment and character-segment the continuous original text, splitting long and continuous text segments, removing invalid spliced ​​characters, and then outputting well-structured and clearly segmented sentence text content, thereby generating multiple segmented speech-to-text information to solve the problems of speech-to-text sentence adhesion and lack of segmentation.

[0030] Step S203: Based on the multiple sentence-segmented human voice-converted text information and the preset human voice-converted text dialect correction sub-model, generate multiple dialect-corrected human voice-converted text information.

[0031] In this embodiment, the preset speech-to-text dialect correction sub-model can be pre-defined and can adopt an RNN-T deep learning model. The dialect semantic matching threshold parameter can be set to 0.8. Then, multiple speech-to-text information after sentence segmentation can be input into the speech-to-text dialect correction sub-model. Subsequently, the speech-to-text dialect correction sub-model uses a massive dialect-to-standard Mandarin mapping dataset learned during training to identify dialect words and accent deviations in the text and convert them into text. It performs standardization correction and semantic error correction on dialect-distorted text, and then uniformly converts non-standard dialect text into standard written text content, thereby generating multiple dialect-corrected speech-to-text information to eliminate the influence of regional dialects on the accuracy of speech-to-text conversion.

[0032] Step S204: Generate multiple voice-converted text information based on the multiple dialect-corrected voice-converted text information and the preset voice-converted text tail sound missing information supplementation sub-model.

[0033] In this embodiment, the preset sub-model for supplementing missing information at the end of a speech-to-text conversion can be manually preset and can adopt a CNN+LSTM hybrid deep learning model. The threshold parameter for identifying missing sounds at the end of a speech can be set to 0.25. Then, multiple dialect-corrected speech-to-text conversion information can be input into the sub-model for supplementing missing information at the end of a speech-to-text conversion. The sub-model then identifies text defects and semantic gaps caused by attenuation of the end of a speech sound or incomplete sound reception. It combines contextual semantic features to intelligently complete and repair missing text and words, and then outputs complete, accurate, and defect-free standardized text content, thereby generating multiple speech-to-text conversion information to achieve accurate conversion of speech to text throughout the entire process.

[0034] The wake word recognition method provided in this application is used to solve the problems of speech confusion, dialect deviation and text incompleteness in the process of speech transcription, which greatly improves the integrity and accuracy of human voice converted text information, provides high-quality text data support for subsequent accurate wake word matching and recognition, and effectively improves the adaptability and recognition accuracy of wake word recognition in complex human voice scenarios.

[0035] Figure 3 The flowchart illustrating the implementation of the wake word recognition method provided in Embodiment 3 of this application is shown. Its difference from Embodiment 1 described above lies in: The preset wake word set includes a preset standard wake word set, a preset synonym variant wake word set, and a preset accent distortion wake word set; Step S104 specifically includes: Step S301: Generate multiple wake-up word information to be matched based on the multiple human voice converted text information and the wake-up word recognition model.

[0036] In this embodiment, the wake word recognition model can be pre-set by the user. A lightweight Transformer semantic matching deep learning model can be used, and the text feature extraction accuracy parameter can be set to 0.9. Multiple voice-converted text information can be uniformly input into the wake word recognition model. Then, the wake word recognition model performs global semantic feature extraction, keyword feature clustering, and effective semantic screening on the standardized text, removing invalid tone words and redundant modifiers from the text. Then, the core semantic segments with wake word features in the text are extracted, thereby generating multiple wake word information to be matched, which prepares for multi-dimensional wake word set matching and screening.

[0037] Step S302: Perform matching processing based on the multiple wake-up word information to be matched and the preset standard wake-up word set to generate multiple standard wake-up word matching information.

[0038] In this embodiment, the preset standard wake-up word set can be preset by humans. The standard wake-up word matching accuracy parameter built into the set can be set to 0.95. Then, multiple wake-up word information to be matched can be compared and matched word by word and semantically by the full set of standard wake-up text in the preset standard wake-up word set. Then, the consistency is checked by referring to the text content, semantic definition and sentence structure of the standard wake-up words. Finally, the text fragments that are completely matched and highly consistent are marked, thereby generating multiple standard wake-up word matching information to achieve accurate recognition and matching of conventional standard wake-up words.

[0039] Step S303: Perform matching processing based on the multiple wake-up word information to be matched and the preset set of synonym variant wake-up words to generate multiple synonym variant wake-up word matching information.

[0040] In this embodiment, the preset set of synonym variant wake words can be preset by humans. The set has a built-in synonym semantic matching threshold parameter that can be set to 0.88. Then, the wake word information that has not matched the standard wake word can be semantically similarly matched with the preset set of synonym variant wake words. Then, the semantic association algorithm is used to identify synonym replacement text and near-synonymous text that are semantically consistent with the standard wake word but have different expressions. Then, the semantically equivalent wake word variant content is determined, thereby generating multiple synonym variant wake word matching information to cover the synonym expression type wake word recognition scenario.

[0041] Step S304: Perform matching processing based on the multiple wake-up word information to be matched and the preset set of accent distortion wake-up words to generate multiple accent distortion wake-up word matching information.

[0042] In this embodiment, the preset set of accent distortion wake words can be preset by humans. The built-in accent distortion fault-tolerant matching parameter of the set can be set to 0.8. Then, the remaining unmatched wake word information to be matched can be fuzzy matched with the preset set of accent distortion wake words. Then, for the text distortion caused by the user's accent deviation, unclear pronunciation, and tone shift, fault-tolerant semantic matching and feature comparison processing are performed. Then, wake word content with pronunciation distortion and text deviation but consistent core semantics is identified, thereby generating multiple accent distortion wake word matching information to adapt to the wake word recognition scenario under accent interference.

[0043] Step S305: Generate multiple wake word information based on the multiple standard wake word matching information, multiple synonym variant wake word matching information, and multiple accent distortion wake word matching information.

[0044] In this embodiment, the wake word fusion judgment weight parameters can be set to 0.5 for standard matching, 0.3 for synonym matching, and 0.2 for aberration matching. This allows for the weight fusion, result integration, and deduplication of multiple standard wake word matching information, multiple synonym variant wake word matching information, and multiple accent aberration wake word matching information. This process eliminates duplicate recognition results, filters invalid matching data, and summarizes all valid wake word recognition results to complete the comprehensive judgment of wake words in multiple scenarios and multiple forms, thereby generating multiple wake word information and achieving accurate wake word recognition output across all scenarios.

[0045] The wake word recognition method provided in this application is used to cover various user speech scenarios such as standard pronunciation, synonyms, and accent distortion. It effectively solves the problems of limited recognition scenarios of single matching patterns, poor accent adaptability, and missed recognition of variant wake words, and comprehensively improves the coverage, error tolerance and recognition accuracy of wake word recognition, and adapts to various complex actual speech usage scenarios.

[0046] Figure 4 The flowchart illustrating the implementation of the wake word recognition method provided in Embodiment 4 of this application is shown. The difference between this method and Embodiment 1 is that step S104 specifically includes: Step S401: Based on the preset wake word text semantic feature extraction rules, perform text semantic feature extraction processing according to the preset wake word set to generate multiple wake word text semantic feature information.

[0047] In this embodiment, the preset rules for extracting semantic features of wake word text can be preset manually. A text deep semantic parsing standardization process can be adopted, and the semantic feature extraction granularity parameter can be set to 0.92. All wake word text data within the preset wake word set can be traversed. Then, the text content of standard wake words, synonym variant wake words, and accent distortion wake words is extracted and structured in a comprehensive manner using lexical features, sentence structure features, and semantic association features. Finally, the high-dimensional semantic features of wake word text are aggregated to generate multiple wake word text semantic feature information, providing standard wake word feature samples for subsequent bidirectional feature matching and model training.

[0048] In this embodiment, the semantic feature extraction granularity parameter can be determined to be 0.92 by calculating the effective proportion of sample features. Specifically, this involves calculating the ratio of the number of effective core semantic features of all standard wake words, synonym variant wake words, and accent distortion wake words in the preset wake word set to the total number of features. This quantifies the optimal feature extraction granularity. Then, all subclass wake word text data under the preset wake word set are traversed in batches. Subsequently, a full-dimensional feature extraction operation is performed on each type of wake word text, including keyword feature decomposition, sentence structure feature parsing, variant semantic association mapping, and accent deviation semantic normalization. Then, redundant and invalid features are removed, and the feature dimensions and representation specifications of all wake word samples are unified, thereby generating structured and highly discriminative semantic feature information of multiple wake word texts, providing accurate standard feature basis for subsequent model semantic matching training.

[0049] Step S402: Perform semantic feature conversion processing on the multiple voice-converted text information to generate multiple voice-converted text semantic feature information.

[0050] In this embodiment, Conformer+BiLSTM can be used to perform semantic feature conversion processing. The text semantic encoding dimension parameter is set to 128 dimensions. Multiple voice-converted text information can be uniformly input into Conformer+BiLSTM to perform context semantic association parsing, keyword weight assignment, and invalid semantic filtering. Then, the natural language text is converted into structured and standardized high-dimensional semantic feature data, thereby generating multiple voice-converted text semantic feature information, realizing the accurate conversion of user-spoken text into semantic features that the model can recognize.

[0051] Step S403: Based on the semantic feature information of the multiple wake word texts and the semantic feature information of the multiple human voice converted texts, the wake word recognition model is trained to generate a trained wake word recognition model.

[0052] In this embodiment, the feature fusion model used for training can be pre-defined and can employ a multi-head attention Transformer deep learning model. The model training iteration count parameter can be set to 100 rounds, and the model convergence error threshold parameter can be set to 0.001. Then, the wake word text semantic feature information can be used as standard feature samples, and the human voice converted text semantic feature information can be used as training feature samples for pairing input. Subsequently, the model completes the association alignment, difference correction, and feature fusion processing of the two types of features, continuously iteratively updates the internal feature matching parameters of the model, and then continuously reduces the error between the predicted features and the standard wake word features until the model parameters converge, thereby generating a trained wake word recognition model, which greatly improves the model's ability to learn wake word semantic features in different scenarios.

[0053] Step S404: Generate multiple wake word information based on the semantic feature information of the human voice converted text and the trained wake word recognition model.

[0054] In this embodiment, the inference confidence threshold of the post-trained wake word recognition model can be preset by the user, and can be set to 0.9. Multiple human voice-converted text semantic feature information can be input into the converged post-trained wake word recognition model. Then, the post-trained wake word recognition model uses the semantic matching rules and feature association logic learned during training to perform wake word adaptation judgment, effective feature filtering, and invalid feature removal on the semantic features of the input text. Finally, it outputs the accurately matched wake word recognition result, thereby generating multiple wake word information to realize real-time wake word recognition inference on the terminal.

[0055] The wake word recognition method provided in this application realizes refined learning and accurate matching of wake word semantic features, effectively solves the problems of weak semantic understanding and insufficient feature adaptability of wake word recognition models in the prior art, and significantly improves the recognition accuracy and stability in complex spoken language scenarios and variant wake word scenarios.

[0056] Figure 5 The flowchart illustrating the implementation of the wake word recognition method provided in Embodiment 5 of this application is shown. The difference between this method and Embodiment 4 above is that step S403 specifically includes: Step S501: Calculate the similarity based on the semantic features of the multiple wake-up word texts and the semantic features of the multiple human voice conversion texts to obtain the semantic feature similarity information between the multiple human voice texts and the wake-up word texts.

[0057] In this embodiment, cosine similarity calculation can be used to perform global feature comparison, dimensional feature matching, and difference quantification on each set of wake-up word text semantic feature information and human voice conversion text semantic feature information. Then, the correlation similarity between the two sets of semantic features can be accurately calculated, and the similarity value of each set of samples can be quantified and output, thereby obtaining multiple human voice text and wake-up word text semantic feature similarity information, providing accurate quantitative basis for subsequent feature selection.

[0058] Step S502: Based on the preset semantic feature similarity threshold information between the human voice text and the wake-up word text, the semantic feature similarity information between the multiple human voice texts and the wake-up word texts is filtered to obtain multiple filtered semantic feature similarity information between the human voice text and the wake-up word texts.

[0059] In this embodiment, the preset semantic feature similarity threshold between the voice text and the wake-up word text can be preset manually, and can be set to 0.75. Then, multiple semantic feature similarity information between the voice text and the wake-up word text can be compared with the preset semantic feature similarity threshold one by one. Then, highly correlated feature samples with similarity values ​​greater than the preset threshold are retained, while invalid and low-correlation feature samples with similarity values ​​lower than the preset threshold are removed. Then, the high-quality feature samples are screened and collected, thereby obtaining multiple filtered semantic feature similarity information between the voice text and the wake-up word text, filtering out invalid training samples, and improving the model training accuracy.

[0060] Step S503: The semantic feature information of multiple wake-up word texts corresponding to the semantic feature similarity information of multiple filtered human voice texts and wake-up word texts, as well as the semantic feature information of multiple human voice converted texts, are spliced ​​together to generate multiple spliced ​​human voice texts and wake-up word texts semantic feature information.

[0061] In this embodiment, a dual-channel feature parallel splicing mode can be adopted, and the feature splicing alignment accuracy parameter can be set to 0.96. Then, the selected highly correlated wake word text semantic feature information and human voice converted text semantic feature information can be dimensionally aligned, positionally matched, and spliced ​​in parallel. Then, the bidirectional feature information of standard wake word features and user actual speech text features can be integrated to generate multiple spliced ​​human voice text and wake word text semantic feature information to enrich the feature dimensions of model training.

[0062] Step S504: Based on the semantic feature information of the multiple concatenated human voice texts and wake-up word texts, the wake-up word recognition model is trained to generate a trained wake-up word recognition model.

[0063] In this embodiment, a Transformer deep learning model optimized with residual networks can be used for incremental training of the model. The incremental training learning rate parameter can be set to 0.0001. Then, multiple concatenated human voice texts and wake word texts can be input into the wake word recognition model in batches. Subsequently, the wake word recognition model can be incrementally iteratively trained, parameter updated, and feature adapted and optimized through multiple concatenated human voice texts and wake word texts. This is used to enhance the wake word recognition model's feature recognition ability for highly similar spoken wake-up scenarios and variant wake-up scenarios, thereby generating a trained wake word recognition model to optimize the model's generalization ability and recognition accuracy.

[0064] The wake word recognition method provided in this application eliminates low-quality training samples, enriches the model feature learning dimensions, and specifically enhances the wake word recognition model's adaptability to spoken language-differentiated wake-up scenarios, effectively reducing the probability of wake word misrecognition and missed recognition, thereby improving the scenario adaptability and recognition accuracy of the wake word recognition model.

[0065] Figure 6 The flowchart illustrating the implementation of the wake word recognition method provided in Embodiment Six of this application is shown. The difference between this method and Embodiment Four is that step S403 specifically includes: Step S601: Perform feature encoding and matrix transformation processing on the multiple human voice-converted text semantic feature information to generate multiple human voice-converted text semantic feature matrix information.

[0066] In this embodiment, the feature encoding matrix transformation can be implemented using a CNN deep learning model. The matrix dimension unification parameter can be 64×128. This allows for the structured encoding, dimension normalization, and matrix mapping transformation of multiple human voice-converted text semantic feature information. Subsequently, the one-dimensional sequence semantic features are uniformly converted into standardized two-dimensional feature matrix data, which is used to achieve the structured normalization of all text semantic features, thereby generating multiple human voice-converted text semantic feature matrix information to adapt to the subsequent matrix mapping operations and model training requirements.

[0067] Step S602: Based on multiple preset human voice-to-text semantic feature mapping matrices, the information of the multiple human voice-to-text semantic feature matrices is mapped to generate multiple mapped human voice-to-text semantic feature matrix information.

[0068] In this embodiment, multiple preset human voice-to-text semantic feature mapping matrices can be pre-set by humans and can adopt a self-attention learnable parameter matrix group. Then, the human voice-to-text semantic feature matrix information can be reconstructed, dimension mapped, and weighted through multiple sets of semantic feature mapping matrices to mine the deep correlation information of text semantic features, weaken the interference of invalid features, and realize the deep optimization mapping of semantic features, thereby generating multiple mapped human voice-to-text semantic feature matrix information to improve the representation ability of text semantic features.

[0069] Step S603: Based on the semantic feature information of the multiple wake-up words and the semantic feature matrix information of the multiple mapped human voice converted text, the wake-up word recognition model is trained to generate a trained wake-up word recognition model.

[0070] In this embodiment, the ECAPA-TDNN deep learning model can be used for feature matching training. The batch parameter of the model training can be 32. Multiple wake word text semantic feature information and multiple mapped human voice converted text semantic feature matrix information can be used for feature matching training. Then, the alignment logic, matching weight and association rules of the two types of features are continuously optimized through the ECAPA-TDNN model. The wake word recognition model parameters are then iteratively updated until the training converges, thereby generating a trained wake word recognition model to improve the wake word recognition model's ability to match and recognize differentiated speech text features.

[0071] The wake word recognition method provided in this application enhances the ability of the wake word recognition model to extract and match deep semantic features, effectively improves the limitations of shallow feature recognition in existing technologies, and significantly improves the accuracy and stability of wake word recognition in complex spoken language scenarios.

[0072] Figure 7 The flowchart illustrating the implementation of the wake word recognition method provided in Embodiment 7 of this application is shown. Its difference from Embodiment 6 described above lies in: Multiple preset human voice-to-text semantic feature mapping matrices include a preset human voice-to-text semantic feature retrieval query matrix, a preset human voice-to-text semantic feature retrieval key-value matrix, and a preset human voice-to-text semantic feature retrieval value matrix; Step S602 specifically includes: Step S701: Generate multiple voice-to-text semantic feature retrieval query matrix information and multiple voice-to-text semantic feature retrieval key value matrix information based on the multiple voice-to-text semantic feature matrix information, the preset voice-to-text semantic feature retrieval query matrix and the preset voice-to-text semantic feature retrieval key value matrix.

[0073] In this embodiment, the preset human voice-to-text semantic feature retrieval query matrix and the preset human voice-to-text semantic feature retrieval key-value matrix can be preset manually. Both can be learnable parameter matrices with a self-attention mechanism. Multiple human voice-to-text semantic feature matrix information can be processed by matrix mapping operations with the preset human voice-to-text semantic feature retrieval query matrix and the preset human voice-to-text semantic feature retrieval key-value matrix, respectively. This can be a multiplication operation. Then, the retrieval query features and associated key-value features of the text semantic features are extracted respectively, thereby generating multiple human voice-to-text semantic feature retrieval query matrix information and multiple human voice-to-text semantic feature retrieval key-value matrix information, which are used to provide basic matrix data for subsequent feature weight calculation.

[0074] Step S702: Multiply the multiple voice-to-text semantic feature retrieval query matrix information and the multiple voice-to-text semantic feature retrieval key value matrix information to obtain the multiple voice-to-text semantic feature retrieval weight matrix information.

[0075] In this embodiment, the matrix multiplication scaling factor can be set to 8.0. This allows for matrix multiplication of the voice-to-text semantic feature retrieval query matrix information with the voice-to-text semantic feature retrieval key value matrix information. Then, the feature correlation scaling factor is used to complete the feature correlation scaling process, avoiding the gradient vanishing problem caused by excessive feature dimensionality. Finally, the correlation weights between different semantic features are quantified, resulting in multiple voice-to-text semantic feature retrieval weight matrix information, thus achieving accurate quantification of text semantic feature correlation.

[0076] Step S703: Generate multiple voice-to-text semantic feature retrieval value matrix information based on the multiple voice-to-text semantic feature matrix information and the preset voice-to-text semantic feature retrieval value matrix.

[0077] In this embodiment, the preset human voice-to-text semantic feature retrieval value matrix can be preset by humans or can be a self-attention mechanism feature value parameter matrix. Then, multiple human voice-to-text semantic feature matrix information can be mapped and processed with the preset human voice-to-text semantic feature retrieval value matrix. Then, the core effective features in the text semantic features are selected and retained, and redundant and invalid features are eliminated. Then, the structured value of the core features is completed, thereby generating multiple human voice-to-text semantic feature retrieval value matrix information, providing accurate feature data for subsequent weighted fusion.

[0078] Step S704: Based on the weight matrix information of the multiple voice-to-text semantic feature retrieval, perform weighted calculation on the multiple voice-to-text semantic feature retrieval value matrix information to obtain the multiple voice-to-text semantic feature retrieval weight matrix information.

[0079] In this embodiment, a global adaptive weighted fusion approach can be adopted. By quantifying the weight values ​​of the human voice-to-text semantic feature retrieval weight matrix information, the human voice-to-text semantic feature retrieval value matrix information can be weighted and fused dimension by dimension and feature by feature. This strengthens highly correlated core semantic features and weakens low-correlation interference features, thereby achieving adaptive optimization fusion of features. This results in multiple human voice-to-text semantic feature retrieval weighted matrix information, which can be used to achieve refined weighted optimization of text semantic features.

[0080] Step S705: Based on the weighted matrix information retrieved from the multiple human voice-converted text semantic features, numerical normalization processing is performed to generate multiple mapped human voice-converted text semantic feature matrix information.

[0081] In this embodiment, the normalization process can be Sigmoid normalization, which can perform global numerical normalization, dimension unification, and data regularization on the weighted matrix information of multiple voice-converted text semantic feature retrieval. This constrains all feature values ​​to a standard range to eliminate training interference caused by differences in feature values ​​of different samples, thereby generating multiple mapped voice-converted text semantic feature matrix information and completing the final optimized mapping of text semantic features.

[0082] The wake word recognition method provided in this application comprehensively optimizes the representation effect of human voice text semantic features, effectively improves the wake word recognition model's ability to recognize subtle semantic differences, accent semantic deviations, and variant wake word semantics, and greatly enhances the accuracy, robustness, and scene adaptability of the wake word recognition model.

[0083] Corresponding to the method in the above embodiments, Figure 8 A structural block diagram of the wake word recognition device provided in the embodiments of this application is shown. For ease of explanation, only the parts related to the embodiments of this application are shown. Figure 8 The wake word recognition device in the example can be the execution subject of the wake word recognition method provided in the aforementioned embodiment 1.

[0084] Reference Figure 8 The wake word recognition device includes: The module 810 for acquiring speech information to be recognized and wake word recognition model is used to acquire multiple speech information to be recognized and wake word recognition models. The voice recognition information generation module 820 is used to generate multiple voice recognition information based on the multiple voice information to be recognized and the preset voice recognition model; The voice-to-text information generation module 830 is used to generate multiple voice-to-text information based on the multiple voice recognition information and the preset voice-to-text conversion model. The wake-up word information generation module 840 is used to generate multiple wake-up word information based on the multiple human voice-converted text information, the wake-up word recognition model, and the preset wake-up word set.

[0085] The process by which each module in the wake word recognition device provided in this application implements its respective function can be found in the foregoing. Figure 1 The description of Embodiment 1 shown will not be repeated here.

[0086] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0087] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0088] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0089] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0090] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance. It should also be understood that although the terms "first," "second," etc., are used in the text to describe various elements in some embodiments of this application, these elements should not be limited by these terms. These terms are merely used to distinguish one element from another. For example, a first table may be named a second table, and similarly, a second table may be named a first table, without departing from the scope of the various described embodiments. Both the first table and the second table are tables, but they are not the same table.

[0091] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0092] The wake word recognition method provided in this application can be applied to terminal devices such as mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality / virtual reality devices, laptops, super mobile personal computers, netbooks, and personal digital assistants. This application does not impose any restrictions on the specific type of terminal device.

[0093] For example, the terminal device may be a station in a WLAN, a cellular phone, a cordless phone, a session initiation protocol phone, a wireless local loop station, a personal digital processing device, a handheld device with wireless communication capabilities, a computing device or other processing device connected to a wireless modem, an in-vehicle device, a vehicle networking terminal, a computer, a laptop computer, a handheld communication device, a handheld computing device, a satellite wireless device, a wireless modem card, a set-top box, a user premises equipment, and / or other devices for communication over a wireless system, as well as next-generation communication systems, such as mobile terminals in 5G networks or mobile terminals in future evolved public terrestrial mobile networks, etc.

[0094] Figure 9 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. For example... Figure 9 As shown, the terminal device 9 of this embodiment includes: at least one processor 90 ( Figure 9 (Only one is shown in the image) A memory 91 stores a computer program 92 that can run on the processor 90. When the processor 90 executes the computer program 92, it implements the steps in the various wake word recognition method embodiments described above, for example... Figure 1 Steps S101 to S104 are shown. Alternatively, when the processor 90 executes the computer program 92, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 8 The functions of modules 810 to 840 are shown.

[0095] The terminal device 9 can be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor 90 and a memory 91. Those skilled in the art will understand that... Figure 9 This is merely an example of terminal device 9 and does not constitute a limitation on terminal device 9. It may include more or fewer components than shown, or combine certain components, or different components. For example, the terminal device may also include input transmission devices, network access devices, buses, etc.

[0096] The processor 90 may be a central processing unit, or it may be other general-purpose processors, digital signal processors, application-specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0097] In some embodiments, the memory 91 may be an internal storage unit of the terminal device 9, such as a hard disk or memory of the terminal device 9. The memory 91 may also be an external storage device of the terminal device 9, such as a plug-in hard disk, smart memory card, secure digital card, flash memory card, etc., equipped on the terminal device 9. Furthermore, the memory 91 may include both internal and external storage units of the terminal device 9. The memory 91 is used to store operating systems, applications, bootloaders, data, and other programs, such as the program code of computer programs. The memory 91 can also be used to temporarily store data that has been sent or will be sent.

[0098] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0099] This application also provides a terminal device, which includes at least one memory, at least one processor, and a computer program stored in the at least one memory and executable on the at least one processor. When the processor executes the computer program, it causes the terminal device to implement the steps in any of the above method embodiments.

[0100] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0101] This application provides a computer program product that, when run on a terminal device, enables the terminal device to implement the steps described in the various method embodiments above.

[0102] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0103] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0104] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0105] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0106] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A wake-word recognition method, characterized in that, include: Acquire multiple speech information to be recognized and a wake word recognition model; Based on the multiple voice information to be recognized and the preset voice recognition model, multiple voice recognition information is generated; Based on the multiple voice recognition information and the preset voice-to-text conversion model, multiple voice-to-text conversion information is generated; Based on the multiple voice-to-text conversion information, the wake word recognition model, and the preset wake word set, multiple wake word information is generated.

2. The wake word recognition method as described in claim 1, characterized in that, The preset human voice text conversion information includes a preset human voice text conversion generation sub-model, a preset human voice text conversion sentence segmentation sub-model, a preset human voice text conversion dialect correction sub-model, and a preset human voice text conversion tail sound missing information supplementation sub-model; The step of generating multiple voice-to-text information based on the multiple voice recognition information and the preset voice-to-text conversion model specifically includes: Based on the multiple human voice recognition information and the preset human voice-to-text generation sub-model, multiple human voice-to-text information to be optimized is generated. Based on the multiple voice-to-text information to be optimized and the preset voice-to-text sentence segmentation model, multiple voice-to-text information after sentence segmentation are generated. Based on the multiple sentence-segmented human voice-converted text information and the preset human voice-converted text dialect correction sub-model, multiple dialect-corrected human voice-converted text information are generated. Based on the dialect-corrected human voice converted text information and the preset human voice converted text tail sound missing information supplementation sub-model, multiple human voice converted text information are generated.

3. The wake word recognition method as described in claim 1, characterized in that, The preset wake word set includes a preset standard wake word set, a preset synonym variant wake word set, and a preset accent distortion wake word set; The step of generating multiple wake-up word information based on the multiple voice-converted text information, the wake-up word recognition model, and the preset wake-up word set specifically includes: Based on the multiple voice-to-text conversion information and the wake word recognition model, multiple wake word information to be matched is generated; The matching process is performed based on the multiple wake-up word information to be matched and the preset standard wake-up word set to generate multiple standard wake-up word matching information; The matching process is performed based on the multiple wake-up word information to be matched and the preset set of synonym variant wake-up words to generate multiple synonym variant wake-up word matching information; The matching process is performed based on the multiple wake-up word information to be matched and the preset set of accent distortion wake-up words to generate multiple accent distortion wake-up word matching information. Multiple wake word information is generated based on the multiple standard wake word matching information, multiple synonym variant wake word matching information, and multiple accent distortion wake word matching information.

4. The wake word recognition method as described in claim 1, characterized in that, The step of generating multiple wake-up word information based on the multiple voice-converted text information, the wake-up word recognition model, and the preset wake-up word set specifically includes: Based on the preset wake word text semantic feature extraction rules, text semantic feature extraction processing is performed according to the preset wake word set to generate multiple wake word text semantic feature information; Based on the multiple voice-converted text information, semantic feature conversion processing is performed to generate multiple voice-converted text semantic feature information. Based on the semantic features of the multiple wake words and the semantic features of the multiple human voice converted text, the wake word recognition model is trained to generate a trained wake word recognition model. Based on the semantic feature information of the human voice converted text and the trained wake word recognition model, multiple wake word information is generated.

5. The wake word recognition method as described in claim 4, characterized in that, The step of training the wake-up word recognition model based on the semantic features of the multiple wake-up word texts and the semantic features of the multiple human voice-converted texts to generate a trained wake-up word recognition model specifically includes: Based on the semantic features of the multiple wake-up word texts and the semantic features of the multiple human voice converted texts, similarity calculation is performed to obtain the semantic feature similarity information between the multiple human voice texts and the wake-up word texts; Based on the preset semantic feature similarity threshold information between human voice text and wake word text, the semantic feature similarity information between the multiple human voice text and wake word text is filtered to obtain multiple filtered semantic feature similarity information between human voice text and wake word text. The semantic feature information of multiple wake word texts corresponding to the semantic feature similarity information of multiple filtered human voice texts and wake word texts, as well as the semantic feature information of multiple human voice converted texts, are concatenated to generate multiple concatenated human voice texts and wake word texts semantic feature information. Based on the semantic feature information of the multiple concatenated human voice texts and wake-up word texts, the wake-up word recognition model is trained to generate a trained wake-up word recognition model.

6. The wake word recognition method as described in claim 4, characterized in that, The step of training the wake-up word recognition model based on the semantic features of the multiple wake-up word texts and the semantic features of the multiple human voice-converted texts to generate a trained wake-up word recognition model specifically includes: The semantic feature information of the multiple voice-converted text is subjected to feature encoding and matrix transformation to generate multiple semantic feature matrix information of voice-converted text. Based on multiple preset human voice-to-text semantic feature mapping matrices, the information of the multiple human voice-to-text semantic feature matrices is mapped to generate multiple mapped human voice-to-text semantic feature matrix information. Based on the semantic feature information of the multiple wake-up words and the semantic feature matrix information of the multiple mapped human voice converted text, the wake-up word recognition model is trained to generate a trained wake-up word recognition model.

7. The wake word recognition method as described in claim 6, characterized in that, Multiple preset human voice-to-text semantic feature mapping matrices include a preset human voice-to-text semantic feature retrieval query matrix, a preset human voice-to-text semantic feature retrieval key-value matrix, and a preset human voice-to-text semantic feature retrieval value matrix; The step of mapping multiple preset human voice-to-text semantic feature matrix information to generate multiple mapped human voice-to-text semantic feature matrix information specifically includes: Based on the multiple human voice-to-text semantic feature matrix information, the preset human voice-to-text semantic feature retrieval query matrix, and the preset human voice-to-text semantic feature retrieval key value matrix, multiple human voice-to-text semantic feature retrieval query matrix information and multiple human voice-to-text semantic feature retrieval key value matrix information are generated. The weight matrix information of multiple voice-to-text semantic feature retrieval query matrix information and multiple voice-to-text semantic feature retrieval key value matrix information are multiplied together to obtain the weight matrix information of multiple voice-to-text semantic feature retrieval. Based on the multiple human voice-to-text semantic feature matrix information and the preset human voice-to-text semantic feature retrieval value matrix, multiple human voice-to-text semantic feature retrieval value matrix information are generated. Based on the weight matrix information of the multiple voice-converted text semantic feature retrieval weight matrix, the multiple voice-converted text semantic feature retrieval value matrix information of the multiple voice-converted text semantic feature retrieval weight matrix is ​​calculated to obtain the multiple voice-converted text semantic feature retrieval weight matrix information. Based on the weighted matrix information retrieved from the multiple voice-converted text semantic features, numerical normalization processing is performed to generate multiple mapped voice-converted text semantic feature matrix information.

8. A wake-word recognition device, characterized in that, include: The module for acquiring speech information to be recognized and wake word recognition model is used to acquire multiple speech information to be recognized and wake word recognition model. The voice recognition information generation module is used to generate multiple voice recognition information based on the multiple voice information to be recognized and the preset voice recognition model; The voice-to-text conversion module is used to generate multiple voice-to-text conversion information based on the multiple voice recognition information and the preset voice-to-text conversion model. The wake-up word information generation module is used to generate multiple wake-up word information based on the multiple human voice-converted text information, the wake-up word recognition model, and the preset wake-up word set.

9. A terminal device, characterized in that, The terminal device includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images