A voice wake-up method and system capable of quickly customizing wake-up words
Through the technology combined with deep neural network and embedded model, the information security, user experience and cost problems of the voice wake-up system are solved, and the support of quickly customized wake-up words and dialect wake-up words is realized, expanding the market for voice interaction products.
Patent Information
- Application Number
- CN202410567198.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-09
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-05-09
AI Technical Summary
The existing voice wake-up system has information security problems, poor user experience, high cost, uncustomized wake-up words, and the Chinese dialect wake-up word technology has not been effectively solved, which limits the popularity of voice interaction products and user experience.
Using an embedded model based on deep neural network and a single-layer full-connect layer model, the feature extractor, scale module and CRNN model are trained in combination with open source Chinese voice data sets to realize offline voice wake-up and allow users to customize wake-up words. On the Android side, the system power consumption and cost are reduced.
It realizes rapid customization of wake-up words, improves response speed and user experience, reduces system power consumption and cost, supports dialect wake-up words, and expands the user group and product market.
Smart Images

Figure CN118248131B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of digital signal processing, deep learning and Android development technology, and in particular to a voice wake-up method and system capable of quickly customizing wake-up words. Background Art
[0002] Voice wake-up is a prerequisite for many voice interaction products. Before users use voice interaction products, they generally need to use specific wake-up words to wake up the device or application from standby mode and then perform subsequent voice control or voice recognition.
[0003] In the current market, most voice wake-up systems have pre-set wake-up words. These products use pre-set wake-up words, which may cause personal devices to be woken up by other users, bringing information security issues. Moreover, these wake-up words require standard Mandarin pronunciation, making it difficult for people with inaccurate pronunciation to effectively use voice interaction products. In addition, users cannot change the wake-up words according to their own preferences, which affects the user experience to a certain extent.
[0004] In recent years, leading companies in the voice industry have also launched customized wake-up word services, but this service is mainly for enterprises, with high costs and difficult to promote to a large number of products. In addition, this service is limited to Mandarin wake-up words. If users need to define wake-up words for dialects (such as Chaoshan dialect and Wenzhou dialect), the effect remains to be verified, which further limits the promotion of customized wake-up word services.
[0005] The core technology of voice wake-up is speech recognition, in which computers analyze the user's voice input and convert it into text. Speech recognition technology is based on acoustic models and language models, and is trained with a large amount of voice data to recognize and understand different categories of voice input. These models usually use pattern recognition algorithms, such as deep neural networks and hidden Markov models, to improve recognition accuracy.
[0006] In order to achieve voice wake-up, it is necessary to add the wake-up word detection function on the basis of voice recognition technology. Keyword Spotting is a technology that can accurately identify specific wake-up words in a continuous voice stream. It can recognize and respond to specific wake-up words, such as "Hey Siri" or "Xiao Ai", to activate the device and start voice interaction.
[0007] The mainstream voice wake-up process is as follows: Figure 1As shown, the device's microphone first receives the user's voice data and captures candidate segments of the wake-up word (there may be multiple segments) based on the algorithm. After simple local preprocessing, the device sends the data to the server through the network, recognizes the wake-up result based on the server-side acoustic model and language model, and then transmits the result back to the device through the network. The device determines whether to respond based on the recognition result.
[0008] Since the voice wake-up process needs to process a small amount of data and has a high response speed requirement, more and more products deploy voice wake-up algorithms locally on the device, that is, offline voice wake-up. However, this situation poses a challenge to the wake-up algorithm and model. In order to reduce power consumption and cost, most devices have limited hardware resources and cannot deploy and run models with large parameters and high complexity. Simple small models are difficult to achieve a satisfactory wake-up effect. Therefore, this offline voice wake-up solution usually requires a trade-off between wake-up effect and power consumption cost.
[0009] In addition, most voice interaction products have preset wake-up words, and users generally cannot customize wake-up words according to their preferences. In the field of deep learning, if a new wake-up system needs to be developed, developers need to collect a large amount of new wake-up word data, annotate and process the data, and then retrain the wake-up model. This process is time-consuming and cumbersome. Some companies have proposed a customized wake-up word business. This type of business mainly develops a set of more efficient acoustic models and language models and deploys them in offline devices. Users only need to enter the new wake-up word text to quickly customize a new wake-up system. This Query by String (QbyS) customized wake-up word method requires users to pronounce standardly during voice wake-up, which has a poor user experience. In addition, this type of algorithm still has the problem of resource constraints, that is, it cannot be deployed in devices with low cost and power consumption.
[0010] Finally, compared with English wake-up words, Chinese wake-up word technology faces some unique challenges. Chinese is a syllabic language with rich syllable combinations and tone changes, which makes the acoustic model and language model for training Chinese wake-up words more complex. In order to meet these challenges, large-scale Chinese speech data collection and annotation are required, and model architectures and training algorithms suitable for the characteristics of Chinese speech are required. At present, the mainstream Chinese speech wake-up word type is Mandarin, and no research has been found involving algorithms for quickly customizing dialect wake-up words. The combination of dialect wake-up and dialect recognition will make voice interaction more humane and improve user experience. Therefore, customized dialect wake-up words are of great significance and have broad market prospects. Summary of the invention
[0011] In order to solve the above problems, the present invention provides a voice wake-up method that can quickly customize the wake-up word, comprising the following steps:
[0012] On the computer side, based on the deep neural network model composed of feature extractor, scaling module and CRNN model, the open source Chinese speech dataset is used for training to obtain the embedding model;
[0013] On the Android side, deploy the embedding model and add a single fully connected layer to the output of the embedding model. Train the single fully connected layer by setting a new wake-up word with low-dimensional background noise characteristics to build a wake-up model.
[0014] On the Android side, voice input information is collected, the new wake-up words in the voice input information are identified through the wake-up model, and the corresponding voice wake-up function is executed.
[0015] Preferably, in the process of building a deep neural network model on the computer side, acoustic features including Mel-spectrogram and Mel-cephalogram coefficients are obtained through a feature extractor.
[0016] Preferably, in the process of building a deep neural network model on the computer side, the Mel-spectrogram and the Mel-cepstral coefficients are fused through a scaling module, wherein the scaling module is a CNN module.
[0017] Preferably, in the process of building a deep neural network model on the computer side, the fusion result is input into the CRNN model to capture and learn features across frequency and time dimensions, wherein the CRNN model is composed of a convolutional neural network and a recurrent neural network.
[0018] Preferably, in the process of obtaining an open source Chinese speech dataset, an open source Chinese speech dataset is generated according to the Multilingual Spoken WordsCorpus dataset, the aidattang_200zh dataset, the AISHELL-3 dataset and the SHALCAS22A dataset, and a background noise dataset is set, the open source Chinese speech dataset is enhanced in the time domain, and the embedding model is trained.
[0019] Preferably, in the process of constructing the embedding model, the time domain and time-frequency domain of the data are enhanced, and the deep neural network model is trained based on the enhanced data, wherein the time domain enhancement includes random time shift and random noise, and the time-frequency domain enhancement includes random time mask, random frequency mask and cutout.
[0020] Preferably, in the process of enhancing the time domain and time-frequency domain of the data, two neural network models with untrainable parameters are constructed according to the process of time domain enhancement and time-frequency domain enhancement and sent to the Android end.
[0021] Preferably, in response to building a wake-up model on the Android side, on the computer side, several pieces of noise data are set, feature extraction is performed through the embedded model, noise samples are generated, and sent to the Android side for model training; on the Android side, data enhancement is performed on the new wake-up word based on a neural network model with two untrainable parameters, and a single-layer fully connected layer is trained in combination with the noise samples to build a wake-up model for identifying new wake-up words.
[0022] The present invention discloses a voice wake-up system capable of quickly customizing a wake-up word, comprising:
[0023] The training module is used to be deployed on the computer side, based on the deep neural network model composed of the feature extractor, the scaling module and the CRNN model, and is trained using the open source Chinese speech dataset to obtain the embedding model;
[0024] The embedding module interacts with the training module for data. It is used to be placed on the Android side to deploy the embedding model and add a single fully connected layer to the output of the embedding model. The single fully connected layer is trained by setting a new wake-up word with low-dimensional background noise characteristics to build a wake-up model.
[0025] The wake-up module interacts with the embedded module for data collection on the Android side, identifies new wake-up words in the voice input information through the wake-up model, and executes the corresponding voice wake-up function.
[0026] Preferably, the training module is used to obtain acoustic features, including Mel-spectrogram and Mel-cepstral coefficients, through a feature extractor;
[0027] A training module, used for fusing the Mel spectrogram and the Mel cepstral coefficients through a scaling module, wherein the scaling module is a CNN module;
[0028] The training module is used to input the fusion results into the CRNN model to capture and learn features across frequency and time dimensions, wherein the CRNN model consists of a convolutional neural network and a recurrent neural network.
[0029] The present invention discloses the following technical effects:
[0030] The technical solution designed by the present invention is an Android application, which realizes offline voice wake-up on the local device without relying on the network, thereby achieving a faster response speed and protecting user privacy;
[0031] The deep neural network model designed by the present invention has a simple structure and a small number of parameters, thus reducing the power consumption and cost of the system;
[0032] The scale module proposed in the present invention is used to integrate multiple acoustic features, which enhances the learning ability and robustness of the model and ensures a better awakening effect, namely, a high awakening rate and a low false awakening rate;
[0033] The present invention combines the acoustic feature extraction process with the CRNN neural network model structure to form an end-to-end wake-up model. When implementing the algorithm on the Android side, only a single model needs to be deployed without additional signal processing steps, which simplifies the engineering implementation.
[0034] The present invention proposes a solution for quickly customizing wake-up words based on transfer learning. This solution is simple and feasible, and can be deployed in a large number of voice interaction products, greatly broadening the voice product market. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0036] Figure 1 This is the mainstream voice wake-up process described in the present invention;
[0037] Figure 2 It is a process overview of the implementation scheme described in the present invention;
[0038] Figure 3 The embedded model structure and the module structure therein described in the present invention;
[0039] Figure 4 It is a schematic diagram of the method flow of the present invention. DETAILED DESCRIPTION
[0040] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.
[0041] like Figure 1-4As shown, the present invention provides a voice wake-up technology that can quickly customize wake-up words, and the problems existing in the prior art are: the mainstream voice wake-up solution relies on the network for data transmission. When the network is congested, the response speed slows down, affecting the user experience. At the same time, the transmission of voice data through the network involves user privacy and puts forward requirements for information security; the offline voice wake-up algorithm that does not rely on the network deploys the acoustic model and language model on the local device side, which will consume a lot of equipment resources and increase the power consumption and cost of the system; the customized wake-up word business of the leading companies in the voice industry is mainly for corporate users and cannot be promoted to a large number of voice interaction products; the function of customizing Chinese voice wake-up words is currently limited to Mandarin, and no research on dialect wake-up words has been carried out, which limits the relevant applications and markets of customized voice wake-up words, among which, the wake-up rate: the recognition accuracy rate of the wake-up word by the wake-up system; the false wake-up rate: the number of times the wake-up system is awakened in a specific time without a wake-up word; the response speed: the speed at which the system responds to the user after the user says the wake-up word. In order to solve the above problems, the present invention belongs to an offline voice wake-up algorithm, which does not rely on the network, and aims to achieve a faster response speed and protect user privacy; the present invention focuses on designing an efficient voice wake-up algorithm and model, that is, maintaining a high wake-up rate and a low false wake-up rate while reducing the number of model parameters, so as to reduce system power consumption and cost; the present invention proposes a simple and easy-to-use fast customization of wake-up words solution, aiming to deploy the user-defined wake-up word function to a large number of voice interaction products; the present invention customizes the wake-up words based on the Query by Example (QbyE) method instead of QbyS, taking into account dialect wake-up, and aims to expand the user group, especially the elderly and children who do not speak standard Mandarin, improve user experience, and expand the product market.
[0042] The technical solution used in the present invention mainly involves three parts, such as Figure 2 The invention discloses a method for training a deep neural network model proposed in the present invention using an open source Chinese speech data set on a computer to obtain an embedding model that can represent the features of 191 types of Chinese words; the method is implemented on an Android side, specifically, the embedding model obtained in the first part is deployed to an Android platform, and a fully connected layer is added to the output end of the embedding model to form a wake-up model, and the wake-up model is trained on the Android side to obtain a wake-up model customized for a new wake-up word; the method is implemented in a third part, wherein the trained wake-up model is saved on the Android side for reasoning, testing and use on the Android side.
[0043] The above three parts of the process are described in detail as follows:
[0044] The first part is to train the embedding model on the computer:
[0045] Prepare data: The sample data used in this step comes from the following four Chinese open source speech datasets: Multilingual Spoken Words Corpus, aidatang_200zh, AISHELL-3, SHALCAS22A. In order to achieve data enhancement more effectively, the present invention screened the samples in these datasets, selected samples within a specific duration and volume range, and finally obtained 7690 Chinese word samples, a total of 191 categories of Chinese words. In addition, the present invention also selected suitable samples from the noise dataset TUT Acoustic Scenes 2017 as the background noise dataset_1, part of which is used for time domain enhancement, and the other part is used to train the embedding model. The sample size statistics of the dataset are shown in Table 1.
[0046] Table 1 Training data of embedding model
[0047]
[0048] Building a model: The embedding model structure proposed in this invention is as follows Figure 3 As shown in the left figure, the model parameters are 167k, and the module structure inside the model is as follows Figure 3 As shown in the right figure of , the specific structural parameters are shown in Table 2. The embedding model mainly consists of the following three parts:
[0049] Feature extractor: This part implements signal processing operations inside the model to extract common acoustic features: Mel-spectrogram and Mel-frequency cepstral coefficients (MFCC);
[0050] Scaling module: This is a simple and very effective module based on Convolutional Neural Network (CNN) proposed in the present invention. Its main function is to integrate the two types of features, Mel spectrogram and Mel cepstral coefficient, to improve the model effect;
[0051] CRNN model: This is the core part of the embedding model, which can effectively capture and learn features across frequency and time dimensions, further improving the effect. It is composed of CNN and recurrent neural networks (RNN), so it is also called convolutional recurrent neural networks (CRNN).
[0052] Table 2 Structural parameters of the embedded model
[0053]
[0054] Training process: The data set is divided into training set, validation set and test set in a ratio of 7:2:1. The training set is used to train the model, and the validation set is used to evaluate the model during the training process. Finally, the model with the highest accuracy on the validation set is used for independent testing on the test set to obtain the test set accuracy.
[0055] The data length accepted by the model is fixed at 2.5 seconds, so the Chinese word samples and noise samples with a duration of less than 2.5 seconds are padded (the filling value is 0), and the noise samples with a duration of more than 2.5 seconds are randomly intercepted for 2.5 seconds of data.
[0056] In order to improve the robustness of the embedded model, the time domain and time-frequency domain of the data are enhanced during the training process. Time domain enhancement includes random time shift, random noise, etc., and time-frequency domain enhancement includes random time mask, random frequency mask and cutout. In actual operation, the present invention defines these two types of enhancement methods as two neural network models with untrainable parameters, with the aim of simplifying the deployment work on the Android side.
[0057] Training results: The embedding model has an accuracy of 78.61% and an area under the ROC surface (AUC) of 0.9941 on an independent test set.
[0058] Part 2: Android training wake-up model:
[0059] Design a software based on the Android terminal. First, click the "Model Training" button to enter the training interface:
[0060] Prepare data: First enter the "Sample Collection" interface, fill in the speaker's information, and click the "Record Audio" button to start collecting new wake-up word data. The sample file will be automatically saved after 2.5 seconds. The file naming format is: wake-up word_timestamp_speaker name_speaker age_speaker gender.wav. The new wake-up word in the example is "Test it". You can click "Play" to play back the saved sample file, or click "Delete" to delete it.
[0061] Then, exit the "Sample Collection" interface, return to the training interface, and then enter the "File Selection" interface, select at least five new wake-up word samples for training, click OK, and the interface will automatically return to the training interface.
[0062] It is worth pointing out that the present invention uses an embedded model to perform feature extraction on 5000 noise data (background noise data set_3) on the computer side in advance, obtains a low-dimensional (128-dimensional) feature file and saves it on the Android side for training the wake-up model. Since the noise data duration is 2.5 seconds, it is 40,000 data points when the sampling rate is 16000Hz, and by saving the low-dimensional feature file, the present invention compresses the data points to 128, and the compression rate is about 312. This method greatly saves storage space. In addition, the present invention also stores 300 noise samples on the Android side for data enhancement in the process of training the wake-up model on the Android side, i.e., background noise data set_2.
[0063] Building the model:
[0064] The parameters of the embedded model trained on the computer are frozen and deployed to the Android side. The embedded model file name used in the present invention is "enable Train Model.tflite". At the same time, the two neural network models with untrainable parameters are also deployed on the Android side to perform the same data enhancement operation, where the present invention only trains a single fully connected layer.
[0065] Training process: Click the "Start Training" button on the training interface to start the data enhancement process. In this process, the selected wake-up word samples are enhanced to 300, and then after the feature extraction operation of the embedded model, 300 low-dimensional (128-dimensional) new wake-up word enhanced features are obtained. Then the training process begins. This process randomly selects 300 features from the 5,000 stored low-dimensional background noise features, and the 300 wake-up word features obtained by the above data enhancement, that is, a total of 600 feature data are used to train a single-layer fully connected layer. It should be noted that during the training process, data enhancement is performed online. In other words, the 300 wake-up word features used in each round of training are randomly generated, which helps to enhance the robustness of the model.
[0066] Training results: The training results (loss value and accuracy) curve will be plotted after the training is completed. From the results, it can be seen that the training process converges quickly. After the training of a single fully connected layer is completed, its parameters are frozen and merged with the embedded model to form a new wake-up model, which is saved as a model file for subsequent model reasoning and testing.
[0067] Part 3: Android test wake-up model:
[0068] Return to the APP homepage, select the correct wake-up model, and click the "Start Test" button. The Android microphone will start to continuously receive the input of ambient sound. The background algorithm will infer and judge the ambient sound every 1.25 seconds, and return the recognition result and confidence level for display. If it is judged as a wake-up word, the wake-up word text "Test it" will be displayed, and the speaker will play a female voice "Hey", otherwise it will display "noise".
[0069] In order to verify the actual effect of the present invention, the present invention invited 10 volunteers (5 boys and 5 girls, aged 19 to 22 years old) to evaluate the application, and each volunteer defined the Mandarin wake-up word according to his or her own preferences. In the new wake-up model obtained, click "Start Test", read the wake-up word 100 times, record the recognition results each time, and calculate the recognition accuracy as shown in Table 3.
[0070] Table 3 Evaluation results of Mandarin wake-up model on Android
[0071]
[0072]
[0073] Finally, in order to illustrate the generalization effect of the present invention on dialect wake-up words, the present invention invited 8 volunteers (4 boys and 4 girls, aged 19 to 22 years old) to define the wake-up words in Chaoshan dialect according to their own preferences, and tested the wake-up model in the same way as above, and the recognition accuracy was statistically shown in Table 4.
[0074] Table 4 Evaluation results of the Chaoshan dialect wake-up model on Android
[0075] Custom keywords Accuracy Take a bath 100% Hug, hug, hug 92% Get up. Get up. 85% Hello, hello 91% Drink tea Drink tea 95% Eat Eat 90% Sleep Sleep 90% Mobile Phone 95%
[0076] The experimental results show that the wake-up model customized according to the solution of the present invention is also compatible with dialects to a certain extent, which improves the user experience, further expands the user group and product market, and solves the following problems, including:
[0077] 1. In the current commercial solutions for customized wake-up words, the neural network model has a complex structure and requires a lot of hardware resources, which increases the power consumption and cost of the system. However, the neural network model proposed in this invention is simple and efficient, which reduces resource usage while ensuring the wake-up effect, which is largely due to the scale module that can integrate multiple acoustic features;
[0078] 2. Currently, most voice wake-up algorithms and solutions include two steps: voice signal processing (especially feature extraction) and deep learning based on neural network models. During the development process, engineers need to have background knowledge of signal processing and the ability to deploy neural network models on the development platform, which undoubtedly increases the development difficulty and prolongs the development time. However, the present invention combines the above two steps into one and constructs an end-to-end wake-up model. Therefore, during the development stage, engineers only need to focus on how to deploy the end-to-end neural network model, which greatly reduces the development difficulty and speeds up the project progress.
[0079] 3. The leading enterprises in the voice industry, such as iFlytek's customized wake-up word business, mainly use enterprises. Its use cost is high and cannot cover small enterprises and cheap products. The fast customized wake-up word solution proposed in the present invention is developed based on the Chinese open source data set, which enables engineers with a certain technical foundation to use the method of the present invention to carry out customized wake-up word business, which lowers the industry threshold and expands the product market;
[0080] 4. As far as we know, there are no products involving dialect wake-up, and the combination of dialect wake-up and dialect recognition will have broad application prospects. Therefore, the compatibility of the present invention with dialect wake-up words will expand the user group and improve the user experience.
[0081] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0082] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.
[0083] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A voice wake-up method capable of quickly customizing a wake-up word, characterized in that: The following steps are involved: On the computer side, based on the deep neural network model composed of feature extractor, scaling module and CRNN model, the open source Chinese speech dataset is used for training to obtain the embedding model; On the Android side, the embedding model is deployed, and a single fully connected layer is added to the output end of the embedding model, and the single fully connected layer is trained by setting a new wake-up word with low-dimensional background noise characteristics to build a wake-up model; On the Android terminal, voice input information is collected, the new wake-up word in the voice input information is identified through the wake-up model, and a corresponding voice wake-up function is executed; In the process of building a deep neural network model on the computer side, the acoustic features including Mel-spectrogram and Mel-cepstral coefficients are obtained through the feature extractor; In the process of building a deep neural network model on a computer, the Mel spectrogram and the Mel cepstral coefficients are fused through the scaling module, wherein the scaling module is a CNN module; The scale module includes a scale module 1 and a scale module 2, wherein the scale module 1 is used to process the Mel spectrogram and the scale module 2 is used to process the Mel cepstral coefficient; wherein, The scale module 1 is composed of Conv2D Block 1#1 and Conv2D Block 1#2, wherein the number of channels of Conv2D Block 1#1 and Conv2D Block 1#2 is 4, the kernel size is (5,5), the step length is (1,1), and the pooling size is (1,2); The scale module 2 is composed of Conv2D Block 2#3 and Conv2D Block 2#4, wherein the number of channels of Conv2D Block 2#3 and Conv2D Block 2#4 is 4, the kernel size is (5,5), and the step length is (1,1); Conv2D Block 1 consists of Conv2D-BatchNorm-ReLU, MaxPool2D, and Dropout; Conv2D Block 2 consists of Conv2D-BatchNorm-ReLU and Dropout; In the process of building a deep neural network model on the computer side, the fusion result is input into the CRNN model to capture and learn features across frequency and time dimensions, wherein the CRNN model is composed of a convolutional neural network and a recurrent neural network; The recurrent neural network is composed of Conv2D Block 1#5, Conv2D Block 1#6, Conv2D Block 1#7, bidirectional GRU, Dense Block#1 and Dense Block#2, wherein the Dense Block is composed of Dense-BatchNorm-ReLU and Dropout, wherein the number of channels of Conv2D Block 1#5 is 64, the kernel size is (5,5), the stride length is (1,1), and the pooling size is (1,2); Conv2D Block 1#6 and Conv2D Block 1#7 have 32 channels, a kernel size of (3,3), a stride length of (1,1), and a pooling size of (1,2).
2. A voice wake-up method capable of quickly customizing a wake-up word according to claim 1, characterized in that: In the process of obtaining the open source Chinese speech dataset, the open source Chinese speech dataset is generated according to the Multilingual Spoken Words Corpus dataset, the aidattang_200zh dataset, the AISHELL-3 dataset and the SHALCAS22A dataset, and a background noise dataset is set, the open source Chinese speech dataset is enhanced in the time domain, and the embedding model is trained.
3. According to claim 2, a voice wake-up method capable of quickly customizing a wake-up word is characterized in that: In the process of building the embedding model, the time domain and time-frequency domain of the data are enhanced, and the deep neural network model is trained based on the enhanced data, wherein the time domain enhancement includes random time shift and random noise addition, and the time-frequency domain enhancement includes random time mask, random frequency mask and cutout.
4. According to claim 3, a voice wake-up method capable of quickly customizing a wake-up word is characterized in that: In the process of enhancing the time domain and time-frequency domain of the data, two neural network models with untrainable parameters are constructed according to the process of time domain enhancement and time-frequency domain enhancement and sent to the Android terminal.
5. According to claim 4, a voice wake-up method capable of quickly customizing a wake-up word is characterized in that: In response to building a wake-up model on the Android side, a number of noise data are set on the computer side, feature extraction is performed through the embedding model, noise samples are generated, and sent to the Android side for model training; on the Android side, data enhancement is performed on the new wake-up word based on a neural network model with two untrainable parameters, and the single-layer fully connected layer is trained in combination with the noise samples to build a wake-up model for identifying new wake-up words.
6. A voice wake-up system capable of quickly customizing a wake-up word, used in a voice wake-up method capable of quickly customizing a wake-up word as claimed in any one of claims 1 to 5, characterized in that: include: The training module is used to be deployed on the computer side, based on the deep neural network model composed of the feature extractor, the scaling module and the CRNN model, and is trained using the open source Chinese speech dataset to obtain the embedding model; An embedding module, which interacts with the training module for data, is arranged on an Android terminal, deploys the embedding model, adds a single fully connected layer at the output of the embedding model, trains the single fully connected layer by setting a new wake-up word with low-dimensional background noise characteristics, and constructs a wake-up model; A wake-up module, which interacts with the embedded module for collecting voice input information on the Android terminal, identifying the new wake-up word in the voice input information through the wake-up model, and executing a corresponding voice wake-up function; The training module is used to obtain acoustic features through the feature extractor, including Mel-spectrogram and Mel-cepstral coefficients; The training module is used to fuse the Mel-spectrogram and the Mel-cepstral coefficients through the scaling module, wherein the scaling module is a CNN module; The training module is used to input the fusion result into the CRNN model to capture and learn features across frequency and time dimensions, wherein the CRNN model is composed of a convolutional neural network and a recurrent neural network; The scale module includes a scale module 1 and a scale module 2, wherein the scale module 1 is used to process a mel-spectrogram, and the scale module 2 is used to process a mel-cepstral coefficient; wherein the scale module 1 is composed of Conv2DBlock1#5 and Conv2DBlock1#5; the scale module 2 is composed of Conv2DBlock2#3 and Conv2DBlock2#4; Conv2DBlock1 is composed of Conv2D-BatchNorm-ReLU, MaxPool2D and Dropout; Conv2DBlock2 is composed of Conv2D-BatchNorm-ReLU and Dropout; The recurrent neural network consists of Conv2DBlock1#5, Conv2DBlock1#6, Conv2DBlock1#7, bidirectional GRU, DenseBlock#1 and DenseBlock#2, wherein DenseBlock consists of Dense-BatchNorm-ReLU and Dropout.
Citation Information
Patent Citations
Voice awakening system and method
CN111653276A
Voice wake-up method based on deep neural network in noise environment
CN114067789A
Low-resource voice keyword detection method based on unsupervised learning and transfer learning
CN116434742A
Guest greeting robot and dialect interaction system and method
CN116721656A
Lightweight personalized voice wake-up method, device and equipment
CN116884407A
