Method, device and system for starting based on wake-up word

Abstract

The invention provides a method, a device and a system for starting based on a wake-up word. The method comprises the following steps: collecting a real-time audio; extracting audio features of the real-time audio; according to a pre-constructed speech recognition model, performing audio comparison on the audio features and a preset audio comparison library to obtain a comparison result; according to a voice recognition model, performing wake-up word confidence matching on the audio features to obtain a confidence matching result; when it is determined that the real-time audio is valid according to the comparison result and the matching result, wake-up word text meaning corresponding to the real-time audio is recognized according to a voice recognition model; and respectively sending alarm information including the wake-up word text meaning to a cloud end and a mobile end, so that the cloud end and the mobile end respectively execute corresponding alarm processing according to the alarm information. According to the method, user requirements can be automatically recognized based on a voice recognition mode, so that assistance timeliness and reliability in various life scenes are improved.

Description

Technical Field

[0001] The present application relates to the technical field, and in particular to a method, device and system for activation based on a wake-up word. Background Art

[0002] In daily life, many people may encounter sudden emergencies, such as sudden deterioration of health or urgent need for external assistance. If there is a lack of rapid and effective external assistance, the consequences are often disastrous. Although there are many technical tools designed to ensure personal safety on the market, these methods are still insufficient in practical application, especially in meeting the urgent needs for immediate and reliable safety responses in various life scenarios. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a method, device and system for activation based on a wake-up word, which can automatically identify user needs based on voice recognition, thereby improving the timeliness and reliability of assistance in various life scenarios.

[0004] In a first aspect, the present application provides a method for starting based on a wake-up word, which is applied to a terminal device and includes:

[0005] Collect real-time audio;

[0006] Extracting audio features of the real-time audio;

[0007] According to the pre-built speech recognition model, the audio feature is compared with a preset audio comparison library to obtain a comparison result;

[0008] According to the speech recognition model, performing a wake-up word confidence match on the audio feature to obtain a confidence match result;

[0009] When it is determined that the real-time audio is valid according to the comparison result and the matching result, identifying the text meaning of the wake-up word corresponding to the real-time audio according to the speech recognition model;

[0010] An alarm message including the text meaning of the wake-up word is sent to the cloud and the mobile terminal respectively, so that the cloud and the mobile terminal respectively perform corresponding alarm processing according to the alarm message.

[0011] Further, after identifying the text meaning of the wake-up word corresponding to the real-time audio according to the speech recognition model, the method further includes:

[0012] When the meaning of the wake-up word text matches the preset emergency wake-up word, a preset emergency preset number is obtained, and an operation of cyclically dialing the emergency preset number is performed.

[0013] Furthermore, before collecting the real-time audio, the method further includes:

[0014] Obtaining a preset wake-up word and the wake-up word text meaning of the preset wake-up word;

[0015] Obtaining audio recording data corresponding to the preset wake-up word;

[0016] When the audio recording data passes the qualified confirmation, the audio recording data is stored in the server database through the preset first interface, and an association relationship between the audio recording data and the meaning of the wake-up word text is established;

[0017] Acquire the audio recording data through a preset second interface;

[0018] When the audio recording data is confirmed through playback, the audio recording data, the preset wake-up word and the textual meaning of the wake-up word are stored in a preset audio comparison library.

[0019] Further, after obtaining the audio recording data corresponding to the preset wake-up word, the method further includes:

[0020] Playing back the audio recording data locally to obtain a playback confirmation result;

[0021] Determining whether the audio recording data is qualified according to the playback confirmation result;

[0022] If yes, the step of storing the audio recording data in a server database through a preset first interface is performed.

[0023] Furthermore, the obtaining of the audio recording data corresponding to the preset wake-up word includes:

[0024] Acquire speech recognition requirements; wherein the speech recognition requirements include at least one wake-up word, the installation environment of the terminal device, and the speech speed corresponding to the preset wake-up word; the speech speed includes fast speed, moderate speed, and slow speed;

[0025] The audio recording data corresponding to the preset wake-up word is obtained according to the voice recognition requirement.

[0026] Furthermore, the preset wake-up word is set based on different usage scenarios;

[0027] The text meaning of the preset wake-up word is set through the cloud or mobile terminal, and the text meaning of the preset wake-up word is stored in the server database.

[0028] Further, the extracting audio features of the real-time audio includes:

[0029] Performing denoising on the real-time audio to obtain a denoised audio signal;

[0030] The denoised audio signal is converted into a frequency spectrum feature map to obtain audio features.

[0031] Furthermore, performing wake-up word confidence matching on the audio feature according to the speech recognition model to obtain a confidence matching result includes:

[0032] The audio features are divided and processed according to a preset sliding window to obtain multiple audio stream segments;

[0033] The wake-up word is predicted for the audio stream segment according to the speech recognition model to obtain a confidence matching result; wherein the confidence matching result includes a wake-up word probability distribution or a wake-up word confidence score.

[0034] Further, the identifying the text meaning of the wake-up word corresponding to the real-time audio according to the speech recognition model includes:

[0035] Determining the detected wake-up word according to the confidence matching result through the speech recognition model;

[0036] The wake-up words are merged and filtered to obtain the text meaning of the wake-up words.

[0037] Furthermore, after storing the audio recording data, the preset wake-up word and the textual meaning of the wake-up word in a preset audio comparison library, the method further includes:

[0038] Acquire valid audio samples from the audio comparison library;

[0039] Preprocessing the valid audio sample to obtain a preprocessed audio sample;

[0040] Constructing a sample data set according to the preprocessed audio samples; wherein the sample data set includes a training set, a test set and a validation set;

[0041] Get pre-built deep convolutional neural networks;

[0042] The deep convolutional neural network is trained using the sample data set and a preset back propagation algorithm to obtain a confusion matrix generated after the training and a trained target model;

[0043] When it is determined according to the confusion matrix that the model performance of the speech recognition model is qualified, the target model is format-converted according to a preset model format to obtain a speech recognition model.

[0044] Furthermore, the training algorithm of the deep convolutional neural network includes a stochastic gradient descent method, a Dropout algorithm, and an adaptive learning rate algorithm;

[0045] The adaptive learning rate algorithm is used to adjust the learning rate of each model parameter in the deep convolutional neural network;

[0046] The Dropout algorithm is used to prevent the deep convolutional neural network from overfitting.

[0047] Further, the preprocessing the valid audio sample to obtain the preprocessed audio sample includes:

[0048] Performing a cropping process on the valid audio sample to obtain a first processed sample;

[0049] Performing denoising on the first processed sample by spectral subtraction to obtain a second processed sample;

[0050] performing smoothing processing on the second processed sample to obtain a third processed sample;

[0051] Perform analog-to-digital conversion on the third processed sample to obtain a preprocessed audio sample.

[0052] Further, the denoising process is performed on the first processed sample by spectral subtraction to obtain a second processed sample, including:

[0053] Performing short-time Fourier transform on the first processed sample by spectral subtraction to obtain a signal spectrum diagram;

[0054] Calculating the sample signal energy of the first processed sample according to the signal spectrum diagram;

[0055] performing noise estimation on the first processed sample to obtain noise signal energy;

[0056] Performing energy subtraction calculation on the first processed sample according to the sample signal energy and the noise signal energy to obtain a noise reduction signal;

[0057] Performing an inverse short-time Fourier transform on the noise reduction signal to obtain a second processed sample.

[0058] Further, the performing smoothing on the second processed sample to obtain a third processed sample includes:

[0059] calculating a maximum noise residual of the second processed sample based on a smoothing mechanism;

[0060] When the amplitude spectrum value of the second processed sample is less than the maximum noise residual, the amplitude spectrum value of the second processed sample is replaced to obtain a replacement signal;

[0061] Performing signal recovery processing on the replacement signal to obtain a recovery signal;

[0062] A speech spectrogram is plotted according to the replacement signal to obtain a third processed sample.

[0063] A second aspect of the present application provides a device for starting based on a wake-up word, wherein the device for starting based on a wake-up word is a terminal device, including:

[0064] A collection unit, used for collecting real-time audio;

[0065] A feature extraction unit, used to extract audio features of the real-time audio;

[0066] A comparison unit, used to perform an audio comparison between the audio feature and a preset audio comparison library according to a pre-built speech recognition model to obtain a comparison result;

[0067] A matching unit, configured to perform a wake-up word confidence match on the audio feature according to the speech recognition model to obtain a confidence matching result;

[0068] a meaning recognition unit, configured to recognize the meaning of the wake-up word text corresponding to the real-time audio according to the speech recognition model when it is determined that the real-time audio is valid according to the comparison result and the matching result;

[0069] The sending unit is used to send alarm information including the text meaning of the wake-up word to the cloud and the mobile terminal respectively, so that the cloud and the mobile terminal respectively perform corresponding alarm processing according to the alarm information.

[0070] A third aspect of the present application provides a system based on wake-up word activation, the system based on wake-up word activation includes a terminal device, a cloud and a mobile terminal, wherein:

[0071] The terminal device collects real-time audio;

[0072] The terminal device extracts audio features of the real-time audio;

[0073] The terminal device performs an audio comparison between the audio feature and a preset audio comparison library according to a pre-built speech recognition model to obtain a comparison result;

[0074] The terminal device performs a wake-up word confidence match on the audio feature according to the speech recognition model to obtain a confidence match result;

[0075] When the terminal device determines that the real-time audio is valid according to the comparison result and the matching result, identifying the text meaning of the wake-up word corresponding to the real-time audio according to the speech recognition model;

[0076] The terminal device sends an alarm message including the text meaning of the wake-up word to the cloud and the mobile terminal respectively;

[0077] After receiving the alarm information, the cloud obtains the location information of the terminal device and the video monitoring information corresponding to the location information according to the alarm information;

[0078] The cloud outputs the alarm information, the location information, and the video surveillance information;

[0079] After receiving the alarm information, the mobile terminal outputs the alarm information and performs a corresponding audio or video call operation on the terminal device according to the audio or video call instruction input by the user in response to the alarm information. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0081] Figure 1 A flowchart of a method for activating based on a wake-up word provided in an embodiment of the present application;

[0082] Figure 2 A flowchart of another method for activating based on a wake-up word provided in an embodiment of the present application;

[0083] Figure 3 A schematic diagram of an example process of wake-up word matching and recognition provided in an embodiment of the present application;

[0084] Figure 4 A partial flow chart of a method for starting based on a wake-up word provided in an embodiment of the present application;

[0085] Figure 5 A schematic diagram of another part of the flow chart of a method for starting based on a wake-up word provided in an embodiment of the present application;

[0086] Figure 6 A schematic diagram of an example process of implementing speech noise reduction using spectral subtraction provided in an embodiment of the present application;

[0087] Figure 7 A schematic diagram of an example process of starting based on a wake-up word provided in an embodiment of the present application;

[0088] Figure 8A schematic diagram of the structure of a device based on wake-up word activation provided in an embodiment of the present application;

[0089] Fig. 9 A schematic diagram of the structure of another device based on wake-up word activation provided in an embodiment of the present application;

[0090] Fig.10 A schematic diagram of the structure of a system based on wake-up word activation provided in an embodiment of the present application. DETAILED DESCRIPTION

[0091] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0092] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0093] Example 1

[0094] Please see Figure 1 , Figure 1 A flowchart of a method for activating a wake-up word provided in this embodiment. The method for activating a wake-up word is applied to a terminal device, including:

[0095] S101, collecting real-time audio.

[0096] S102: extracting audio features of real-time audio.

[0097] In this embodiment, the terminal device may not perform real-time recording based on the recorded audio, but may only recognize the partial audio of the wake-up word, and then perform a wake-up word detection comparison based on the partial audio of the wake-up word and the data in the model.

[0098] S103: According to the pre-built speech recognition model, the audio features are compared with a preset audio comparison library to obtain a comparison result.

[0099] In this embodiment, the method can collect audio in real time, and perform voiceprint comparison between the real-time audio and the data in the audio comparison library through an AI speech recognition model to obtain a comparison result.

[0100] In this embodiment, the method can be compared through terminal settings, or it can be uploaded to the cloud for comparison. In both methods, the model can be deployed to the terminal device or cloud environment by simply saving and converting it to the required format.

[0101] S104: Perform wake-up word confidence matching on the audio features according to the speech recognition model to obtain a confidence matching result.

[0102] S105: When the real-time audio is determined to be valid based on the comparison results and the matching results, the text meaning of the wake-up word corresponding to the real-time audio is identified based on the speech recognition model.

[0103] In this embodiment, the method automatically confirms the wake-up word after the voiceprint comparison is successful, and further confirms the textual meaning expressed by the wake-up word (ie, the textual meaning of the wake-up word).

[0104] S106. Send an alarm message including the text meaning of the wake-up word to the cloud and the mobile terminal respectively, so that the cloud and the mobile terminal perform corresponding alarm processing according to the alarm message.

[0105] In this embodiment, the method can immediately send out corresponding trigger information after confirming the wake-up word, upload the wake-up word recognized by the corresponding audio, and then synchronously remind the cloud and mobile terminal (the mobile terminal can perform corresponding operations through the APP or mini program), and can make audio and video calls with the cloud or mobile terminal (APP / mini program), and can make audio and video calls with multiple people and multiple devices at the same time.

[0106] In this embodiment, after receiving the trigger information of the terminal device, the cloud and mobile terminal (APP / mini program) will have corresponding reminders, and can see the location of the terminal device and the wake-up word currently triggered. At the same time, they can make phone calls according to different emergency situations, or use the mobile terminal (APP / mini program) to make audio and video calls to confirm and view the corresponding situations.

[0107] In this embodiment, after the audio and video call confirms the situation and handles the situation, the method also allows the wake-up record to be viewed on the cloud or mobile terminal to cancel the latest triggered wake-up. After canceling, the method can push the cancellation record to all guardian mobile terminals (applets / APPs) to easily remind other guardians so that they can understand the situation, or they can go up to view the video capture record when it is triggered when they are free.

[0108] In this embodiment, the terminal device may be a smart device such as a camera.

[0109] It can be seen that the method of starting based on the wake-up word described in this embodiment can automatically identify user needs based on voice recognition, thereby improving the timeliness and reliability of assistance in various life scenarios.

[0110] Example 2

[0111] Please see Figure 2 , Figure 2A flowchart of a method for activating a wake-up word provided in this embodiment. The method for activating a wake-up word is applied to a terminal device, including:

[0112] S201, collecting real-time audio.

[0113] In this embodiment, the method can collect real-time audio through a microphone array and perform preprocessing.

[0114] S202: Perform denoising on the real-time audio to obtain a denoised audio signal.

[0115] In this embodiment, the method can apply noise cancellation and echo cancellation technology to improve the quality of audio signals.

[0116] S203: Convert the denoised audio signal into a frequency spectrum feature map to obtain audio features.

[0117] In this embodiment, the method can convert the original audio signal into a frequency spectrum feature graph and use it as the audio feature.

[0118] S204: According to the pre-built speech recognition model, the audio features are compared with a preset audio comparison library to obtain a comparison result.

[0119] S205: Divide and process the audio features according to a preset sliding window to obtain multiple audio stream segments.

[0120] In this embodiment, the method can divide the audio stream into sliding windows of fixed length, each window length is 1 second, and then set the step size of the sliding window in the audio stream according to the sensitivity requirement and the computational burden.

[0121] S206: Predict the wake-up word for the audio stream segment according to the speech recognition model to obtain a confidence matching result.

[0122] In this embodiment, the confidence matching result includes the wake-up word probability distribution or the wake-up word confidence score.

[0123] In this embodiment, the method can use the trained speech recognition model to predict the wake-up word for the sliding window, and the output is usually a probability distribution or a confidence score, indicating whether the target wake-up word exists in the sliding window. When the confidence score exceeds a specified threshold, it is determined that the wake-up word is detected.

[0124] S207: When the real-time audio is determined to be valid based on the comparison results and the matching results, the detected wake-up word is determined based on the confidence matching result through the speech recognition model.

[0125] S208: Merge and filter the wake-up words to obtain the text meaning of the wake-up words.

[0126] In this embodiment, the method can merge and filter the detected wake-up words.

[0127] For example: merge the same wake-up word detected in multiple sliding windows into a single event to reduce false positives and false alarms.

[0128] S209: When the meaning of the wake-up word text matches the preset emergency wake-up word, obtain the preset emergency preset number, and perform the operation of cyclically dialing the emergency preset number.

[0129] In this embodiment, after the terminal device triggers the wake-up word, the reminder information will be pushed to the mobile terminal immediately. The mobile terminal can view the status of the terminal device and make an audio or video call with the terminal device. At this time, the terminal device can determine whether it is an emergency according to the literal meaning of the wake-up word. When the triggered wake-up word is an emergency, it will automatically dial the preset number for an audio or video call.

[0130] In this embodiment, the method can automatically dial the next preset number when the first preset number cannot be answered, and dial three times in a cycle until a preset number is answered. Four preset numbers can be set.

[0131] Please see Figure 3 , Figure 3 An example flowchart of a wake-up word matching and recognition is shown.

[0132] It can be seen that the method of starting based on the wake-up word described in this embodiment can automatically identify user needs based on voice recognition, thereby improving the timeliness and reliability of assistance in various life scenarios.

[0133] Example 3

[0134] Please see Figure 4 and Figure 5 , Figure 4 and Figure 5 Two parts of a flowchart of a method for starting based on a wake-up word provided in this embodiment. The method for starting based on a wake-up word is applied to a terminal device, including:

[0135] S301: Obtain a preset wake-up word and a textual meaning of the preset wake-up word.

[0136] In this embodiment, the preset wake-up words are set based on different usage scenarios;

[0137] The text meaning of the preset wake-up word is set through the cloud or mobile terminal, and the text meaning of the preset wake-up word is stored in the server database.

[0138] In this embodiment, the method can set the text meaning of the wake-up word according to different usage scenarios. The text meaning of the wake-up word can be set in the cloud or on a mobile terminal (APP / mini program), and the setting result is saved to the server database.

[0139] S302: Obtain speech recognition requirements.

[0140] In this embodiment, the voice recognition requirements include at least one wake-up word, the installation environment of the terminal device, and the voice speed corresponding to the preset wake-up word; the voice speed includes fast speed, moderate speed and slow speed.

[0141] For example, the voice recognition requirement is that the same wake-up word needs to be recorded three times according to three speech speeds (fast, medium, and slow). At the same time, the audio needs to be recorded in a simulated manner according to the environment in which the terminal device is installed. It cannot be recorded in a very quiet place; that is, it needs to be recorded in the actual environment to take into account the environmental noise.

[0142] In this embodiment, the recorded audio needs to be recorded according to the requirements of AI voice recognition. Multiple wake-up words can be recorded. One wake-up word can record multiple audio samples for AI algorithm to compare voiceprints. The audio language recorded in the same usage environment needs to be the same. Dialects can be recorded, that is, the audio dialects recorded by multiple devices and multiple wake-up words in the same place must be consistent. Among them, the audio can be recorded through the cloud or mobile terminal (applet / APP) or terminal device.

[0143] S303: Obtain audio recording data corresponding to a preset wake-up word according to speech recognition requirements.

[0144] S304: Play back the audio recording data locally to obtain a playback confirmation result.

[0145] In this embodiment, after the cloud or mobile terminal (APP / mini program) or terminal device completes recording, audio playback is required to confirm the accuracy of the recorded audio.

[0146] In this embodiment, the playback forms include local playback confirmation and server playback. Among them, local playback means that the playback confirmation is performed immediately after recording, and then uploaded to the server database after playback confirmation; server playback means that the audio data after saving the database is obtained in the cloud through the getOpenharmony() interface for playback confirmation.

[0147] In this embodiment, the playback confirmation result may be input after manual confirmation, and may not involve an algorithm, that is, the user confirms whether the audio he / she said conforms to the literal meaning of the wake-up word.

[0148] S305: Determine whether the audio recording data is qualified according to the playback confirmation result. If so, execute step S306; if not, end this process.

[0149] In this embodiment, the method can confirm the validity (i.e., whether it is qualified) of the uploaded wake-up word audio (i.e., audio recording data) through AI audio processing technology after a series of confirmation actions such as local playback confirmation or cloud-based playback confirmation of uploaded audio files. The valid audio is stored in the voiceprint comparison library of the wake-up word as a valid sample.

[0150] S306: Store the audio recording data into a server database through a preset first interface, and establish an association relationship between the audio recording data and the meaning of the wake-up word text.

[0151] S307: Acquire audio recording data through a preset second interface.

[0152] S308. When the audio recording data is confirmed by playback, the audio recording data, the preset wake-up word, and the textual meaning of the wake-up word are stored in a preset audio comparison library.

[0153] In this embodiment, the cloud, mobile terminal (APP / applet), and terminal device can record audio according to the literal meaning of the wake-up word to collect a large amount of recording data from different people, covering different speaking speeds and sound characteristics. Among them, the speaking language requirements in the same usage environment are consistent, and the recording must be recorded in a real environment.

[0154] In this embodiment, the method can confirm whether the audio recording data is qualified after the wake-up word text meaning is set. If the audio recording data is qualified, the audio recording data is stored in the server database at a position corresponding to the wake-up word text meaning through the setOpenharmony() interface.

[0155] In this embodiment, the recorded audio data is transmitted to the server, and the server uses the ffmpeg tool to convert the audio format and adjust the sampling rate, and the recording files are classified and stored according to the environment.

[0156] S309: Obtain valid audio samples from the audio comparison library.

[0157] S310: Perform cropping processing on the valid audio sample to obtain a first processed sample.

[0158] In this embodiment, the method can process the collected recording data and use a cropping algorithm to process the audio file to a predetermined size.

[0159] S311 . Perform short-time Fourier transform on the first processed sample by spectral subtraction to obtain a signal spectrum diagram.

[0160] In this embodiment, the method can obtain a spectrum diagram by performing short-time Fourier transform through spectral subtraction (Spectral Subtracion).

[0161] S312: Calculate the sample signal energy of the first processed sample according to the signal spectrum diagram.

[0162] S313: Perform noise estimation on the first processed sample to obtain noise signal energy.

[0163] S314 , performing energy subtraction calculation on the first processed sample according to the sample signal energy and the noise signal energy to obtain a noise reduction signal.

[0164] In this embodiment, the method can further perform noise estimation, and then use spectral subtraction - over-subtraction method to subtract excess noise.

[0165] S315: Perform inverse short-time Fourier transform on the noise reduction signal to obtain a second processed sample.

[0166] In this embodiment, the method can also obtain clean and enhanced signal speech through short-time inverse Fourier transform.

[0167] S316. Calculate the maximum noise residual of the second processed sample based on the smoothing mechanism.

[0168] In this embodiment, the method introduces a smoothing mechanism to calculate the maximum noise residual.

[0169] S317: When the amplitude spectrum value of the second processed sample is less than the maximum noise residual, the amplitude spectrum value of the second processed sample is replaced to obtain a replacement signal.

[0170] In this embodiment, the method can determine whether the amplitude spectrum value of the signal is smaller than the maximum noise residual based on the obtained enhanced signal, and if so, replace it with the minimum value in the left and right K frames.

[0171] S318: Perform signal recovery processing on the replacement signal to obtain a recovery signal.

[0172] In this embodiment, after completing the above replacement, the method restores the signal and then draws a speech spectrum diagram to obtain smooth speech with good denoising effect.

[0173] In this embodiment, when the generated speech fails the algorithm detection, the method automatically discards the audio file.

[0174] S319: Draw a speech spectrogram according to the replacement signal to obtain a third processed sample.

[0175] S320: Perform analog-to-digital conversion on the third processed sample to obtain a preprocessed audio sample.

[0176] In this embodiment, the method can perform data preprocessing on the denoised speech and convert the speech into a digital signal through an analog-to-digital converter for use in AI algorithm model training.

[0177] Please see Figure 6 , Figure 6 A schematic diagram of an example process of implementing speech noise reduction using spectral subtraction is shown.

[0178] For example, this method can first calculate the spectrum of the recorded speech, then calculate the energy of the speech signal, then estimate the energy of the noise signal, and then use direct energy subtraction to ensure that the remaining energy after subtracting the noise energy is greater than 0, then restore the speech signal after subtracting the noise energy, and then plot the signal into a speech spectrogram to obtain the denoised speech. The denoised speech can be calculated based on the following formula:

[0179]

[0180] At this time, the smoothing mechanism is introduced, based on the formula Calculating the maximum noise residual, we can get:

[0181]

[0182] Where E(ω) is the estimated noise (E t On this basis, the part that is too small is replaced by the minimum value of adjacent frames.

[0183] After the above two formulas are processed, smooth and noise-reduced speech is obtained, and the pre-processed speech can be used to train the speech recognition model.

[0184] S321. Construct a sample data set according to the preprocessed audio samples.

[0185] In this embodiment, the sample data set includes a training set, a test set, and a validation set.

[0186] In this embodiment, the method divides the prepared speech data set (ie, sample data set) into a training set, a test set, and a validation set.

[0187] S322. Obtain a pre-built deep convolutional neural network.

[0188] In this embodiment, the training algorithm of the deep convolutional neural network includes the stochastic gradient descent method, the Dropout algorithm and the adaptive learning rate algorithm;

[0189] The adaptive learning rate algorithm is used to adjust the learning rate of each model parameter in the deep convolutional neural network;

[0190] The Dropout algorithm is used to prevent overfitting of deep convolutional neural networks.

[0191] In this embodiment, the deep convolutional neural network (DSCNN) can extract speech features and then learn the time domain features of speech signals. Among them, the deep convolutional neural network (DSCNN) training adopts the stochastic gradient descent method, which calculates the gradient through back propagation, and then updates the model parameters according to the gradient. After obtaining the parameters, the adaptive learning rate algorithm is used to automatically adjust the learning rate of each model parameter, and finally the Dropout algorithm is used to prevent the problem of model overfitting, thereby realizing continuous training of speech data and achieving high-accuracy speech recognition.

[0192] S323. The deep convolutional neural network is trained using a sample data set and a preset back-propagation algorithm to obtain a confusion matrix generated after the training and a trained target model.

[0193] In this embodiment, the method can convert the speech spectrum into a feature vector, and then use large-scale data for model learning, and continuously update the parameters through the back propagation algorithm to improve the accuracy of speech recognition. At the same time, a deep convolutional neural network (DSCNN) model is used for iterative training, and the loss curve is visualized to monitor the training effect and trend. After the training is completed, a confusion matrix can also be generated to intuitively display the model performance.

[0194] S324. When it is determined according to the confusion matrix that the model performance of the speech recognition model is qualified, the target model is formatted according to a preset model format to obtain a speech recognition model.

[0195] In this embodiment, the method can save and convert the model into a required format and deploy it to an embedded lightweight device or a cloud environment.

[0196] In this embodiment, the speech recognition model is used to match the acoustic features with the wake-up words in the training set, and then to match and recognize the speech (voiceprint). Specifically, when the speech recognition model is used, it first recognizes the audio voiceprint, and then recognizes the specific wake-up word.

[0197] S325, collect real-time audio.

[0198] S326: extract audio features of real-time audio.

[0199] S327: According to the pre-built speech recognition model, the audio features are compared with a preset audio comparison library to obtain a comparison result.

[0200] S328. Perform wake-up word confidence matching on the audio features according to the speech recognition model to obtain a confidence matching result.

[0201] S329: When the real-time audio is determined to be valid based on the comparison results and the matching results, the text meaning of the wake-up word corresponding to the real-time audio is identified based on the speech recognition model.

[0202] S330, sending alarm information including the text meaning of the wake-up word to the cloud and the mobile terminal respectively, so that the cloud and the mobile terminal respectively perform corresponding alarm processing according to the alarm information.

[0203] Please see Figure 7 , Figure 7 An example flowchart of a wake-up word based activation is shown.

[0204] It can be seen that the method of starting based on the wake-up word described in this embodiment can automatically identify user needs based on voice recognition, thereby improving the timeliness and reliability of assistance in various life scenarios.

[0205] Example 4

[0206] Please see Figure 8 , Figure 8 This is a schematic diagram of a structure of a device based on wake-up word activation provided in this embodiment. Figure 8 As shown, the device activated based on the wake-up word is a terminal device, including:

[0207] A collection unit 410 is used to collect real-time audio;

[0208] A feature extraction unit 420, configured to extract audio features of real-time audio;

[0209] The comparison unit 430 is used to perform an audio comparison between the audio features and a preset audio comparison library according to a pre-built speech recognition model to obtain a comparison result;

[0210] A matching unit 440 is used to perform a wake-up word confidence match on the audio feature according to the speech recognition model to obtain a confidence match result;

[0211] The meaning recognition unit 450 is used to recognize the meaning of the wake-up word text corresponding to the real-time audio according to the speech recognition model when the real-time audio is determined to be valid according to the comparison result and the matching result;

[0212] The sending unit 460 is used to send alarm information including the text meaning of the wake-up word to the cloud and the mobile terminal respectively, so that the cloud and the mobile terminal perform corresponding alarm processing according to the alarm information.

[0213] In this embodiment, the explanation of the device activated based on the wake-up word can refer to the description in Example 1, Example 2 or Example 3, which will not be repeated in this embodiment.

[0214] It can be seen that the device based on wake-up word activation described in this embodiment can automatically identify user needs based on voice recognition, thereby improving the timeliness and reliability of assistance in various life scenarios.

[0215] Example 5

[0216] Please see Fig. 9 , Fig. 9 This is a schematic diagram of a structure of a device based on wake-up word activation provided in this embodiment. Fig. 9 As shown, the device activated based on the wake-up word is a terminal device, including:

[0217] A collection unit 410 is used to collect real-time audio;

[0218] A feature extraction unit 420, configured to extract audio features of real-time audio;

[0219] The comparison unit 430 is used to perform an audio comparison between the audio features and a preset audio comparison library according to a pre-built speech recognition model to obtain a comparison result;

[0220] A matching unit 440 is used to perform a wake-up word confidence match on the audio feature according to the speech recognition model to obtain a confidence match result;

[0221] The meaning recognition unit 450 is used to recognize the meaning of the wake-up word text corresponding to the real-time audio according to the speech recognition model when the real-time audio is determined to be valid according to the comparison result and the matching result;

[0222] The sending unit 460 is used to send alarm information including the text meaning of the wake-up word to the cloud and the mobile terminal respectively, so that the cloud and the mobile terminal perform corresponding alarm processing according to the alarm information.

[0223] As an optional implementation, the device activated based on the wake-up word further includes:

[0224] The dialing unit 470 is used to obtain a preset emergency preset number and perform an operation of cyclically dialing the emergency preset number after the meaning recognition unit 450 recognizes the text meaning of the wake-up word corresponding to the real-time audio according to the speech recognition model, and when the text meaning of the wake-up word matches the preset emergency wake-up word.

[0225] As an optional implementation, the device activated based on the wake-up word further includes:

[0226] The first acquisition unit 480 is used to acquire a preset wake-up word and a textual meaning of the preset wake-up word before the acquisition unit 410 acquires the real-time audio;

[0227] The first acquisition unit 480 is further used to acquire audio recording data corresponding to the preset wake-up word;

[0228] An establishing unit 490 is used to store the audio recording data in a server database through a preset first interface when the audio recording data passes the qualified confirmation, and establish an association relationship between the audio recording data and the meaning of the wake-up word text;

[0229] The second acquisition unit 500 is used to acquire audio recording data through a preset second interface;

[0230] The storage unit 510 is used to store the audio recording data and the preset wake-up word in a preset audio comparison library when the audio recording data passes the playback confirmation.

[0231] As an optional implementation, the device activated based on the wake-up word further includes:

[0232] The playback unit 520 is configured to locally play back the audio recording data after the first acquisition unit 480 acquires the audio recording data corresponding to the preset wake-up word to obtain a playback confirmation result;

[0233] A judging unit 530, configured to judge whether the audio recording data is qualified according to the playback confirmation result;

[0234] The establishing unit 490 is used to store the audio recording data into the server database through a preset first interface when the determination result of the determining unit 530 is yes.

[0235] As an optional implementation, the first acquisition unit 480 is specifically used to acquire a speech recognition requirement; wherein the speech recognition requirement includes at least one wake-up word, an installation environment of the terminal device, and a speech speed corresponding to a preset wake-up word; the speech speed includes a fast speech speed, a moderate speech speed, and a slow speech speed;

[0236] The first acquisition unit 480 is further specifically configured to acquire audio recording data corresponding to a preset wake-up word according to a speech recognition requirement.

[0237] In this embodiment, the preset wake-up words are set based on different usage scenarios;

[0238] The text meaning of the preset wake-up word is set through the cloud or mobile terminal, and the text meaning of the preset wake-up word is stored in the server database.

[0239] As an optional implementation, the feature extraction unit 420 includes:

[0240] The first denoising subunit 421 is used to perform denoising on the real-time audio to obtain a denoised audio signal;

[0241] The first conversion subunit 422 is used to perform frequency spectrum feature graph conversion on the noise-removed audio signal to obtain audio features.

[0242] As an optional implementation, the matching unit 440 includes:

[0243] A division subunit 441 is used to divide the audio features according to a preset sliding window to obtain multiple audio stream segments;

[0244] The prediction subunit 442 is used to predict the wake-up word of the audio stream segment according to the speech recognition model to obtain a confidence matching result; wherein the confidence matching result includes the wake-up word probability distribution or the wake-up word confidence score.

[0245] As an optional implementation, the meaning recognition unit 450 includes:

[0246] A determination subunit 451 is used to determine the detected wake-up word according to the confidence matching result through the speech recognition model;

[0247] The filtering subunit 452 is used to merge and filter the wake-up words to obtain the text meaning of the wake-up words.

[0248] As an optional implementation, the device activated based on the wake-up word further includes:

[0249] A third acquisition unit 540 is used to acquire a valid audio sample in the audio comparison library after the storage unit 510 stores the audio recording data, the preset wake-up word and the text meaning of the wake-up word in the preset audio comparison library;

[0250] A preprocessing unit 550, configured to preprocess the valid audio sample to obtain a preprocessed audio sample;

[0251] A construction unit 560 is used to construct a sample data set according to the preprocessed audio sample; wherein the sample data set includes a training set, a test set and a validation set;

[0252] A fourth acquisition unit 570, configured to acquire a pre-built deep convolutional neural network;

[0253] A training unit 580 is used to train the deep convolutional neural network using a sample data set and a preset back propagation algorithm to obtain a confusion matrix generated after the training and a trained target model;

[0254] The conversion unit 590 is used to convert the target model into a format according to a preset model format to obtain a speech recognition model when it is determined that the model performance of the speech recognition model is qualified according to the confusion matrix.

[0255] In this embodiment, the training algorithm of the deep convolutional neural network includes the stochastic gradient descent method, the Dropout algorithm and the adaptive learning rate algorithm;

[0256] The adaptive learning rate algorithm is used to adjust the learning rate of each model parameter in the deep convolutional neural network;

[0257] The Dropout algorithm is used to prevent overfitting of deep convolutional neural networks.

[0258] As an optional implementation, the preprocessing unit 550 includes:

[0259] The cropping subunit 551 is used to crop the valid audio sample to obtain a first processed sample;

[0260] A second denoising subunit 552, configured to perform denoising on the first processed sample by spectral subtraction to obtain a second processed sample;

[0261] A smoothing subunit 553 is used to perform smoothing on the second processed sample to obtain a third processed sample;

[0262] The second conversion subunit 554 is used to perform analog-to-digital conversion on the third processed sample to obtain a preprocessed audio sample.

[0263] As an optional implementation, the second denoising subunit 552 includes:

[0264] A transform module, used for performing short-time Fourier transform on the first processed sample by spectral subtraction to obtain a signal spectrum diagram;

[0265] A first calculation module, used for calculating the sample signal energy of the first processed sample according to the signal spectrum diagram;

[0266] An estimation module, used for performing noise estimation on the first processed sample to obtain noise signal energy;

[0267] The first calculation module is further used to perform energy subtraction calculation on the first processed sample according to the sample signal energy and the noise signal energy to obtain a noise reduction signal;

[0268] The transform module is also used to perform short-time inverse Fourier transform on the noise reduction signal to obtain a second processed sample.

[0269] As an optional implementation, the smoothing subunit 553 includes:

[0270] A second calculation module, used for calculating the maximum noise residual of the second processed sample based on a smoothing mechanism;

[0271] A replacement module, used for replacing the amplitude spectrum value of the second processed sample to obtain a replacement signal when the amplitude spectrum value of the second processed sample is less than the maximum noise residual;

[0272] A recovery module, used for performing signal recovery processing on the replacement signal to obtain a recovery signal;

[0273] The drawing module is used to draw a speech spectrum diagram according to the replacement signal to obtain a third processed sample.

[0274] In this embodiment, the explanation of the device activated based on the wake-up word can refer to the description in Example 1, Example 2 or Example 3, which will not be repeated in this embodiment.

[0275] It can be seen that the device based on wake-up word activation described in this embodiment can automatically identify user needs based on voice recognition, thereby improving the timeliness and reliability of assistance in various life scenarios.

[0276] Example 6

[0277] Please see Fig.10 , Fig.10 The present embodiment provides a schematic diagram of the structure of a system based on wake-up word activation. Fig.10 As shown, the system based on the wake-up word activation includes a terminal device, a cloud and a mobile terminal, wherein:

[0278] The terminal device collects real-time audio;

[0279] The terminal device extracts audio features of the real-time audio;

[0280] The terminal device performs an audio comparison between the audio features and a preset audio comparison library according to a pre-built speech recognition model to obtain a comparison result;

[0281] The terminal device performs a wake-up word confidence match on the audio features according to the speech recognition model to obtain a confidence match result;

[0282] When the terminal device determines that the real-time audio is valid according to the comparison result and the matching result, the terminal device identifies the text meaning of the wake-up word corresponding to the real-time audio according to the speech recognition model;

[0283] The terminal device sends an alarm message including the text meaning of the wake-up word to the cloud and the mobile terminal respectively;

[0284] After receiving the alarm information, the cloud obtains the location information of the terminal device and the video surveillance information corresponding to the location information according to the alarm information;

[0285] Output alarm information, location information and video surveillance information to the cloud;

[0286] After receiving the alarm information, the mobile terminal outputs the alarm information and performs a corresponding audio and video call operation on the terminal device according to the audio and video call instruction input by the user in response to the alarm information.

[0287] In this embodiment, the explanation of the system activated based on the wake-up word can refer to the description in Example 1, Example 2 or Example 3, which will not be further elaborated in this embodiment.

[0288] It can be seen that the system based on wake-up word activation described in this embodiment can automatically identify user needs based on voice recognition, thereby improving the timeliness and reliability of assistance in various life scenarios.

[0289] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0290] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0291] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A method for starting based on a wake-up word, the method is applied to a terminal device, characterized in that: include: Collect real-time audio; Extracting audio features of the real-time audio; According to the pre-built speech recognition model, the audio feature is compared with a preset audio comparison library to obtain a comparison result; According to the speech recognition model, performing a wake-up word confidence match on the audio feature to obtain a confidence match result; When it is determined that the real-time audio is valid according to the comparison result and the matching result, identifying the text meaning of the wake-up word corresponding to the real-time audio according to the speech recognition model; An alarm message including the text meaning of the wake-up word is sent to the cloud and the mobile terminal respectively, so that the cloud and the mobile terminal respectively perform corresponding alarm processing according to the alarm message.

2. The method for starting based on a wake-up word according to claim 1, characterized in that: After identifying the text meaning of the wake-up word corresponding to the real-time audio according to the speech recognition model, the method further includes: When the meaning of the wake-up word text matches the preset emergency wake-up word, a preset emergency preset number is obtained, and an operation of cyclically dialing the emergency preset number is performed; Before collecting the real-time audio, the method further includes: Obtaining a preset wake-up word and the wake-up word text meaning of the preset wake-up word; Obtaining audio recording data corresponding to the preset wake-up word; When the audio recording data passes the qualified confirmation, the audio recording data is stored in the server database through the preset first interface, and an association relationship between the audio recording data and the meaning of the wake-up word text is established; Acquire the audio recording data through a preset second interface; When the audio recording data is confirmed by playback, the audio recording data, the preset wake-up word and the textual meaning of the wake-up word are stored in a preset audio comparison library; Wherein, after obtaining the audio recording data corresponding to the preset wake-up word, the method further includes: Playing back the audio recording data locally to obtain a playback confirmation result; Determining whether the audio recording data is qualified according to the playback confirmation result; If yes, the step of storing the audio recording data in a server database through a preset first interface is performed.

3. The method for starting based on a wake-up word according to claim 2, characterized in that: The obtaining of the audio recording data corresponding to the preset wake-up word includes: Acquire speech recognition requirements; wherein the speech recognition requirements include at least one wake-up word, the installation environment of the terminal device, and the speech speed corresponding to the preset wake-up word; the speech speed includes fast speed, moderate speed, and slow speed; Acquire the audio recording data corresponding to the preset wake-up word according to the voice recognition requirement; Wherein, the preset wake-up word is set based on different usage scenarios; The text meaning of the preset wake-up word is set through the cloud or mobile terminal, and the text meaning of the preset wake-up word is stored in the server database.

4. The method for starting based on a wake-up word according to claim 1, characterized in that: The extracting audio features of the real-time audio includes: Performing denoising on the real-time audio to obtain a denoised audio signal; Performing frequency spectrum feature map conversion on the de-noised audio signal to obtain audio features; The step of performing wake-up word confidence matching on the audio feature according to the speech recognition model to obtain a confidence matching result includes: The audio features are divided and processed according to a preset sliding window to obtain multiple audio stream segments; Predicting the wake-up word for the audio stream segment according to the speech recognition model to obtain a confidence matching result; wherein the confidence matching result includes a wake-up word probability distribution or a wake-up word confidence score; The step of identifying the text meaning of the wake-up word corresponding to the real-time audio according to the speech recognition model includes: Determining the detected wake-up word according to the confidence matching result through the speech recognition model; The wake-up words are merged and filtered to obtain the text meaning of the wake-up words.

5. The method for starting based on a wake-up word according to claim 2, characterized in that: After storing the audio recording data, the preset wake-up word, and the textual meaning of the wake-up word in a preset audio comparison library, the method further includes: Acquire valid audio samples from the audio comparison library; Preprocessing the valid audio sample to obtain a preprocessed audio sample; Constructing a sample data set according to the preprocessed audio samples; wherein the sample data set includes a training set, a test set and a validation set; Get pre-built deep convolutional neural networks; The deep convolutional neural network is trained using the sample data set and a preset back propagation algorithm to obtain a confusion matrix generated after the training and a trained target model; When it is determined according to the confusion matrix that the model performance of the speech recognition model is qualified, the target model is format-converted according to a preset model format to obtain a speech recognition model.

6. The method for starting based on a wake-up word according to claim 5, characterized in that: The training algorithms of the deep convolutional neural network include stochastic gradient descent method, Dropout algorithm and adaptive learning rate algorithm; The adaptive learning rate algorithm is used to adjust the learning rate of each model parameter in the deep convolutional neural network; The Dropout algorithm is used to prevent the deep convolutional neural network from overfitting.

7. The method for starting based on a wake-up word according to claim 5, characterized in that: The preprocessing of the valid audio sample to obtain the preprocessed audio sample includes: Performing a cropping process on the valid audio sample to obtain a first processed sample; Performing denoising on the first processed sample by spectral subtraction to obtain a second processed sample; performing smoothing processing on the second processed sample to obtain a third processed sample; Performing analog-to-digital conversion on the third processed sample to obtain a preprocessed audio sample; The step of performing denoising on the first processed sample by spectral subtraction to obtain a second processed sample includes: Performing short-time Fourier transform on the first processed sample by spectral subtraction to obtain a signal spectrum diagram; Calculating the sample signal energy of the first processed sample according to the signal spectrum diagram; performing noise estimation on the first processed sample to obtain noise signal energy; Performing energy subtraction calculation on the first processed sample according to the sample signal energy and the noise signal energy to obtain a noise reduction signal; Performing an inverse short-time Fourier transform on the noise reduction signal to obtain a second processed sample; The step of performing smoothing on the second processed sample to obtain a third processed sample includes: calculating a maximum noise residual of the second processed sample based on a smoothing mechanism; When the amplitude spectrum value of the second processed sample is less than the maximum noise residual, the amplitude spectrum value of the second processed sample is replaced to obtain a replacement signal; Performing signal recovery processing on the replacement signal to obtain a recovery signal; A speech spectrogram is plotted according to the replacement signal to obtain a third processed sample.

8. A device based on a wake-up word startup, the device based on a wake-up word startup is a terminal device, characterized in that: include: A collection unit, used for collecting real-time audio; A feature extraction unit, used to extract audio features of the real-time audio; A comparison unit, used to perform an audio comparison between the audio feature and a preset audio comparison library according to a pre-built speech recognition model to obtain a comparison result; A matching unit, configured to perform a wake-up word confidence match on the audio feature according to the speech recognition model to obtain a confidence matching result; a meaning recognition unit, configured to recognize the meaning of the wake-up word text corresponding to the real-time audio according to the speech recognition model when it is determined that the real-time audio is valid according to the comparison result and the matching result; The sending unit is used to send alarm information including the text meaning of the wake-up word to the cloud and the mobile terminal respectively, so that the cloud and the mobile terminal respectively perform corresponding alarm processing according to the alarm information.

9. A system based on wake-up word activation, characterized in that: The system based on wake-up word activation includes a terminal device, a cloud terminal and a mobile terminal, wherein: The terminal device collects real-time audio; The terminal device extracts audio features of the real-time audio; The terminal device performs an audio comparison between the audio feature and a preset audio comparison library according to a pre-built speech recognition model to obtain a comparison result; The terminal device performs a wake-up word confidence match on the audio feature according to the speech recognition model to obtain a confidence match result; When the terminal device determines that the real-time audio is valid according to the comparison result and the matching result, identifying the text meaning of the wake-up word corresponding to the real-time audio according to the speech recognition model; The terminal device sends an alarm message including the text meaning of the wake-up word to the cloud and the mobile terminal respectively; After receiving the alarm information, the cloud obtains the location information of the terminal device and the video monitoring information corresponding to the location information according to the alarm information; The cloud outputs the alarm information, the location information, and the video surveillance information; After receiving the alarm information, the mobile terminal outputs the alarm information and performs a corresponding audio or video call operation on the terminal device according to the audio or video call instruction input by the user in response to the alarm information.

Images