Wake-up method, device, electronic device and storage medium

Abstract

The present application relates to a wake-up method, device, electronic device and storage medium, which are applied to the field of speech recognition technology. The wake-up method includes: obtaining the speech frame corresponding to the wake-up speech in real time; if the first wake-up score corresponding to the 1st to f1st speech frames in the first time period is less than the first score threshold, it is determined that the smart device cannot be woken up; if the first wake-up score is greater than the second score threshold, the 1st to f1st speech frames are determined as the first target sequence; after a preset time period, when the f2nd to f3rd speech frames in the second time period are obtained, if the second wake-up score corresponding to the f2nd to f3rd speech frames is less than the first score threshold, it is determined that the smart device cannot be woken up; if the second wake-up score is greater than the second score threshold, the f2nd to f3rd speech frames are determined as the second target sequence; the first target sequence and the second target sequence are input into the speech judgment model to determine whether to wake up the smart device. The present application can improve the wake-up rate of secondary wake-up.

Description

Technical Field

[0001] The present application relates to the field of speech recognition technology, and in particular to a wake-up method, device, electronic device and storage medium. Background Art

[0002] With the development of smart devices, voice interaction has achieved unprecedented growth. Wake-up is the most important part of it and is the portal for users to interact with smart devices. In the presence of background noise, if the user fails to wake up the smart device when shouting the wake-up word for the first time, the user can shout the wake-up word again to wake up the smart device for the second time. However, in the existing wake-up algorithms, the wake-up rate of the second wake-up is low. Summary of the invention

[0003] In order to solve the above technical problems, the present application provides a wake-up method, device, electronic device and storage medium.

[0004] According to a first aspect of the present application, a wake-up method is provided, comprising:

[0005] Acquire the voice frame corresponding to the wake-up voice in real time;

[0006] When the 1st to f1th voice frames in the first time period are obtained, if the first wake-up scores corresponding to the 1st to f1th voice frames are less than the first score threshold, it is determined that the smart device cannot be woken up;

[0007] If the first wake-up score is greater than a second score threshold, determining the 1st to f1th speech frames as a first target sequence; wherein the second score threshold is less than the first score threshold;

[0008] After the preset time period, when the f2-th to f3-th voice frames in the second time period are obtained, if the second wake-up scores corresponding to the f2-th to f3-th voice frames are less than the first score threshold, it is determined that the smart device cannot be woken up;

[0009] If the second wake-up score is greater than the second score threshold, determine the f2th to f3th speech frames as the second target sequence; wherein f1, f2, and f3 are positive integers greater than 1 and increase in sequence;

[0010] The first target sequence and the second target sequence are input into a pre-trained speech judgment model to determine whether to wake up the smart device.

[0011] Optionally, the training method of the speech judgment model includes:

[0012] Acquire a target sample voice whose sample type includes a positive sample and a negative sample, wherein the target sample voice whose sample type is a positive sample includes a wake-up word voice, and the target sample voice whose sample type is a negative sample does not include a wake-up word voice;

[0013] Arrange and combine the target sample voices to obtain a sample voice pair including two target sample voices; wherein the sample voice pair is input data;

[0014] Determining label data corresponding to the sample speech pair according to sample types of two target sample speech pairs included in the sample speech pair;

[0015] The speech judgment model is trained and generated based on the sample speech pairs and the label data corresponding to the sample speech pairs.

[0016] Optionally, determining the label data corresponding to the sample speech pair according to the sample types of the two target sample speech pairs included in the sample speech pair includes:

[0017] If the sample types of the two target sample voices included in the sample voice pair are both positive samples, determining that the label data corresponding to the sample voice pair is a first value;

[0018] Otherwise, determine that the label data corresponding to the sample speech pair is a second value, wherein the second value is smaller than the first value.

[0019] Optionally, inputting the first target sequence and the second target sequence into a pre-trained speech judgment model to determine whether to wake up the smart device includes:

[0020] Inputting the first target sequence and the second target sequence into a pre-trained speech judgment model to obtain a third value;

[0021] If the third value is greater than or equal to a preset value, waking up the smart device; wherein the preset value is determined based on the first value and the second value;

[0022] If the third value is smaller than the preset value, the smart device is not woken up.

[0023] Optionally, the method further includes:

[0024] Selecting a first speech frame sequence and a second speech frame sequence having a preset length from the first target sequence and the second target sequence respectively;

[0025] The step of inputting the first target sequence and the second target sequence into a pre-trained speech judgment model to determine whether to wake up the smart device includes:

[0026] The first speech frame sequence and the second speech frame sequence are input into a pre-trained speech judgment model to determine whether to wake up the smart device.

[0027] Optionally, the method further includes:

[0028] Inputting the 1st to f1th speech frames into a pre-trained wake-up model to obtain the first wake-up score, where the first wake-up score is used to indicate the probability that the text information corresponding to the 1st to f1th speech frames belongs to the wake-up word;

[0029] The f2-th to f3-th speech frames are input into the wake-up model to obtain the second wake-up score, where the second wake-up score is used to indicate the probability that the text information corresponding to the f2-th to f3-th speech frames belongs to the wake-up word.

[0030] Optionally, the method further includes:

[0031] Obtain wake-up scores corresponding to multiple voices that cannot wake up the smart device under multiple signal-to-noise ratios;

[0032] The second score threshold is determined according to the wake-up scores respectively corresponding to the multiple voices.

[0033] According to a second aspect of the present application, a wake-up device is provided, including:

[0034] A voice frame real-time acquisition module is used to acquire the voice frame corresponding to the wake-up voice in real time;

[0035] A first target sequence determination module is used to, when acquiring the 1st to f1th voice frames within a first time period, determine that the smart device cannot be woken up if the first wake-up scores corresponding to the 1st to f1th voice frames are less than a first score threshold; if the first wake-up scores are greater than a second score threshold, determine the 1st to f1th voice frames as a first target sequence; wherein the second score threshold is less than the first score threshold;

[0036] A second target sequence determination module is used for, after a preset time period, when the f2-f3th voice frames in the second time period are obtained, if the second wake-up scores corresponding to the f2-f3th voice frames are less than the first score threshold, determining that the smart device cannot be woken up; if the second wake-up scores are greater than the second score threshold, determining the f2-f3th voice frames as the second target sequence; wherein f1, f2, and f3 are positive integers greater than 1 and increase in sequence;

[0037] The wake-up judgment module is used to input the first target sequence and the second target sequence into a pre-trained speech judgment model to determine whether to wake up the smart device.

[0038] Optionally, the awakening device further includes:

[0039] A target sample speech acquisition module, used to acquire a target sample speech whose sample type includes a positive sample and a negative sample, wherein the target sample speech whose sample type is a positive sample includes a wake-up word speech, and the target sample speech whose sample type is a negative sample does not include a wake-up word speech;

[0040] An input data construction module, used for arranging and combining the target sample voices to obtain a sample voice pair including two target sample voices; wherein the sample voice pair is input data;

[0041] A label data construction module, used to determine the label data corresponding to the sample speech pair according to the sample types of the two target sample speech included in the sample speech pair;

[0042] The speech judgment model training module is used to train and generate the speech judgment model based on the sample speech pairs and the label data corresponding to the sample speech pairs.

[0043] Optionally, the label data construction module is specifically used to determine that the label data corresponding to the sample speech pair is a first value if the sample types of the two target sample speech pairs contained in the sample speech pair are both positive samples; otherwise, determine that the label data corresponding to the sample speech pair is a second value, wherein the second value is smaller than the first value.

[0044] Optionally, the wake-up judgment module is specifically used to input the first target sequence and the second target sequence into a pre-trained speech judgment model to obtain a third value; if the third value is greater than or equal to a preset value, the smart device is woken up; wherein the preset value is determined based on the first value and the second value; if the third value is less than the preset value, the smart device is not woken up.

[0045] Optionally, the awakening device further includes:

[0046] A speech frame sequence selection module, used to select a first speech frame sequence and a second speech frame sequence with a preset length from the first target sequence and the second target sequence respectively;

[0047] The wake-up judgment module is specifically used to input the first voice frame sequence and the second voice frame sequence into a pre-trained voice judgment model to determine whether to wake up the smart device.

[0048] Optionally, the awakening device further includes:

[0049] a first wake-up score determination module, configured to input the 1st to f1th speech frames into a pre-trained wake-up model to obtain the first wake-up score, where the first wake-up score is used to indicate the probability that the text information corresponding to the 1st to f1th speech frames belongs to the wake-up word;

[0050] The second wake-up score determination module is used to input the f2-th to f3-th speech frames into the wake-up model to obtain the second wake-up score, where the second wake-up score is used to indicate the probability that the text information corresponding to the f2-th to f3-th speech frames belongs to the wake-up word.

[0051] Optionally, the awakening device further includes:

[0052] A sample wake-up score acquisition module, used to obtain wake-up scores corresponding to multiple voices that cannot wake up the smart device under multiple signal-to-noise ratios;

[0053] The second score threshold determination module is used to determine the second score threshold according to the wake-up scores corresponding to the multiple voices respectively.

[0054] According to a third aspect of the present application, an electronic device is provided, comprising: a processor, wherein the processor is used to execute a computer program stored in a memory, wherein the computer program implements the wake-up method described in the first aspect when executed by the processor.

[0055] According to a fourth aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the wake-up method described in the first aspect is implemented.

[0056] According to a fifth aspect of the present application, a computer program product is provided. When the computer program product is run on a computer, the computer is enabled to execute the wake-up method described in the first aspect.

[0057] Compared with the prior art, the technical solution provided by the embodiments of the present application has the following advantages:

[0058] In the case where the user fails to wake up the smart device for the first time, a second wake-up is usually performed. Since the two wake-ups usually have a certain time interval, the first target sequence and the second target sequence with a certain time interval can be extracted. That is, it can be pre-determined that the user has performed a second wake-up. Furthermore, during the second wake-up, the same wake-up algorithm as the first wake-up is no longer used, that is, the first target sequence is treated as noise, and the second target sequence is processed by the wake-up algorithm. Instead, the correlation between the two wake-ups is considered, and the first target sequence and the second target sequence are input into the pre-trained voice judgment model to determine whether to wake up the smart device. Therefore, the wake-up rate of the second wake-up can be improved, and the user's interactive experience can be enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0060] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0061] Figure 1 A schematic diagram showing a system architecture of an exemplary application environment that can be applied to the wake-up method of an embodiment of the present application;

[0062] Figure 2 A flowchart of the wake-up method in an embodiment of the present application;

[0063] Figure 3 A flow chart of a method for training a speech recognition model in an embodiment of the present application;

[0064] Figure 4 A schematic diagram of a structure of a wake-up device in an embodiment of the present application;

[0065] Figure 5 A schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0066] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the scheme of the present application will be further described below. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0067] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only part of the embodiments of the present application, rather than all of the embodiments.

[0068] Figure 1 Schematic diagram of an application scenario of the wake-up method in the embodiment of the present application. Figure 1As shown, the smart TV has a voice wake-up function, and the user can shout a specific wake-up word to wake up the smart TV. If the smart TV has not been awakened in the process of the user shouting the wake-up word, it can be determined whether the user has performed a second wake-up based on the wake-up voice generated during the wake-up process. If it is determined that the user has performed a second wake-up, it is determined whether to wake up the smart TV based on the first target sequence during the first wake-up and the second target sequence during the second wake-up. The present application can improve the wake-up rate of the second wake-up when the user is not awakened the first time and wakes up the second time.

[0069] See also Figure 2 , Figure 2 A flowchart of the wake-up method in an embodiment of the present application may include the following steps:

[0070] Step S210, obtaining a voice frame corresponding to the wake-up voice in real time.

[0071] Step S220: when the 1st to f1th voice frames in the first time period are obtained, if the first wake-up scores corresponding to the 1st to f1th voice frames are less than the first score threshold, it is determined that the smart device cannot be woken up.

[0072] Step S230: If the first wake-up score is greater than the second score threshold, the 1st to f1th speech frames are determined as the first target sequence.

[0073] Step S240, after the preset time period, when the f2-th to f3-th voice frames in the second time period are obtained, if the second wake-up scores corresponding to the f2-th to f3-th voice frames are less than the first score threshold, it is determined that the smart device cannot be woken up.

[0074] Step S250: If the second wake-up score is greater than the second score threshold, the f2th to f3th speech frames are determined as the second target sequence; wherein f1, f2, and f3 are positive integers greater than 1 and increase in sequence.

[0075] Step S260: input the first target sequence and the second target sequence into a pre-trained speech judgment model to determine whether to wake up the smart device.

[0076] The wake-up method of the embodiment of the present application will usually perform a second wake-up if the user fails to wake up the smart device for the first time. Since the two wake-ups usually have a certain time interval, the first target sequence and the second target sequence with a certain time interval can be extracted. That is, it can be pre-determined that the user has performed a second wake-up. Furthermore, during the second wake-up, the same wake-up algorithm as the first wake-up is no longer used, that is, the first target sequence is treated as noise, and the second target sequence is processed by the wake-up algorithm. Instead, the correlation between the two wake-ups is considered, and the first target sequence and the second target sequence are input into a pre-trained voice judgment model to determine whether to wake up the smart device. Therefore, the wake-up rate of the second wake-up can be improved, and the user's interactive experience can be enhanced.

[0077] See also Figure 3 , Figure 3 A flowchart of a method for training a speech recognition model in an embodiment of the present application may include the following steps:

[0078] Step S310: obtaining target sample speech whose sample types include positive samples and negative samples.

[0079] The speech judgment model is used to determine whether to wake up the smart device when the user performs a second wake-up. Considering the correlation between the two wake-ups, the input of the speech judgment model may include the speech data of the first wake-up and the speech data of the second wake-up. Accordingly, when training the speech judgment model, a target sample speech whose sample type includes positive samples and negative samples can be obtained. Among them, the target sample speech whose sample type is a positive sample is a speech containing a wake-up word, and the target sample speech whose sample type is a negative sample is a speech that does not contain a wake-up word.

[0080] It should be noted that since the process of the user shouting the wake-up word usually corresponds to a time period (for example, 2 seconds), multiple voice frames can be included in this time period. Therefore, when the voice frame reaches the preset length, the smart device can be woken up. In other words, if the length of the voice frame does not reach the preset length, the smart device cannot be woken up. Among them, the value of the preset length can be determined according to the length of the wake-up word. The longer the length of the wake-up word, the larger the value of the preset length will be.

[0081] Optionally, multiple first sample wake-up voices that have been awakened once may be obtained, and the value of the preset length may be determined according to the length of the voice frame corresponding to the first sample wake-up voice. For example, the length of the voice frame corresponding to each first sample wake-up voice may be obtained, and the lengths of the voice frames may be sorted from high to low, and the lengths of the first 10% may be selected and averaged to obtain the value of the preset length.

[0082] Accordingly, when constructing training data, original sample speech containing positive samples and negative samples of sample types under multiple signal-to-noise ratios can be obtained, and speech frames of preset length can be extracted from each original sample speech to obtain a target sample speech containing speech frames of preset length. In this way, by comparing two target sample speech with equal length, the accuracy of correlation judgment can be improved. The length of the speech frame contained in the original sample speech may be less than the preset length, may be equal to the preset length, or may be greater than the preset length. In the case where the length of the speech frame contained in the original sample speech is equal to the preset length, the original sample speech is directly used as the target sample speech. In the case where the length of the speech frame contained in the original sample speech is greater than the preset length, the intermediate continuous speech frames of preset length can be extracted. In the case where the length of the speech frame contained in the original sample speech is less than the preset length, the start frame of the original sample speech can be filled with the same speech frame as the start frame, and / or the end frame of the original sample speech can be filled with the same speech frame as the end frame, so that the total length of the filled speech frame is equal to the preset length. Alternatively, the original sample speech can be directly discarded.

[0083] Step S320: Arrange and combine the target sample voices to obtain a sample voice pair including two target sample voices.

[0084] As mentioned above, the input of the speech judgment model may include the speech data of the first awakening and the speech data of the second awakening. The target sample speech is the speech data corresponding to the first awakening. Therefore, the target sample speech can be arranged and combined in pairs to obtain a sample speech pair, which is the input data for training the speech judgment model.

[0085] Step S330: determining label data corresponding to the sample speech pair according to the sample types of the two target sample speech pairs contained in the sample speech pair.

[0086] After the input of the training speech judgment model is constructed, the corresponding label data can be constructed. If the speech data at the first wake-up and the speech data at the second wake-up both contain the wake-up word speech, it can be considered that the smart device can be woken up after the second wake-up. If the speech data at the first wake-up and the speech data at the second wake-up do not contain the wake-up word speech, or one of the speech data at the first wake-up and the speech data at the second wake-up contains the wake-up word speech and the other does not contain the wake-up word speech, it is considered that the smart device cannot be woken up after the second wake-up.

[0087] Accordingly, if the sample types of the two target sample voices included in the sample voice pair are both positive samples, the label data corresponding to the sample voice pair is determined to be a first value; otherwise, the label data corresponding to the sample voice pair is determined to be a second value, wherein the first value is used to indicate that the smart device can be awakened, and the second value is used to indicate that the smart device cannot be awakened, and the second value can be smaller than the first value. For example, the first value can be 1, indicating that the smart device can be awakened, and the second value can be 0, indicating that the smart device cannot be awakened.

[0088] Step S340: training and generating a speech judgment model based on the sample speech pairs and the label data corresponding to the sample speech pairs.

[0089] During the training process, the sample speech is input into the initial speech judgment model whose network parameters have not been adjusted, and a prediction value for predicting whether to wake up the smart device can be output. Based on the prediction value and the label data, a loss value can be obtained. Then, the network parameters in the initial speech judgment model are optimized and adjusted according to the loss value, and finally the speech judgment model is trained and generated.

[0090] It should be noted that the input of the speech judgment model can be speech data as well as speech features extracted from the speech data, such as spectrum features, Fbank features, Mel-frequency cepstral coefficient MFCC features, etc. During the training process, the speech features of the two target sample speech included in the sample speech pair can be used as input data, and the label data remains unchanged to train and generate the speech judgment model.

[0091] After training and generating the voice judgment model, when the user wakes up the device twice, the voice data corresponding to the two wake-ups can be obtained. The voice judgment model can be used to accurately determine whether to wake up the smart device.

[0092] The wake-up method of the embodiment of the present application is described in detail below.

[0093] In step S210, a speech frame corresponding to the wake-up speech is acquired in real time.

[0094] When a user uses a wake-up word to wake up a smart device (such as a smart TV, smart speaker, etc.), a corresponding wake-up voice may be generated. The wake-up voice usually includes multiple voice frames, and the smart device may obtain the multiple voice frames in real time.

[0095] In step S220, when the 1st to f1th voice frames in the first time period are obtained, if the first wake-up scores corresponding to the 1st to f1th voice frames are less than the first score threshold, it is determined that the smart device cannot be woken up.

[0096] In the embodiment of the present application, since the voice frames are acquired in real time, that is, the smart device acquires the 1st voice frame, the 2nd voice frame..., until the last voice frame in sequence. When the 1st to f1th voice frames in the first time period are acquired, it can be determined whether the smart device can be woken up based on the 1st to f1th voice frames. Optionally, the 1st to f1th voice frames can be input into a pre-trained wake-up model to obtain a first wake-up score, and the first wake-up score is used to indicate the probability that the text information corresponding to the 1st to f1th voice frames belongs to the wake-up word. The wake-up model corresponds to a first score threshold, and the first score threshold refers to the minimum score value that can wake up the smart device. The first score threshold can be a pre-set value, for example, it can be 80%, 90%, etc. The first score threshold is a larger value, so as to avoid false wake-ups.

[0097] If the first wake-up scores corresponding to the 1st to f1th voice frames are greater than or equal to the first score threshold, it means that the smart device can be woken up, and the smart device can be directly woken up, and the process ends. If the first wake-up scores corresponding to the 1st to f1th voice frames are less than the first score threshold, it is determined that the smart device cannot be woken up, and the following step S230 can be continued.

[0098] In step S230, if the first wake-up score is greater than the second score threshold, the 1st to f1th speech frames are determined as the first target sequence.

[0099] If the smart device cannot be woken up, it can be further determined whether the first wake-up score is greater than the second score threshold, wherein the second score threshold is less than the first score threshold. If the first wake-up score is less than or equal to the second score threshold, it means that the first wake-up score is low, and the user may not intend to wake up the smart device. When making a secondary wake-up judgment, the 1st to f1th voice frames are not very referenceable. If the first wake-up score is greater than the second score threshold, it means that when the smart device cannot be woken up, the value of the first wake-up score is not particularly low. It may be that the user intends to wake up the smart device, but fails to wake up the smart device due to environmental noise and other reasons. The 1st to f1th voice frames can be used for secondary wake-up judgment. Therefore, the 1st to f1th voice frames are determined as the first target sequence, that is, the voice frame sequence corresponding to the first wake-up failure. The present application can also store the first target sequence in the first cache area.

[0100] Optionally, the wake-up scores corresponding to multiple voices that cannot wake up the smart device under multiple signal-to-noise ratios can be obtained, and the second score threshold can be determined according to the wake-up scores corresponding to the multiple voices. For example, a portion of the wake-up scores with lower scores (for example, 30%) among all the wake-up scores obtained can be eliminated, and the average of the remaining wake-up scores can be used as the second score threshold. In this way, the second score threshold can be prevented from being too low, thereby obtaining a more accurate first target sequence.

[0101] In step S240, after the preset time period, when the f2-th to f3-th voice frames in the second time period are obtained, if the second wake-up scores corresponding to the f2-th to f3-th voice frames are less than the first score threshold, it is determined that the smart device cannot be woken up.

[0102] In the case that the user fails to wake up the smart device after the first wake-up, the user can wake up the device for the second time. When the user wakes up the device for the second time, there is usually a short time interval between the two wake-ups. Therefore, the second target sequence can be obtained after a preset time period. If a speech frame sequence of a time period is obtained before the preset time period, the first target sequence can be updated according to the speech frame sequence, that is, the speech frame sequence of the time period is used as the first target sequence, and new speech frame sequences are continuously obtained until the time interval of the acquired speech frame sequence is greater than the time interval of the first target sequence, and the time period corresponding to the acquired speech frame sequence is used as the second time period, that is, the speech frame sequence of the second time period is obtained.

[0103] Optionally, multiple second sample wake-up voices that have been awakened at least twice can be obtained; the preset time period is determined according to the time interval between the end frame of the first awakening and the start frame of the second awakening in the second sample wake-up voice. For example, the multiple obtained time intervals are sorted from low to high, the top 10% of the time intervals are selected and averaged to obtain the preset time period.

[0104] Among them, the second sample wake-up voice is the voice data of the second wake-up, that is, the user did not wake up the smart device when shouting the wake-up word for the first time, and the user woke it up for the second time in a short period of time. If the user did not wake up the smart device when shouting the wake-up word for the first time, and shouted the wake-up word again after a long time, then this wake-up is a new wake-up, that is, the two wake-up voices do not belong to the second sample wake-up voice. If the user says a word close to the wake-up word twice in a short period of time, the user may not intend to wake up the device at this time. If the wake-up score is in the preset range (that is, greater than the second score threshold and less than the first score threshold), it can also be determined as a second wake-up.

[0105] If a speech frame sequence of a second time period is obtained after a preset time period, and if the speech frame sequence is the f2th to f3th speech frames, determine whether the smart device can be woken up based on the f2th to f3th speech frames. Optionally, the f2th to f3th speech frames can be input into the wake-up model to obtain a second wake-up score, and the second wake-up score is used to indicate the probability that the text information corresponding to the f2th to f3th speech frames belongs to the wake-up word. If the second wake-up score is greater than or equal to the first score threshold, it means that the smart device can be woken up, and the smart device can be directly woken up, and the process technology. If the second wake-up score is less than the first score threshold, it means that the smart device cannot be woken up, and the following step S250 can be continued.

[0106] In step S250, if the second wake-up score is greater than the second score threshold, the f2th to f3th speech frames are determined as the second target sequence; wherein f1, f2, and f3 are positive integers greater than 1 and increase sequentially.

[0107] Similar to the aforementioned step S230, if the smart device cannot be woken up, it can be further determined whether the second wake-up score is greater than the second score threshold. If the second wake-up score is less than or equal to the second score threshold, it means that the second wake-up score is low, and the f2-f3 voice frames are not very referenceable when making a secondary wake-up judgment. If the second wake-up score is greater than the second score threshold, it means that when the smart device cannot be woken up, the value of the second wake-up score is not particularly low, and the f2-f3 voice frames can be used for secondary wake-up judgment. That is, the f2-f3 voice frames are determined as the second target sequence, that is, the voice frame sequence corresponding to the second wake-up failure. The present application can also store the second target sequence in the second cache area.

[0108] Optionally, multiple second sample wake-up voices that have been awakened at least twice can be obtained; the preset time period is determined according to the time interval between the end frame of the first awakening and the start frame of the second awakening in the second sample wake-up voice. For example, the multiple time periods obtained are sorted from low to high, the top 10% of the number is selected and the average is calculated to obtain the preset time period.

[0109] In step S260, the first target sequence and the second target sequence are input into a pre-trained speech judgment model to determine whether to wake up the smart device.

[0110] In an embodiment of the present application, the first target sequence and the second target sequence can be directly input into a pre-trained speech judgment model to obtain a third value; if the third value is greater than or equal to a preset value, the smart device is awakened; if the third value is less than the preset value, the smart device is not awakened. The preset value can be determined based on the first value and the second value. For example, during the training process, if the first value of the label data is 1, it means that the smart device can be awakened, and the second value of the label data is 0, it means that the smart device cannot be awakened, the preset value can be 0.8, 0.9, etc.

[0111] In some embodiments, a first speech frame sequence and a second speech frame sequence of a preset length can be selected from the first target sequence and the second target sequence, respectively, so that the first speech frame sequence and the second speech frame sequence have the same speech frame length as the input data when training the speech judgment model. Optionally, a speech frame of a preset length located between the first target sequence and the second target sequence can be selected, or a speech frame of a preset length can be selected backward from the starting frame of the first target sequence and the second target sequence, or a speech frame of a preset length can be selected forward from the ending frame of the first target sequence and the second target sequence.

[0112] Afterwards, the first speech frame sequence and the second speech frame sequence are input into a pre-trained speech judgment model to determine whether to wake up the smart device. In this way, by comparing the correlation between two speech frame sequences of equal length, the accuracy of correlation judgment can be improved.

[0113] The wake-up method of the embodiment of the present application can pre-determine whether the user has performed a second wake-up by extracting a first target sequence and a second target sequence with a certain time interval. Furthermore, during the second wake-up, the same wake-up algorithm as that used for the first wake-up is no longer used, that is, the first target sequence is treated as noise, and the second target sequence is processed by the wake-up algorithm. Instead, the correlation between the two wake-ups is considered, and the first target sequence and the second target sequence are input into a pre-trained voice judgment model to determine whether to wake up the smart device. Therefore, the wake-up rate of the second wake-up can be improved, and the user's interactive experience can be enhanced.

[0114] Corresponding to the above method embodiment, the present application embodiment also provides a wake-up device, see Figure 4 , the awakening device 400 includes:

[0115] A voice frame real-time acquisition module 410 is used to acquire the voice frame corresponding to the wake-up voice in real time;

[0116] The first target sequence determination module 420 is used to, when acquiring the 1st to f1th speech frames in the first time period, determine that the smart device cannot be woken up if the first wake-up scores corresponding to the 1st to f1th speech frames are less than the first score threshold; if the first wake-up scores are greater than the second score threshold, determine the 1st to f1th speech frames as the first target sequence; wherein the second score threshold is less than the first score threshold;

[0117] The second target sequence determination module 430 is used to, after a preset time period, when the f2-th to f3-th voice frames in the second time period are obtained, if the second wake-up score corresponding to the f2-th to f3-th voice frames is less than the first score threshold, determine that the smart device cannot be woken up; if the second wake-up score is greater than the second score threshold, determine the f2-th to f3-th voice frames as the second target sequence; wherein f1, f2, and f3 are positive integers greater than 1 and increase in sequence;

[0118] The wake-up judgment module 440 is used to input the first target sequence and the second target sequence into a pre-trained speech judgment model to determine whether to wake up the smart device.

[0119] Optionally, the awakening device 400 further includes:

[0120] A target sample speech acquisition module is used to acquire a target sample speech whose sample types include positive samples and negative samples, wherein the target sample speech whose sample type is a positive sample includes a wake-up word speech, and the target sample speech whose sample type is a negative sample does not include a wake-up word speech;

[0121] An input data construction module is used to arrange and combine the target sample voices to obtain a sample voice pair including two target sample voices; wherein the sample voice pair is the input data;

[0122] A label data building module, used to determine label data corresponding to the sample speech pair according to the sample types of the two target sample speech included in the sample speech pair;

[0123] The speech judgment model training module is used to train and generate a speech judgment model based on sample speech pairs and label data corresponding to the sample speech pairs.

[0124] Optionally, the label data construction module is specifically used to determine that the label data corresponding to the sample speech pair is a first value if the sample types of the two target sample speech pairs contained in the sample speech pair are both positive samples; otherwise, determine that the label data corresponding to the sample speech pair is a second value, wherein the second value is smaller than the first value.

[0125] Optionally, the wake-up judgment module 440 is specifically used to input the first target sequence and the second target sequence into a pre-trained speech judgment model to obtain a third value; if the third value is greater than or equal to a preset value, the smart device is woken up; wherein the preset value is determined based on the first value and the second value; if the third value is less than the preset value, the smart device is not woken up.

[0126] Optionally, the awakening device 400 further includes:

[0127] A speech frame sequence selection module, used to select a first speech frame sequence and a second speech frame sequence with a preset length from the first target sequence and the second target sequence respectively;

[0128] The wake-up judgment module 440 is specifically used to input the first voice frame sequence and the second voice frame sequence into a pre-trained voice judgment model to determine whether to wake up the smart device.

[0129] Optionally, the awakening device 400 further includes:

[0130] A first wake-up score determination module, used to input the 1st to f1th speech frames into a pre-trained wake-up model to obtain a first wake-up score, where the first wake-up score is used to indicate the probability that the text information corresponding to the 1st to f1th speech frames belongs to the wake-up word;

[0131] The second wake-up score determination module is used to input the f2-th to f3-th speech frames into the wake-up model to obtain a second wake-up score, and the second wake-up score is used to indicate the probability that the text information corresponding to the f2-th to f3-th speech frames belongs to the wake-up word.

[0132] Optionally, the awakening device 400 further includes:

[0133] A sample wake-up score acquisition module is used to obtain the wake-up scores corresponding to multiple voices that cannot wake up the smart device under multiple signal-to-noise ratios;

[0134] The second score threshold determination module is used to determine the second score threshold according to the wake-up scores corresponding to the multiple voices.

[0135] The specific details of each module or unit in the above device have been described in detail in the corresponding method, so they will not be repeated here.

[0136] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0137] In an exemplary embodiment of the present application, an electronic device is further provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the above-mentioned wake-up method in this exemplary embodiment.

[0138] Figure 5 This is a schematic diagram of the structure of an electronic device in an embodiment of the present application. It should be noted that: Figure 5 The electronic device 500 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0139] like Figure 5 As shown, the electronic device 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage part 508 to the random access memory (RAM) 503. In the RAM 503, various programs and data required for system operation are also stored. The central processing unit 501, the ROM 502 and the RAM 503 are connected to each other through a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0140] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a local area network (LAN) card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage section 508 as needed.

[0141] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 509, and / or installed from a removable medium 511. When the computer program is executed by the central processing unit 501, various functions defined in the device of the present application are executed.

[0142] In an embodiment of the present application, a computer-readable storage medium is further provided, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned wake-up method is implemented.

[0143] It should be noted that the computer-readable storage medium shown in the present application may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, radio frequency, etc., or any suitable combination of the above.

[0144] In an embodiment of the present application, a computer program product is also provided. When the computer program product is run on a computer, the computer executes the above-mentioned wake-up method.

[0145] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0146] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A wake-up method, characterized in that: The method comprises: Acquire the voice frame corresponding to the wake-up voice in real time; When the 1st to f1th voice frames in the first time period are obtained, if the first wake-up scores corresponding to the 1st to f1th voice frames are greater than or equal to the first score threshold, the smart device is woken up and the process ends; if the first wake-up scores corresponding to the 1st to f1th voice frames are less than the first score threshold, it is determined that the smart device cannot be woken up; If the first wake-up score is greater than a second score threshold, determining the 1st to f1th speech frames as a first target sequence; wherein the second score threshold is less than the first score threshold; After the preset time period, when the f2-th to f3-th voice frames in the second time period are obtained, if the second wake-up scores corresponding to the f2-th to f3-th voice frames are less than the first score threshold, it is determined that the smart device cannot be woken up; If the second wake-up score is greater than the second score threshold, determine the f2th to f3th speech frames as the second target sequence; wherein f1, f2, and f3 are positive integers greater than 1 and increase in sequence; Inputting the first target sequence and the second target sequence into a pre-trained speech judgment model to determine whether to wake up the smart device; Inputting the first target sequence and the second target sequence into a pre-trained voice judgment model to determine whether to wake up the smart device includes: Inputting the first target sequence and the second target sequence into a pre-trained speech judgment model to obtain a third value; If the third value is greater than or equal to a preset value, waking up the smart device; wherein the preset value is determined based on the first value and the second value; If the third value is smaller than the preset value, the smart device is not woken up.

2. The method according to claim 1, characterized in that The training method of the speech judgment model includes: Acquire a target sample voice whose sample type includes a positive sample and a negative sample, wherein the target sample voice whose sample type is a positive sample includes a wake-up word voice, and the target sample voice whose sample type is a negative sample does not include a wake-up word voice; Arrange and combine the target sample voices to obtain a sample voice pair including two target sample voices; wherein the sample voice pair is input data; Determining label data corresponding to the sample speech pair according to sample types of two target sample speech pairs included in the sample speech pair; The speech judgment model is trained and generated based on the sample speech pairs and the label data corresponding to the sample speech pairs.

3. The method according to claim 2, characterized in that The step of determining label data corresponding to the sample speech pair according to the sample types of the two target sample speech pairs included in the sample speech pair includes: If the sample types of the two target sample voices included in the sample voice pair are both positive samples, determining that the label data corresponding to the sample voice pair is a first value; Otherwise, determine that the label data corresponding to the sample speech pair is a second value, wherein the second value is smaller than the first value.

4. The method according to claim 2, characterized in that: The method further comprises: Selecting a first speech frame sequence and a second speech frame sequence having a preset length from the first target sequence and the second target sequence respectively; The step of inputting the first target sequence and the second target sequence into a pre-trained speech judgment model to determine whether to wake up the smart device includes: The first speech frame sequence and the second speech frame sequence are input into a pre-trained speech judgment model to determine whether to wake up the smart device.

5. The method according to claim 1, characterized in that The method further comprises: Inputting the 1st to f1th speech frames into a pre-trained wake-up model to obtain the first wake-up score, where the first wake-up score is used to indicate the probability that the text information corresponding to the 1st to f1th speech frames belongs to the wake-up word; The f2-th to f3-th speech frames are input into the wake-up model to obtain the second wake-up score, where the second wake-up score is used to indicate the probability that the text information corresponding to the f2-th to f3-th speech frames belongs to the wake-up word.

6. The method according to claim 1, characterized in that The method further comprises: Obtain wake-up scores corresponding to multiple voices that cannot wake up the smart device under multiple signal-to-noise ratios; The second score threshold is determined according to the wake-up scores respectively corresponding to the multiple voices.

7. A wake-up device, characterized in that: The device comprises: A voice frame real-time acquisition module is used to acquire the voice frame corresponding to the wake-up voice in real time; The first target sequence determination module is used to, when the 1st to f1th voice frames in the first time period are acquired, if the first wake-up scores corresponding to the 1st to f1th voice frames are greater than or equal to the first score threshold, wake up the smart device and end the process; if the first wake-up scores corresponding to the 1st to f1th voice frames are less than the first score threshold, determine that the smart device cannot be woken up; if the first wake-up scores are greater than the second score threshold, determine the 1st to f1th voice frames as the first target sequence; wherein the second score threshold is less than the first score threshold; A second target sequence determination module is used for, after a preset time period, when the f2-f3th voice frames in the second time period are obtained, if the second wake-up scores corresponding to the f2-f3th voice frames are less than the first score threshold, determining that the smart device cannot be woken up; if the second wake-up scores are greater than the second score threshold, determining the f2-f3th voice frames as the second target sequence; wherein f1, f2, and f3 are positive integers greater than 1 and increase in sequence; A wake-up judgment module, used for inputting the first target sequence and the second target sequence into a pre-trained speech judgment model to determine whether to wake up the smart device; The wake-up judgment module is specifically used to input the first target sequence and the second target sequence into a pre-trained speech judgment model to obtain a third value; if the third value is greater than or equal to a preset value, the smart device is woken up; wherein the preset value is determined based on the first value and the second value; if the third value is less than the preset value, the smart device is not woken up.

8. An electronic device, characterized in that: include: A processor, wherein the processor is used to execute a computer program stored in a memory, and when the computer program is executed by the processor, the wake-up method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the wake-up method according to any one of claims 1 to 6 is implemented.

Images