Voice wake-up method, device, electronic device and storage medium

Abstract

The present invention provides a voice wake-up method, device, electronic device and storage medium, wherein the method comprises: determining a voice to be classified; determining the wake-up category of the voice to be classified based on the correlation between the voice features of the auxiliary voice and the voice features of the voice to be classified, and the voice features of the auxiliary voice and the voice features of the voice to be classified, wherein the auxiliary voice includes a non-wake-up voice and / or a false wake-up voice; and performing a wake-up operation when the wake-up category is a wake-up voice. The method, device, electronic device and storage medium provided by the present invention can obtain the wake-up category of the voice to be classified based on the correlation between the auxiliary voice and the voice to be classified, and the voice features of the combined auxiliary voice and the voice to be classified, thereby realizing the introduction of auxiliary voice to assist in recognition when performing wake-up classification on the voice, improving the accuracy of identifying the wake-up category, reducing the probability of non-wake-up and false wake-up, and thus improving the user experience.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a voice wake-up method, device, electronic device and storage medium. Background Art

[0002] With the development of artificial intelligence technology, voice interaction has been widely used in various scenarios. The intelligent voice wake-up system is the first link of voice interaction. Voice wake-up is a machine learning method that allows the machine to automatically determine whether to respond based on a fixed wake-up word.

[0003] The specific process of existing voice interaction is generally that the microphone device first receives voice data, and then uses the preset intelligent voice wake-up model to determine whether the current continuous voice stream belongs to the preset wake-up word. If the cumulative score of the output of the intelligent voice wake-up model exceeds the preset threshold, the device is in the awakened state and starts to respond to subsequent voice interactions.

[0004] However, the existing intelligent voice wake-up system is a fixed version. In certain scenarios, the device may be woken up by mistake even though the user has not issued a wake-up command to the device. In addition, due to factors such as the user's accent or pronunciation habits, the user may say the correct wake-up word, but the system recognition result does not exceed the set wake-up threshold, making it difficult or even impossible for the intelligent voice wake-up system to be woken up. At a time when users have increasingly higher requirements for user experience, the frequent failure to wake up or mistaken wake-up will greatly affect the user experience and lead to a low user experience. Summary of the invention

[0005] The present invention provides a voice wake-up method, device, electronic device and storage medium, which are used to solve the defect that the intelligent voice wake-up system in the prior art often fails to wake up or wakes up by mistake, resulting in a low user experience.

[0006] The present invention provides a voice wake-up method, comprising:

[0007] Determine the speech to be classified;

[0008] Determining the wake-up category of the speech to be classified based on the correlation between the speech features of the auxiliary speech and the speech features of the speech to be classified, and the speech features of the auxiliary speech and the speech features of the speech to be classified, wherein the auxiliary speech includes a non-wake-up speech and / or a false wake-up speech;

[0009] When the wake-up category is a wake-up voice, a wake-up operation is performed.

[0010] According to a voice wake-up method provided by the present invention, the wake-up category of the voice to be classified is determined based on the correlation between the voice features of the auxiliary voice and the voice features of the voice to be classified, and the voice features of the auxiliary voice and the voice features of the voice to be classified, including:

[0011] Determining an attention weight based on a correlation between the speech feature of the speech to be classified and the speech feature of the auxiliary speech;

[0012] Performing mixed feature extraction on the speech features of the auxiliary speech and the speech features of the speech to be classified to obtain mixed speech features;

[0013] Based on the attention weight and the mixed speech feature, a wake-up category of the speech to be classified is determined.

[0014] According to a voice wake-up method provided by the present invention, the voice features of the auxiliary voice and the voice features of the voice to be classified are mixed and extracted to obtain mixed voice features, including:

[0015] Performing feature fusion on the speech features of the auxiliary speech and the speech features of the speech to be classified to obtain fused speech features;

[0016] Performing feature extraction on the fused speech features to obtain high-level fused features;

[0017] The mixed speech feature is determined based on the high-level fusion feature and the speech feature of the auxiliary speech.

[0018] According to a voice wake-up method provided by the present invention, determining the wake-up category of the speech to be classified based on the attention weight and the mixed speech feature includes:

[0019] Based on the weight value of each frame in the attention weight, weighting the speech features in the mixed speech feature corresponding to the positions of each frame in the attention weight, to obtain the speech weighted features corresponding to each frame in the mixed speech feature;

[0020] Based on the speech weighted features corresponding to each frame in the mixed speech features, the wake-up category of the speech to be classified is determined.

[0021] According to a voice wake-up method provided by the present invention, the attention weight is determined based on the correlation between the voice features of the to-be-classified voice and the voice features of the auxiliary voice, including:

[0022] Determine the attention value of each frame position based on the correlation between the speech features of each frame in the speech to be classified and the speech features in the auxiliary speech corresponding to each frame position in the speech to be classified;

[0023] Based on the attention value of each frame position, a weight value of each frame in the attention weight is determined.

[0024] According to a voice wake-up method provided by the present invention, the non-wake-up voice is determined based on the following steps:

[0025] Acquire a first sample voice without performing a wake-up operation;

[0026] If the probability that the first sample voice belongs to the wake-up category is greater than a preset threshold, and there is a wake-up operation within a preset time after the first sample voice is received, it is determined that the first sample voice is a non-wake-up voice.

[0027] According to a voice wake-up method provided by the present invention, the false wake-up voice is determined based on the following steps:

[0028] Acquire a second sample voice for performing a wake-up operation;

[0029] Performing semantic recognition on subsequent speech of the second sample speech to obtain a semantic recognition result;

[0030] If the semantic recognition result is non-semantic, it is determined that the second sample voice is a false wake-up voice.

[0031] The present invention also provides a voice wake-up device, comprising:

[0032] A determination module, used for determining the speech to be classified;

[0033] A classification module, configured to determine the wake-up category of the speech to be classified based on the correlation between the speech features of the auxiliary speech and the speech features of the speech to be classified, and the speech features of the auxiliary speech and the speech features of the speech to be classified, wherein the auxiliary speech includes a non-wake-up speech and / or a false wake-up speech;

[0034] The wake-up module is used to perform a wake-up operation when the wake-up category is a wake-up voice.

[0035] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-mentioned voice wake-up methods are implemented.

[0036] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned voice wake-up methods are implemented.

[0037] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of any of the above-mentioned voice wake-up methods are implemented.

[0038] The voice wake-up method, device, electronic device and storage medium provided by the present invention obtain the wake-up category of the voice to be classified through the correlation between the auxiliary voice and the voice to be classified, and the voice features of the combined auxiliary voice and the voice to be classified, thereby realizing the introduction of auxiliary voice to assist in recognition when performing wake-up classification on the voice, improving the accuracy of identifying the wake-up category, reducing the probability of non-awakening and false awakening, and thus improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 It is a flowchart of the voice wake-up method provided by the present invention;

[0041] Figure 2 This is one of the flowcharts of the method for obtaining the wake-up category provided by the present invention;

[0042] Figure 3 It is a flow chart of the method for obtaining mixed speech features provided by the present invention;

[0043] Figure 4 This is the second flow chart of the method for obtaining the wake-up category provided by the present invention;

[0044] Figure 5 It is a flowchart of a method for obtaining non-awakening voice provided by the present invention;

[0045] Figure 6 It is a flowchart of the method for obtaining false wake-up voice provided by the present invention;

[0046] Figure 7 It is a flowchart of a method for obtaining a voice wake-up category based on multi-head attention provided by the present invention;

[0047] Figure 8 It is a structural schematic diagram of the voice wake-up device provided by the present invention;

[0048] Fig. 9 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] The current intelligent voice wake-up system determines whether to respond based on the wake-up score of the current audio segment. Therefore, it is inevitable that the user's accent, pronunciation habits, and speaking speed may affect the situation that the correct wake-up word is said, but the wake-up score does not exceed the preset wake-up threshold, resulting in the intelligent voice wake-up system not waking up. In addition, in a quiet or noisy environment, the user does not issue an effective command to the device, and the intelligent voice wake-up system responds, which is a false wake-up. And if the intelligent voice wake-up system frequently fails to wake up or wakes up by mistake, it will greatly affect the user experience and lead to a low user experience.

[0051] Therefore, how to improve the accuracy of voice wake-up classification and reduce the frequency of non-wake-up and false wake-up to improve user experience is a technical problem that needs to be solved urgently in this field.

[0052] In view of the above situation, an embodiment of the present invention provides a voice wake-up method. Figure 1 Schematic diagram of the flow of the voice wake-up method provided by the present invention. Figure 1 As shown, the method includes:

[0053] Step 110: determine the speech to be classified.

[0054] Specifically, the speech to be classified is the speech received by the microphone in real time. The speech to be classified may be the speech of a single wake-up word, the speech of a sentence with a wake-up word, or the speech without a wake-up word. The embodiment of the present invention does not limit this.

[0055] Step 120, based on the correlation between the voice features of the auxiliary voice and the voice features of the voice to be classified, and the voice features of the auxiliary voice and the voice features of the voice to be classified, determine the wake-up category of the voice to be classified, the auxiliary voice includes the non-wake-up voice and / or the false wake-up voice;

[0056] Step 130: When the wake-up category is a wake-up voice, perform a wake-up operation.

[0057] Most of the current intelligent voice wake-up systems encode the input voice and then decode the extracted high-level voice features to obtain the wake-up category of the input voice. During the decoding process, the intelligent voice wake-up system does not consider the impact of each user's accent, pronunciation habits, and speech speed on the prediction of the wake-up category, resulting in the intelligent voice wake-up system not waking up and erroneous wake-up. Therefore, the embodiment of the present invention uses the user's unawakened voice and / or erroneous wake-up voice as auxiliary voice, and when decoding the high-level voice features of the input voice, the auxiliary voice assists in predicting the wake-up category of the input voice. Among them, the unawakened voice is the voice in which the user says the correct wake-up word, but fails to respond to the wake-up due to misjudgment, and the erroneous wake-up voice is the voice in which the user does not issue a valid command to the intelligent voice wake-up system, but responds due to misjudgment.

[0058] Considering that the auxiliary voice is composed of the user's non-wake-up voice and / or false wake-up voice, the non-wake-up voice and / or false wake-up voice records voices that are similar to the pronunciation of the correct wake-up word, but due to the user's accent, pronunciation, speaking speed and other habitual factors or environmental factors such as noise, the intelligent voice wake-up system may misjudge the operation. Therefore, the embodiment of the present invention uses the correlation between the voice features of the auxiliary voice and the voice features of the voice to be classified as one of the parameters for predicting the wake-up category of the voice to be classified.

[0059] At the same time, considering that the auxiliary voice is the user's non-awakened voice and / or false awakening voice, which records the user's accent, pronunciation habits, speaking speed and other information, the voice features of the auxiliary voice can enhance the voice features of the voice to be classified. Therefore, the embodiment of the present invention also uses the voice feature information of the auxiliary voice as one of the parameters for predicting the wake-up type of the voice to be classified.

[0060] Specifically, the auxiliary voice is composed of the user's non-wake-up voice and / or false wake-up voice, and the user's voice feature information can be extracted from the auxiliary voice. The correlation between the voice feature information of the auxiliary voice and the voice feature information to be classified is combined with the voice feature information of the auxiliary voice and the voice feature information to be classified to perform wake-up category prediction to obtain the wake-up category of the voice to be classified, and when the obtained wake-up category is a wake-up voice, the intelligent voice wake-up system performs a wake-up operation, and then receives and / or executes subsequent voice commands.

[0061] It should be noted that, according to the correlation between the voice features of the auxiliary speech and the voice features to be classified, the similarity score of the voice features of the two can be obtained, or the attention information between the two can be obtained by performing attention interaction based on the voice features of the two, and the embodiment of the present invention is not limited to this. In addition, the correlation between the voice features of the auxiliary speech and the voice features to be classified, and the wake-up category prediction based on the voice features of the auxiliary speech and the voice features to be classified can be to directly predict the wake-up category of the voice to be classified based on the voice features of the auxiliary speech, the voice features to be classified, and the correlation between the voice information of the two, or it can be to first perform feature fusion on the voice features of the auxiliary speech and the voice features to be classified, and then combine the voice features to be classified and the correlation between the voice information of the two to predict the wake-up category of the voice to be classified, and the embodiment of the present invention is not limited to this.

[0062] The voice wake-up method provided in the embodiment of the present invention obtains the wake-up category of the voice to be classified through the correlation between the auxiliary voice and the voice to be classified, and combines the voice features of the auxiliary voice and the voice to be classified, thereby realizing the introduction of auxiliary voice to assist in recognition when performing wake-up classification on the voice, improving the accuracy of identifying the wake-up category, reducing the probability of non-awakening and false awakening, and thereby improving the user experience.

[0063] Based on the above embodiments, Figure 2 FIG. 1 is one of the flow charts of the method for obtaining the wake-up category provided by the present invention. Figure 2 As shown, step 120 includes:

[0064] Step 210: determining an attention weight based on the correlation between the speech features of the speech to be classified and the speech features of the auxiliary speech.

[0065] The intelligent voice wake-up system may have deviations in understanding semantics due to the user's accent, pronunciation, speaking speed and other habits, resulting in failure to wake up or false awakening. Therefore, the embodiment of the present invention introduces a multi-head attention mechanism to convert the correlation between the speech features of the speech to be classified and the speech features of the auxiliary speech into attention weights, so that the prediction results can be adjusted based on the attention weights when predicting the subsequent wake-up category.

[0066] It should be noted that the attention weight can be the weight value of each frame in the speech to be classified. For any frame, the weight value here can reflect the correlation between the frame and the corresponding frame in the auxiliary speech. The higher the correlation, the higher the weight of the frame, and the higher its importance in the wake-up category prediction.

[0067] Step 220, performing mixed feature extraction on the speech features of the auxiliary speech and the speech features of the speech to be classified to obtain mixed speech features;

[0068] Taking into account that the auxiliary voice is the user's historical non-awakening voice and / or false awakening voice, when predicting the wake-up category of the voice to be classified, the user's speaking habits can be obtained based on the voice features in the historical auxiliary voice. Based on the commonality of the user's speaking habits, the wake-up category of the voice to be classified can be predicted more accurately, that is, the common voice features of the auxiliary voice and the voice to be classified can be used to assist the intelligent voice wake-up system in predicting the wake-up category of the voice to be classified.

[0069] Specifically, in order to obtain a mixed speech feature of the speech feature of the auxiliary speech and the speech feature of the speech to be classified, the speech feature of the auxiliary speech and the speech feature of the speech to be classified can be fused, and then the high-level speech feature of the fused speech feature can be extracted, and the high-level speech feature can be considered as the final mixed speech feature. At the same time, considering that the high-level speech feature has undergone multi-layer feature extraction, in order to ensure the accuracy of the classification of the predicted wake-up category, the high-level speech feature and the speech feature of the auxiliary speech can also be subjected to a residual operation to obtain the final mixed speech feature, and the embodiment of the present invention does not limit this.

[0070] Step 230: Determine the wake-up category of the speech to be classified based on the attention weight and the mixed speech feature.

[0071] It should be noted that the wake-up category of the speech to be classified can be first predicted based on the mixed speech features, and the predicted result can be adjusted according to the attention weight; the mixed speech features can also be first weighted according to the attention weight to obtain the weighted speech features, and then predicted based on the weighted speech features. The embodiments of the present invention are not limited to this.

[0072] Based on the above embodiments, Figure 3 FIG. 1 is a flow chart of the method for obtaining mixed speech features provided by the present invention. Figure 3 As shown, step 220 includes:

[0073] Step 310, performing feature fusion on the speech features of the auxiliary speech and the speech features of the speech to be classified to obtain fused speech features;

[0074] Step 320, extracting the fused speech features to obtain high-level fused features;

[0075] Considering that the voice features of the auxiliary voice and the voice features of the voice to be classified are fused at a low level, more voice detail features can be included, and it is easier to find the common voice features between the voice features of the auxiliary voice and the voice features of the voice to be classified from the voice detail features. Therefore, the embodiment of the present invention first fuses the voice features of the auxiliary voice with the voice features of the voice to be classified.

[0076] Specifically, the speech features of the auxiliary speech are fused with the speech features of the speech to be classified, wherein the fusion method can be fused by dimensional splicing (concat feature fusion), or the fusion method can also be based on the dimension of the speech features of the speech to be classified, and fused by feature splicing (Add feature fusion), which is not limited in the embodiment of the present invention. In addition, after the speech features of the auxiliary speech are fused with the speech features of the speech to be classified, feature extraction is performed on the fused speech features to obtain high-level fusion features.

[0077] Step 330, determining mixed speech features based on the high-level fusion features and the speech features of the auxiliary speech.

[0078] Considering that the high-level speech features are obtained through multi-layer feature extraction, there may be loss of information about the auxiliary speech itself. In order to ensure the accuracy of the classification of the predicted wake-up category, the embodiment of the present invention performs a residual operation on the speech features of the auxiliary speech. It should be noted that the residual operation can be a weighted operation of the high-level fusion features and the speech features of the auxiliary speech, or a standard deviation operation of the high-level fusion features and the speech features of the auxiliary speech, which is not limited in the embodiment of the present invention.

[0079] Based on the above embodiments, Figure 4 FIG. 2 is a flow chart of the method for obtaining the wake-up category provided by the present invention. Figure 4 As shown, step 230 includes:

[0080] Step 410, based on the weight value of each frame in the attention weight, weight the speech features corresponding to the positions of each frame in the attention weight in the mixed speech feature to obtain the speech weighted features corresponding to each frame in the mixed speech feature;

[0081] Step 420: Determine the wake-up category of the speech to be classified based on the weighted speech features corresponding to each frame in the mixed speech features.

[0082] Considering that when predicting the wake-up category of the speech to be classified, if the speech features of the useful frames can be strengthened and the speech features of the useless frames can be weakened, the predicted wake-up category of the speech to be classified will be more accurate. Therefore, the embodiment of the present invention weights the mixed speech features of each frame for wake-up category prediction according to the weight value of each frame in the attention weight.

[0083] Specifically, taking the total number of frames of the mixed speech feature as a benchmark, the speech features of each frame in the mixed speech feature are weighted according to the weight value of the frame corresponding to the position of each frame in the attention weight, so as to obtain the speech weighted features corresponding to each frame in the mixed speech feature, and then based on the speech weighted features corresponding to each frame in the mixed speech feature obtained in step 410, the wake-up category of the speech to be classified is predicted, so as to obtain the wake-up category of the speech to be classified.

[0084] It should be noted that the weight value of each frame in the attention weight represents the importance of the frame at the corresponding position in the mixed speech feature. The importance of each frame can better distinguish the wake-up category when making predictions. For any frame, the higher the weight value of the frame, the more the weighted speech feature corresponding to the frame can be strengthened and highlighted in the weighted mixed speech feature, and the stronger the distinction in the wake-up category prediction; the lower the weight value of the frame, the more the weighted speech feature corresponding to the frame will be weakened in the weighted mixed speech feature, and the less interference in the wake-up category prediction.

[0085] Based on the above embodiment, step 210 includes:

[0086] Determine the attention value of each frame position based on the correlation between the speech features of each frame in the speech to be classified and the speech features in the auxiliary speech corresponding to the position of each frame in the speech to be classified;

[0087] Based on the attention value of each frame position, the weight value of each frame in the attention weight is determined.

[0088] Specifically, the speech features of each frame in the speech to be classified and the speech features in the auxiliary speech corresponding to the frame position in the speech to be classified are converted into attention, so as to obtain the attention value of each frame position, and the attention values ​​of each frame in the speech to be classified are normalized to obtain the weight value of each frame in the attention weight. Here, for any frame position, the larger the attention value of the frame position, the stronger the correlation between the speech features of the frame in the speech to be classified and the speech features corresponding to the corresponding frame in the auxiliary speech, and the smaller the attention value of the frame position, the weaker the correlation between the speech features of the frame in the speech to be classified and the speech features corresponding to the corresponding frame in the auxiliary speech.

[0089] Taking into account the situation that the number of frames in the auxiliary speech is different from the number of frames in the speech to be classified, the embodiment of the present invention first integrates the frame lengths in the speech to be classified and the auxiliary speech, so that the number of frames in the speech to be classified and the auxiliary speech are the same, and then subsequently calculates the weight value operation of each frame in the attention weight.

[0090] Based on any of the above embodiments, the present invention also provides an embodiment for predicting wake-up categories in the absence of auxiliary speech. Specifically, in the absence of auxiliary speech, the speech features of the auxiliary speech are set to null values, and the speech features of each frame of the speech to be classified are first encoded to obtain high-level speech features of each frame, and then the attention weights of the high-level speech features of each frame are converted based on the speech features of the auxiliary speech being null values ​​to obtain the attention weights of each frame. At the same time, the high-level speech features of each frame are decoded based on the speech features of the auxiliary speech being null values ​​to obtain decoded features of each frame, and then the decoded features are weighted by the attention weights of each frame to obtain weighted speech features of each frame. Finally, the wake-up category is predicted based on the weighted speech features of each frame. If the wake-up category is wake-up speech, the wake-up operation is performed.

[0091] At present, due to the small number of daily interactions by users, the existing intelligent voice wake-up will have fewer cases of failure to wake up and false wake-up in the limited number of uses, which further makes it difficult to collect samples of failure to wake up voice and false wake-up voice.

[0092] Therefore, how to collect the non-awakening voice and the false awakening voice during use, and apply the collected non-awakening voice and the false awakening voice to the awakening category prediction is a problem to be solved in the art.

[0093] Based on any of the above embodiments, Figure 5 FIG. 1 is a flow chart of the method for obtaining the non-awakened voice provided by the present invention. Figure 5 As shown, the non-wake-up voice in step 110 is determined based on the following steps:

[0094] Step 510, obtaining a first sample voice without performing a wake-up operation;

[0095] Specifically, the first sample speech comes from the speech to be classified whose wake-up category is predicted to be non-wake-up speech in the method in the above embodiment.

[0096] Step 520: If the probability that the first sample voice belongs to the wake-up category is greater than a preset threshold, and there is a wake-up operation within a preset time after receiving the first sample voice, then the first sample voice is determined to be a non-wake-up voice.

[0097] The wake-up category of non-wake-up speech can be divided into two cases, one is that the user says the wake-up word but the speech is not awakened, and the other is the user's ordinary conversation speech. Considering that the probability of the wake-up speech in the wake-up category of non-wake-up speech is not much different from the value of the wake-up threshold, and is much higher than the probability of the wake-up speech in the wake-up category of ordinary conversation speech, therefore, the preset threshold value in the embodiment of the present invention is used to distinguish the above two cases.

[0098] At the same time, it is taken into account that the user may use a wake-up word in a normal conversation, and the probability of the wake-up speech in the wake-up category of the wake-up word exceeds the preset threshold but does not exceed the wake-up threshold. At this time, the user is not performing a wake-up operation. In this case, the wake-up word speech should not be considered as non-wake-up speech. Therefore, the embodiment of the present invention presets a time period. During this time period, if the user performs a wake-up operation, the probability of the wake-up speech in the previous wake-up category exceeds the preset threshold but does not exceed the wake-up threshold as non-wake-up speech.

[0099] Specifically, when the probability of the wake-up speech in the wake-up category predicted by the first sample voice is greater than a preset threshold, and the preset threshold is less than the wake-up threshold, the time when the first sample voice is received is taken as the start time. Within a preset time period after the start time, if the user initiates a wake-up command and the intelligent voice wake-up system responds by performing a wake-up operation, then the first sample voice is taken as a non-wake-up voice.

[0100] It should be noted that after the first sample voice is used as the non-awakening voice, the non-awakening voice can be stored in the intelligent voice awakening system for the intelligent voice awakening system to read and call when predicting the awakening category. Among them, the non-awakening voice can be stored in the form of audio or in the form of voice features. At this time, in the case of multiple non-awakening voices or false awakening voices, they can be framed with the same duration, and the voice features of each frame can be fused. The embodiment of the present invention does not limit this.

[0101] Based on any of the above embodiments, Figure 6 FIG. 1 is a flow chart of the method for obtaining false wake-up voice provided by the present invention. Figure 6 As shown, the false wake-up voice in step 110 is determined based on the following steps:

[0102] Step 610, obtaining a second sample voice for performing a wake-up operation;

[0103] Specifically, the second sample speech comes from the speech to be classified whose wake-up category is predicted as the wake-up speech in the method in the above embodiment.

[0104] Step 620, performing semantic recognition on the subsequent speech of the second sample speech to obtain a semantic recognition result;

[0105] Step 630: If the semantic recognition result is non-semantic, determine that the second sample voice is a false wake-up voice.

[0106] Considering the case of false awakening, the subsequent speech after the wake-up word is often meaningless interactive speech irrelevant to human-computer interaction. Therefore, the embodiment of the present invention performs semantic recognition on the subsequent speech to be classified after the wake-up operation is performed.

[0107] Specifically, after the intelligent voice wake-up system predicts that the wake-up category of the second sample voice is the wake-up voice, it will perform semantic recognition on the subsequent voice after the second sample voice. If the recognition result is non-semantic, that is, it is not an instruction for human-computer interaction, the second sample voice will be regarded as a false wake-up voice.

[0108] It should be noted that semantic recognition of subsequent voices after the second sample voice can be performed in the device where the intelligent voice wake-up system is located, or the subsequent voices can be sent to the cloud for semantic recognition by the cloud. This embodiment of the present invention does not limit this.

[0109] In addition, just like the subsequent operation of the non-awakening voice, after the second sample voice is used as the false awakening voice, the false awakening voice can be stored in the intelligent voice awakening system for the intelligent voice awakening system to read and call when predicting the awakening category. Among them, the false awakening voice can be stored in the form of audio or in the form of voice features. At this time, when there are multiple non-awakening voices or false awakening voices, they can be framed with the same duration, and the voice features of each frame can be fused. The embodiment of the present invention does not limit this.

[0110] The voice wake-up method provided in the embodiment of the present invention dynamically obtains and stores the non-awakened voice and the false awakening voice, and updates the information of the auxiliary voice in real time, thereby accumulating information such as the accent, pronunciation habits and speaking speed of the user, and obtaining more and more accurate voice features of the auxiliary voice, so that the wake-up category based on the auxiliary voice assisted prediction is also more accurate, further reducing the situations of non-awakening and false awakening, thereby improving the user experience.

[0111] Figure 7 : is a flowchart of a method for obtaining a voice wake-up category based on multi-head attention provided by the present invention. Figure 7 As shown in the figure: the Source Embedding on the left is the Embedding of the speech to be classified; DropOut, Forward LSTM, ForwardLSTM, DropOut, Add and Projection Layer constitute the encoding network, where DropOut represents the information loss network, Forward LSTM represents the forward LSTM (Long Short-Term Memory) network, Add represents the feature fusion in the residual network, and Projection Layer represents the projection layer used to reduce the dimension of the feature.

[0112] The encoding network first sends the Embedding of the speech to be classified to the Dropout network to prevent overfitting due to data sparsity. Then it is processed by the forward LSTM to encode each input Embedding of the speech to be classified. After the output of the forward LSTM is obtained, the overfitting is removed and then weighted with the Embedding extracted from the original speech to be classified, and the weighted features are reduced in dimensionality to obtain the speech features of the speech to be classified.

[0113] The Multi-Head Attention layer in the middle is a multi-head attention layer. The multi-head attention layer obtains the speech features of the speech to be classified and the speech features of the auxiliary speech, and outputs the speech features of the speech to be classified and the auxiliary speech according to the speech features of the speech to be classified and the speech features of the auxiliary speech. Among them, p represents the maximum number of frames, p>1, which is used to represent the weight value of each frame in the attention weight. The weight value of each frame in the attention weight represents the importance of the frame at the corresponding position in the mixed speech feature. The importance of each frame can better distinguish the wake-up category when making predictions. The specific calculation method is as follows:

[0114] First, according to the speech feature h of the i-th frame of the current speech to be classified i The speech feature s of the i-th frame in the auxiliary speech in the decoding network i Calculate the attention value e of the i-th frame i , the specific calculation method is shown in the following formula:

[0115] e i =tanh(W h ×h i +W s ×s i +b)

[0116] Among them, W h , W s and b are model parameters, obtained by training with audio data;

[0117] Then, the weight value of the i-th frame, that is, the importance, is calculated according to the attention value of the i-th frame. The specific calculation method is shown in the following formula:

[0118]

[0119] in, is the weight value of the i-th frame, p represents the maximum number of frames, and p>1.

[0120] The Target Embedding on the right is the Embedding of the auxiliary voice. DropOut, Forward LSTM, Concat, Forward LSTM, DropOut, Add and Softmax constitute the decoding network. Among them, DropOut represents the information loss network, Forward LSTM represents the forward LSTM network, Add represents the feature fusion in the residual network, Concat represents the feature fusion of the voice features of the voice to be classified and the voice features of the auxiliary voice, and Softmax represents the calculation of the probability of each category in the wake-up category.

[0121] The decoding network extracts features from the embedding of each frame of the auxiliary speech through forward LSTM to obtain the speech features of the auxiliary speech, and fuses the features with the speech features of the speech to be classified output by the encoding layer. The fused speech features are then used to extract high-level speech features through forward LSTM, and then the high-level speech features and the fused speech features are weighted to obtain the mixed speech features of each frame. Finally, the mixed speech features of each frame are weighted according to the weight values ​​of each frame output by the attention layer to obtain the weighted speech features of each frame, and the weighted speech features of each frame are input into the Softmax layer to predict the probability of each category in the wake-up category.

[0122] The voice wake-up device provided by the present invention is described below. The voice wake-up device described below and the voice wake-up method described above can be referenced to each other.

[0123] Figure 8 Schematic diagram of the structure of the voice wake-up device provided by the present invention. Figure 8 As shown, the device includes: a determination module 810, a classification module 820 and a wake-up module 830;

[0124] in,

[0125] A determination module 810, configured to determine the speech to be classified;

[0126] A classification module 820, configured to determine the wake-up category of the speech to be classified based on the correlation between the speech features of the auxiliary speech and the speech features of the speech to be classified, and the speech features of the auxiliary speech and the speech features of the speech to be classified, wherein the auxiliary speech includes the non-wake-up speech and / or the false wake-up speech;

[0127] The wake-up module 830 is used to perform a wake-up operation when the wake-up category is a wake-up voice.

[0128] In an embodiment of the present invention, a determination module 810 is used to determine the speech to be classified; a classification module 820 is used to determine the wake-up category of the speech to be classified based on the correlation between the auxiliary speech and the speech to be classified, as well as the speech features of the auxiliary speech and the speech features of the speech to be classified, wherein the auxiliary speech includes non-awakening speech and / or false awakening speech; and a wake-up module 830 is used to perform a wake-up operation when the wake-up category is the wake-up speech, thereby realizing the introduction of auxiliary speech to assist in recognition when performing wake-up classification on the speech, improving the accuracy of identifying the wake-up category, reducing the probability of non-awakening and false awakening, and thereby improving the user experience.

[0129] Based on any of the above embodiments, the classification module 820 includes:

[0130] A weight determination submodule, used to determine the attention weight based on the correlation between the speech features of the speech to be classified and the speech features of the auxiliary speech;

[0131] A feature extraction submodule, used for performing mixed feature extraction on the speech features of the auxiliary speech and the speech features of the speech to be classified to obtain mixed speech features;

[0132] The prediction category submodule is used to determine the wake-up category of the speech to be classified based on the attention weight and the mixed speech features.

[0133] Based on any of the above embodiments, the feature extraction submodule includes:

[0134] A fusion submodule, used to fuse the speech features of the auxiliary speech with the speech features of the speech to be classified to obtain fused speech features;

[0135] A high-level feature extraction submodule is used to extract the fused speech features to obtain high-level fused features;

[0136] The weighted submodule is used to determine the mixed speech features based on the high-level fusion features and the speech features of the auxiliary speech.

[0137] Based on any of the above embodiments, the prediction category submodule includes:

[0138] A feature weighting submodule, for weighting the speech features corresponding to the positions of the frames in the attention weight in the mixed speech features based on the weight values ​​of the frames in the attention weight, to obtain the speech weighted features corresponding to the frames in the mixed speech features;

[0139] The category determination submodule is used to determine the wake-up category of the speech to be classified based on the speech weighted features corresponding to each frame in the mixed speech features.

[0140] Based on any of the above embodiments, the weight determination submodule includes:

[0141] A weight calculation submodule, for determining the attention value of each frame position based on the correlation between the speech features of each frame in the speech to be classified and the speech features in the auxiliary speech corresponding to the position of each frame in the speech to be classified;

[0142] The submodule for determining the weight of each frame is used to determine the weight value of each frame in the attention weight based on the attention value of each frame position.

[0143] Based on any of the above embodiments, the voice awakening device provided by the present invention further includes a non-awakening voice collection module, which specifically includes:

[0144] An acquisition submodule, used for acquiring a first sample voice without performing a wake-up operation;

[0145] The wake-up voice determination submodule is used to determine that the first sample voice is a non-wake-up voice if the probability that the first sample voice belongs to the wake-up category is greater than a preset threshold and there is a wake-up operation within a preset time after receiving the first sample voice.

[0146] Based on any of the above embodiments, the voice wake-up device provided by the present invention further includes a false wake-up voice collection module, which specifically includes:

[0147] An acquisition submodule, used for acquiring a second sample voice for performing a wake-up operation;

[0148] A semantic recognition submodule, used for performing semantic recognition on the subsequent speech of the second sample speech to obtain a semantic recognition result;

[0149] The wake-up speech determination submodule is used to determine that the second sample speech is a false wake-up speech if the semantic recognition result is non-semantic.

[0150] Fig. 9 An example of a physical structure diagram of an electronic device is shown in FIG. Fig. 9 As shown, the electronic device may include: a processor 910, a communication interface 920, a memory 930 and a communication bus 940, wherein the processor 910, the communication interface 920 and the memory 930 communicate with each other through the communication bus 940. The processor 910 may call the logic instructions in the memory 930 to execute the voice wake-up method, which includes: determining the voice to be classified; based on the correlation between the voice features of the auxiliary voice and the voice features of the voice to be classified, and the voice features of the auxiliary voice and the voice features of the voice to be classified, determining the wake-up category of the voice to be classified, the auxiliary voice includes the non-wake-up voice and / or the false wake-up voice; when the wake-up category is the wake-up voice, performing the wake-up operation.

[0151] In addition, the logic instructions in the above-mentioned memory 930 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0152] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the voice wake-up method provided by the above methods, which includes: determining a speech to be classified; based on the correlation between the speech features of the auxiliary speech and the speech features of the speech to be classified, as well as the speech features of the auxiliary speech and the speech features of the speech to be classified, determining the wake-up category of the speech to be classified, the auxiliary speech including non-wake-up speech and / or false wake-up speech; when the wake-up category is a wake-up speech, performing a wake-up operation.

[0153] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the voice wake-up method provided by the above-mentioned methods, the method comprising: determining a speech to be classified; determining the wake-up category of the speech to be classified based on the correlation between the speech features of the auxiliary speech and the speech features of the speech to be classified, as well as the speech features of the auxiliary speech and the speech features of the speech to be classified, the auxiliary speech including non-wake-up speech and / or false wake-up speech; when the wake-up category is a wake-up speech, performing a wake-up operation.

[0154] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0155] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A voice wake-up method, characterized in that: include: Determine the speech to be classified; Based on the correlation between the voice features of the auxiliary voice and the voice features of the voice to be classified, and the voice features of the auxiliary voice and the voice features of the voice to be classified, determine the wake-up category of the voice to be classified, the auxiliary voice includes a non-wake-up voice and / or a false wake-up voice, the non-wake-up voice is a voice that contains a correct wake-up word but fails to respond to the wake-up, and the false wake-up voice is a voice that does not issue a valid command but responds to the answer; When the wake-up category is a wake-up voice, performing a wake-up operation; The step of determining the wake-up category of the speech to be classified based on the correlation between the speech features of the auxiliary speech and the speech features of the speech to be classified, and the speech features of the auxiliary speech and the speech features of the speech to be classified, comprises: Determining an attention weight based on a correlation between the speech feature of the speech to be classified and the speech feature of the auxiliary speech; Performing mixed feature extraction on the speech features of the auxiliary speech and the speech features of the speech to be classified to obtain mixed speech features; Based on the attention weight and the mixed speech feature, a wake-up category of the speech to be classified is determined.

2. The voice wake-up method according to claim 1, characterized in that: The step of extracting mixed features from the speech features of the auxiliary speech and the speech features of the speech to be classified to obtain mixed speech features includes: Performing feature fusion on the speech features of the auxiliary speech and the speech features of the speech to be classified to obtain fused speech features; Performing feature extraction on the fused speech features to obtain high-level fused features; The mixed speech feature is determined based on the high-level fusion feature and the speech feature of the auxiliary speech.

3. The voice wake-up method according to claim 1, characterized in that: The step of determining the wake-up category of the speech to be classified based on the attention weight and the mixed speech feature includes: Based on the weight value of each frame in the attention weight, weighting the speech features in the mixed speech feature corresponding to the positions of each frame in the attention weight, to obtain the speech weighted features corresponding to each frame in the mixed speech feature; Based on the speech weighted features corresponding to each frame in the mixed speech features, the wake-up category of the speech to be classified is determined.

4. The voice wake-up method according to claim 1, characterized in that: The step of determining the attention weight based on the correlation between the speech feature of the speech to be classified and the speech feature of the auxiliary speech comprises: Determine the attention value of each frame position based on the correlation between the speech features of each frame in the speech to be classified and the speech features in the auxiliary speech corresponding to each frame position in the speech to be classified; Based on the attention value of each frame position, a weight value of each frame in the attention weight is determined.

5. The voice wake-up method according to any one of claims 1 to 4, characterized in that: The non-wake-up voice is determined based on the following steps: Acquire a first sample voice without performing a wake-up operation; If the probability that the first sample voice belongs to the wake-up category is greater than a preset threshold, and there is a wake-up operation within a preset time after the first sample voice is received, the first sample voice is determined to be a non-wake-up voice.

6. The voice wake-up method according to any one of claims 1 to 4, characterized in that: The false wake-up voice is determined based on the following steps: Acquire a second sample voice for performing a wake-up operation; Performing semantic recognition on subsequent speech of the second sample speech to obtain a semantic recognition result; If the semantic recognition result is non-semantic, it is determined that the second sample voice is a false wake-up voice.

7. A voice wake-up device, characterized in that: include: A determination module, used for determining the speech to be classified; A classification module, configured to determine the wake-up category of the voice to be classified based on the correlation between the voice features of the auxiliary voice and the voice features of the voice to be classified, and the voice features of the auxiliary voice and the voice features of the voice to be classified, wherein the auxiliary voice includes a non-wake-up voice and / or a false wake-up voice, wherein the non-wake-up voice is a voice that contains a correct wake-up word but fails to respond to the wake-up, and the false wake-up voice is a voice that does not issue a valid command but responds to the answer; A wake-up module, used for performing a wake-up operation when the wake-up category is a wake-up voice; The classification module is specifically used for: Determining an attention weight based on a correlation between the speech feature of the speech to be classified and the speech feature of the auxiliary speech; Performing mixed feature extraction on the speech features of the auxiliary speech and the speech features of the speech to be classified to obtain mixed speech features; Based on the attention weight and the mixed speech feature, a wake-up category of the speech to be classified is determined.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the voice wake-up method according to any one of claims 1 to 6 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the voice wake-up method according to any one of claims 1 to 6 are implemented.

Images