Voice wake-up method, device, and storage medium

Abstract

The present disclosure relates to the technical field of speech recognition, and provides a speech wake-up method, device and storage medium, comprising determining a character probability matrix corresponding to a plurality of to-be-processed speech frames, wherein the character probability matrix comprises a wake-up character probability value corresponding to a wake-up character; performing sliding window processing on the plurality of to-be-processed speech frames based on a preset sliding window, determining a smoothed wake-up character probability value corresponding to the wake-up character; determining a wake-up word probability value of a wake-up word composed of the wake-up character based on the fact that a first frame length determined based on at least part of the to-be-processed speech frames conforms to a preset wake-up frame length, and the smoothed wake-up character probability value of each wake-up character conforms to a preset threshold; and performing a speech wake-up operation in the case where the wake-up word probability value is greater than a preset wake-up word probability value, thereby realizing a decoding process with less calculation amount and flexibility by combining the preset sliding window, the preset wake-up frame length, the preset threshold and the preset wake-up word probability value.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of speech recognition, and in particular to a speech wake-up method, device and storage medium. BACKGROUND

[0002] Speech recognition and other technologies have been applied in multiple technical fields. In actual applications, a device equipped with a speech recognition system usually responds to a speech signal after being woken up.

[0003] There are various speech wake-up methods based on a neural network acoustic model. For example, after extracting speech features, a two-classification model that does not require decoding and a wake-up model based on decoding are included according to whether there is a back-end decoding. For the wake-up model requiring decoding, since the words of the wake-up word are fixed, the accuracy of the decoding algorithm is not high when the number of words is small and the combination is simple. Moreover, when the threshold value is adjusted, the decoding algorithm needs to be retrained, so that the sensitivity of the decoding algorithm is not high. SUMMARY

[0004] In a first aspect, an embodiment of the present disclosure provides a speech wake-up method, comprising:

[0005] determining a character probability matrix corresponding to a plurality of to-be-processed speech frames, wherein the character probability matrix comprises a wake-up character probability value corresponding to a wake-up character;

[0006] performing sliding window processing on the plurality of to-be-processed speech frames based on a preset sliding window, and determining a smoothed wake-up character probability value corresponding to the wake-up character;

[0007] determining a wake-up word probability value of a wake-up word composed of the wake-up character based on the fact that a first frame length determined based on at least part of the to-be-processed speech frames meets a preset wake-up frame length, and the smoothed wake-up character probability value of each of the wake-up characters meets a preset threshold;

[0008] performing a speech wake-up operation in a case where the wake-up word probability value is greater than a preset wake-up word probability value.

[0009] According to the speech wake-up method provided by an embodiment of the present disclosure, after the first frame length determined based on at least part of the to-be-processed speech frames meets the preset wake-up frame length, the method comprises:

[0010] determining that a second frame length determined based on at least part of the to-be-processed speech frames after the to-be-processed speech frame currently processed meets a second preset wake-up frame length.

[0011] According to the speech wake-up method provided by an embodiment of the present disclosure, the method further comprises:

[0012] The sequence of the speech frames corresponding to the maximum smoothed wake-up character probability value of each wake-up character conforms to a preset syntax sequence.

[0013] According to the voice wake-up method provided in the embodiments of the present disclosure, the determination of the wake-up word probability value of the wake-up word composed of the wake-up characters comprises:

[0014] The wake-up word probability value of the wake-up word composed of the wake-up characters is determined according to the maximum smoothed wake-up character probability value of the wake-up characters.

[0015] According to the voice wake-up method provided in the embodiments of the present disclosure, the determination of the character probability matrix corresponding to the plurality of to-be-processed speech frames comprises:

[0016] The FBANK features of the plurality of to-be-processed speech frames are input into a preset acoustic model frame by frame, and the probability values of each wake-up character output by the preset acoustic model are obtained.

[0017] According to the voice wake-up method provided in the embodiments of the present disclosure, the determination of the character probability matrix corresponding to the plurality of to-be-processed speech frames comprises:

[0018] Obtaining a plurality of initial speech frames;

[0019] Frame splicing processing is performed on the plurality of initial speech frames to obtain a plurality of spliced speech frames;

[0020] Frame skipping processing is performed on the plurality of spliced speech frames to obtain a plurality of to-be-processed speech frames.

[0021] According to the voice wake-up method provided in the embodiments of the present disclosure, the size of the preset sliding window is equal to N times the frame length of the to-be-processed speech frame, and the sliding window step of the preset sliding window is a positive integer greater than or equal to 1, wherein N is a positive integer.

[0022] According to the voice wake-up method provided in the embodiments of the present disclosure, the method further comprises:

[0023] In the case of receiving a wake-up parameter adjustment request, the corresponding wake-up parameter is adjusted, wherein the wake-up parameter comprises at least one of a preset wake-up frame length, a preset threshold and a preset wake-up word probability value.

[0024] In a second aspect, the embodiments of the present disclosure further provide a voice wake-up device, comprising:

[0025] A character unit is configured to determine a character probability matrix corresponding to a plurality of to-be-processed speech frames, wherein the character probability matrix comprises a wake-up character probability value corresponding to a wake-up character.

[0026] a sliding window unit configured to perform sliding window processing on the plurality of to-be-processed voice frames based on a preset sliding window to determine a smoothed wake-up character probability value corresponding to the wake-up character;

[0027] a first wake-up unit configured to determine a wake-up word probability value of a wake-up word composed of the wake-up character based on the first frame length determined based on at least part of the to-be-processed voice frames meeting a preset wake-up frame length and the smoothed wake-up character probability value of each of the wake-up characters meeting a preset threshold;

[0028] a second wake-up unit configured to perform a voice wake-up operation in a case where the wake-up word probability value is greater than a preset wake-up word probability value.

[0029] In a third aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor implements the voice wake-up method according to any of the above aspects when executing the program.

[0030] In a fourth aspect, a non-transitory computer-readable storage medium is provided, which stores a computer program capable of being executed by a processor to implement the voice wake-up method according to any of the above aspects. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the technical solutions of the present disclosure or the prior art, the following will briefly introduce the drawings needed in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present disclosure, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0032] Figure 1 is one of the flow diagrams of the voice wake-up method provided by one embodiment of the present disclosure;

[0033] Figure 2 is one of the scene diagrams of the voice wake-up method provided by one embodiment of the present disclosure;

[0034] Figure 3 is the second scene diagram of the voice wake-up method provided by one embodiment of the present disclosure;

[0035] Figure 4 is the third scene diagram of the voice wake-up method provided by one embodiment of the present disclosure;

[0036] Figure 5 is the second flow diagram of the voice wake-up method provided by one embodiment of the present disclosure;

[0037] Figure 6FIG. 1 is a structural schematic diagram of a voice wake-up device provided by an embodiment of the present disclosure;

[0038] Figure 7 FIG. 2 is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0039] In order to make the technical solutions and advantages of the present disclosure clearer, the technical solutions in the present disclosure will be described clearly and completely below in combination with the drawings in the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the protection scope of the present disclosure.

[0040] In actual applications, the device equipped with the voice recognition system usually responds to the voice signal after being woken up, for example, when the user issues the voice signal of "Xiao H classmate, the indoor air is not good, purify the air...", the device equipped with the voice recognition system analyzes and processes the voice signal and parses the set wake-up word "purify the air" to wake up the corresponding function.

[0041] There are various voice wake-up methods based on the neural network acoustic model at present, and the current general wake-up method adopts CTC decoding wake-up, such as CTC greedy search, CTC beam search and CTC prefix beam search. Their decoding process is equivalent to space search, and the result with the maximum probability is sought from all possible string sequences. This process needs to search for several string sequences and calculate the corresponding probability, resulting in a relatively large amount of CTC decoding calculation, and there is also a possibility that the best result cannot be found.

[0042] In addition, in actual applications, a large amount of training data must be used to train the CTC decoding process before using the CTC decoding, and the decoding result is closely related to the training effect. When facing different decoding effects, the CTC decoding process needs to be retrained, so that the sensitivity of the CTC decoding result is not convenient to control.

[0043] The present disclosure provides a voice wake-up method, the execution subject of which can be any mobile terminal or functional module in the mobile terminal that can implement the voice wake-up method. The mobile terminal mentioned in the present disclosure includes but is not limited to a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), and the like. The mobile terminal can also include a database, a server, and a terminal artificial intelligence-based service response system. The voice wake-up method provided by the present disclosure is described below by taking a mobile terminal as an example. It should be noted that the execution subject described above does not constitute a limitation on the present disclosure.

[0044] The voice wake-up method provided by an embodiment of the present disclosure will be described below in combination with Figure 1 The voice wake-up method provided by an embodiment of the present disclosure will be described below in combination with Figure 1 The flowchart of the voice wake-up method provided by an embodiment of the present disclosure includes the following steps.

[0045] Step 101: Determine a character probability matrix corresponding to a plurality of to-be-processed voice frames.

[0046] The to-be-processed voice frames can be voice frames extracted from all voice data collected in real time, or voice frames extracted from voice data collected after a preset voice activation condition is met.

[0047] For example, a voice activity detection (VAD) module is set, when human voice feature data is detected through the VAD module, it is determined that the preset voice activation condition is met, and the voice frames extracted from the voice data collected from this moment are to-be-processed voice frames.

[0048] In some embodiments, the timestamps of the plurality of to-be-processed voice frames can not have repeated parts. For example, starting from the beginning, the voice data is cut into a frame every 10 ms, that is, the timestamp of the first to-be-processed voice frame is t (ms) to t+10 (ms), the timestamp of the second to-be-processed voice frame is t+11 (ms) to t+20 (ms), and so on.

[0049] In some embodiments, the timestamps of the plurality of to-be-processed speech frames can also have repeated parts, for example, in order to avoid spectral leakage when subsequently processing the to-be-processed speech frames, a speech frame windowing manner is adopted to obtain the to-be-processed speech frames, for example, each to-be-processed speech frame has a frame length of 25 ms, of which 15 ms is historical information, that is, there are 15 ms of overlapping data between the to-be-processed speech frames, and only 10 ms of adjacent to-be-processed speech frames are moved.

[0050] In some embodiments, the to-be-processed speech frame can also be a speech frame containing valid speech data, wherein the valid speech data refers to data that can be used for wake-up operation.

[0051] For example, there can be invalid data in the collected speech data, which does not contain human voice feature data for a certain period of time. In order to reduce the amount of calculation in the decoding process, after a plurality of speech frames are sequentially extracted from the collected speech data from beginning to end, the speech frames that do not contain human voice feature data or the speech frames that have too much noise to interfere with the human voice feature data can be removed, and the remaining speech frames are reserved as to-be-processed speech frames. In this way, the number of to-be-processed speech frames is reduced, thereby reducing the amount of calculation in the subsequent decoding process.

[0052] The character probability matrix includes the wake-up character probability value corresponding to the wake-up character, and in addition, the character probability matrix also includes other predicted characters and other predicted character probability values, such as a blank label character, which will not be described here.

[0053] For example, the character probability matrix is a (T, C) matrix of the prediction result of the to-be-processed speech frame, wherein T represents the number of to-be-processed speech frames processed by the preset acoustic model, C represents the number of predicted characters set in the preset acoustic model, and each element in the matrix represents the probability distribution of each predicted character in the corresponding to-be-processed speech frame. That is, the element in the nth row and the mth column of the matrix is the probability distribution of the nth predicted character in the mth to-be-processed speech frame.

[0054] For example, 16 to-be-processed speech frames are currently obtained, and 100 predicted characters are set in the preset acoustic model. Therefore, the current character probability matrix can be represented as a (16, 100) matrix. Assuming that the element in the first row and the second column of the matrix is 0.02, it means that the probability of the first frame containing the second character is 0.02. The predicted characters include wake-up characters and other preset characters, which will not be described here.

[0055] Step 102, performing sliding window processing on the plurality of to-be-processed speech frames based on a preset sliding window to determine a smoothed wake-up character probability value corresponding to the wake-up character;

[0056] Since the frame length in the frame processing of the speech data can be relatively short, the to-be-processed speech frame can have burr, thereby affecting the wake-up result. For example, a certain to-be-processed speech frame has a peak phenomenon. Therefore, a preset sliding window is used to smooth the to-be-processed speech frame to remove the interference caused by the burr. Specifically, the preset sliding window is used to smooth the wake-up character probability value of each wake-up character in the to-be-processed speech frame.

[0057] As an implementable manner, the size of the preset sliding window is equal to N times of the frame length of the to-be-processed speech frame, and the sliding window step of the preset sliding window is a positive integer greater than or equal to 1, where N is a positive integer. It can be understood that 1 refers to the frame length of a to-be-processed speech frame, that is, the minimum sliding step of the preset sliding window is the frame length of a to-be-processed speech frame.

[0058] For example, the sliding window is taken backward, the size is 10, and the sliding window step is 1. Then the first sliding window is from the first frame to the tenth frame, which is used to determine the smoothed wake-up character probability value corresponding to the wake-up character of the first frame. The second sliding window is from the second frame to the eleventh frame, which is used to determine the smoothed wake-up character probability value corresponding to the wake-up character of the second frame.

[0059] For example, the sliding window is taken forward, the size is 10, and the step is 1. Then the first sliding window is from the first frame to the second frame, which is used to determine the smoothed wake-up character probability value corresponding to the wake-up character of the first frame. The second sliding window is from the second frame to the eighth frame, which is used to determine the smoothed wake-up character probability value corresponding to the wake-up character of the second frame.

[0060] The deviation between the smoothed wake-up character probability value of each wake-up character in the sliding window and the wake-up character probability value before processing is approximately equal, thereby eliminating the influence of the prediction result of a frame when the frame has burr and other interference factors, and eliminating the influence of the prediction result of each to-be-processed speech frame when the sliding window is processed, so that the accuracy of the subsequent speech wake-up result based on the smoothed wake-up character probability value is higher.

[0061] In step 103, the first frame length determined based on at least part of the to-be-processed speech frame meets a preset wake-up frame length, and the smoothed wake-up character probability value of each wake-up character meets a preset threshold, and the wake-up word probability value of the wake-up word composed of the wake-up characters is determined.

[0062] The first frame length refers to the frame length of the plurality of to-be-processed speech frames between the first to-be-processed speech frame on which the wake-up processing starts and the to-be-processed speech frame on which the wake-up processing is currently performed in the speech wake-up stage.

[0063] For example, Figure 2Each vertical bar represents a to-be-processed voice frame after the voice data is framed, the characters below the vertical bar represent the first frame, the second frame, the third frame, and the like, the arrow direction represents that the sliding window is taken backward, the 50th frame is a to-be-processed voice frame currently undergoing the wake-up processing, and the first frame length can be determined based on the 1st frame to the 50th frame, can be determined from a middle frame to the 50th frame, or can be determined based on other to-be-processed voice frames.

[0064] The preset wake-up frame length refers to the minimum frame length of the to-be-processed voice frames used for the wake-up processing in the current voice wake-up operation, that is, the minimum voice duration required by the user to speak the wake-up word.

[0065] Since the wake-up word usually needs to last for a specific length of time, voice shorter than the specific length can be considered as a glitch interference. For example, usually one word needs to take 100-300 ms, and for a wake-up word with four wake-up characters, the first frame length determined based on the to-be-processed voice frames corresponding to the wake-up word should be no less than 400 ms, so that the voice wake-up can be performed.

[0066] In this embodiment, only when the first frame length determined based on the frame length of at least part of the to-be-processed voice frames used for the wake-up processing meets the preset wake-up frame length, the subsequent voice wake-up operation is continued, thereby improving the voice wake-up success rate.

[0067] In an embodiment, when the determined first frame length is less than the preset wake-up frame length, it is determined that the wake-up fails, and a corresponding wake-up failure prompt is output, thereby prompting the user to re-input valid voice data.

[0068] The preset threshold is a threshold for the numerical value of the smoothed wake-up character probability value, and the preset thresholds of the multiple wake-up characters can be the same or different.

[0069] For example, the smoothed wake-up character probability values of the four wake-up characters are all greater than 0.02, or the smoothed wake-up character probability values of the first and fourth wake-up characters are greater than 0.02, and the smoothed wake-up character probability values of the second and third wake-up characters are greater than 0.015.

[0070] In some embodiments, the wake-up word probability value of the wake-up word can be determined according to all the smoothed wake-up character probability values of the wake-up characters.

[0071] For example, if the current wake-up word includes four wake-up characters, and the smoothed wake-up character probability value of each wake-up character includes 50, the average of all 200 smoothed wake-up character probability values can be taken as the wake-up word probability value of the wake-up word, the average of the 50 smoothed wake-up character probability values of each wake-up character can be taken first, and then the sum of the four averages can be taken as the wake-up word probability value of the wake-up word, or the result of multiplying the four averages and taking the fourth power can be taken as the wake-up word probability value of the wake-up word, and the like, which is not limited.

[0072] In some embodiments, in order to reduce the amount of calculation, the wake-up word probability value of the wake-up word composed of the wake-up character can also be determined according to the maximum smoothed wake-up character probability value of the wake-up character.

[0073] For example, if the current wake-up word includes four wake-up characters, the result of multiplying the maximum smoothed wake-up character probability values of the four wake-up characters and taking the fourth power can be taken as the wake-up word probability value of the wake-up word, the average corresponding to the maximum smoothed wake-up character probability values of the four wake-up characters can be taken as the wake-up word probability value of the wake-up word, and the like, which is not limited.

[0074] Step 104: In a case where the wake-up word probability value is greater than a preset wake-up word probability value, performing a voice wake-up operation.

[0075] The voice wake-up operation is an operation that needs to be performed next, which is represented by the wake-up word.

[0076] For example, when the wake-up word is "recording", the terminal starts the microphone to record, and when the wake-up word is "taking a photo", the terminal starts the camera to take a photo.

[0077] For example, Figure 3 For example, in the case where the wake-up word of the terminal is "taking a photo", the current display interface of the terminal is the home interface, and the user outputs the voice data of "taking a photo", after the terminal performs the above steps 101, 102, 103 and 104, the display interface of the terminal is switched to the photo interface.

[0078] For example, in some embodiments, when the terminal is in the sleep mode, the terminal can first enter the working mode, and then perform the operation that needs to be performed next, which is represented by the wake-up word.

[0079] The voice wake-up method proposed in this embodiment reduces the subsequent calculation amount and reduces the influence of interference factors such as burrs on the wake-up result by using a preset sliding window, and performs wake-up decoding in the subsequent decoding process by setting the preset wake-up frame length, the preset threshold and the preset wake-up word probability value. Since the above setting values can be dynamically adjusted, the model does not need to be retrained again during wake-up decoding, thereby increasing the decoding flexibility.

[0080] It should be noted that each embodiment of the present application can be freely combined, exchanged order or executed alone, and does not need to rely on or depend on the fixed execution order.

[0081] In some embodiments, the determining the character probability matrix corresponding to the plurality of to-be-processed speech frames comprises:

[0082] Obtaining a plurality of initial speech frames;

[0083] Performing frame splicing processing on the plurality of initial speech frames to obtain a plurality of spliced speech frames;

[0084] Performing frame skipping processing on the plurality of spliced speech frames to obtain a plurality of to-be-processed speech frames.

[0085] In one example, the frame splicing processing can be performed on each initial speech frame based on the frames before and after each initial speech frame, for example, "J past frames + current frame + K future frames" together as the input data of the current frame, so as to infer the predicted label of the current frame. J and K are positive integers.

[0086] In this embodiment, since one frame of feature predicts one label sometimes appears to be insufficient in information amount, especially when the pronunciation of the label is relatively long, for example, the frame length of the speech frame required when using pinyin as the label is longer than that when using phonemes as the label, therefore, frame splicing is adopted in this embodiment to overcome this defect.

[0087] In some embodiments, the spliced speech frames can have redundant information amount, and then the frame skipping sampling can be used to reduce the data amount and subsequent processing calculation amount. For example, one frame is taken out from the two continuous spliced speech frames for subsequent processing, or one frame is taken out from the three continuous spliced speech frames for subsequent processing, etc.

[0088] In one example, when "J past frames + current frame + K future frames" together as the spliced speech frame of the current frame, the frame skipping length is determined based on the J, K values in the frame splicing process, thereby reducing the information amount redundancy under the condition of ensuring that the to-be-processed speech frame after processing has sufficient information amount, so as to achieve the purpose of improving the decoding effect while reducing the number and subsequent processing calculation amount.

[0089] In some embodiments, after the first frame length determined based on at least part of the to-be-processed speech frames meets the preset wake-up frame length, the method comprises:

[0090] The second frame length determined based on at least part of the to-be-processed speech frames after the currently processed to-be-processed speech frame meets the second preset wake-up frame length.

[0091] The following will be described in combination with Figure 4 The present embodiment is described as follows, as Figure 4 shown,Figure 4 Each vertical bar represents a speech frame to be processed after the speech data is framed, the characters below the vertical bar represent the first frame, the second frame, the third frame, and so on, the arrow direction represents that the sliding window is taken backward, the preset wake-up frame length is defined as equal to the frame length of 50 frames of the speech frames to be processed, and the second preset wake-up frame length is equal to the frame length of 10 frames of the speech frames to be processed.

[0092] When the wake-up processing is performed, first, the first frame length is determined, for example, the first frame length corresponding to the first frame to the i-th frame in the current sliding window processing is determined from the first frame in the preset sliding window, when the current sliding window processing reaches the 50th frame, the determined first frame length (i.e., from the first frame to the 50th frame) meets the preset wake-up frame length, and it is determined that the subsequent wake-up processing steps can be continued.

[0093] Then, the second frame length is determined, that is, the second frame length is determined based on the speech frames to be processed after the 50th frame in the current sliding window processing, for example, the second frame length is determined from the 51st frame, when there are speech frames to be processed after the 50th frame, such as the 51st frame, the 52nd frame, the 53rd frame, and the 60th frame, it is determined that the determined second frame length (i.e., from the 51st frame to the 60th frame) meets the second preset wake-up frame length.

[0094] In this embodiment, because the speech frames to be processed may be subjected to frame splicing processing of the previous and subsequent frames, the wake-up inference result of the 50th frame (i.e., the current frame) is related to the frame length of the spliced frames, in an example, when the "J past frames + the current frame + K future frames" are used together as the spliced speech frames of the current frame, usually, the second preset wake-up frame length is equal to the frame length of the J past frames, or equal to the frame length of the K future frames, for example, the frame length of each frame is 10 ms, J = K = 5 is set during the frame splicing, and then 50 ms is continued to wait after the current sliding window processing reaches the 50th frame.

[0095] Usually, only in the case that the last wake-up word has a tail sound, the second preset wake-up frame length is added for judgment, for example, in order to prevent misjudgment of "ka" and "kang", after the frame length of the speech frame to be processed in the current processing meets the preset wake-up frame length, at least part of the speech frames to be processed after the current processing are continued to be added for secondary judgment, thereby reducing the false wake-up and improving the decoding result accuracy of the flexible decoding.

[0096] In some embodiments, the maximum smoothed wake-up word probability value of each wake-up word corresponds to a speech frame order that meets a preset speech order.

[0097] For example, a wake-up word is "Hello little coffee", which includes four wake-up characters "you", "good", "small" and "coffee". After obtaining the maximum smoothed wake-up character probability value of the four wake-up characters, and sorting the voice frames corresponding to the respective maximum smoothed wake-up character probability values, the voice wake-up operation can be performed when the time of "you", "good", "small" and "coffee" is reached, otherwise, the voice wake-up operation is not performed.

[0098] In the embodiment, the order between the voice frames corresponding to the maximum smoothed wake-up character probability values of the wake-up characters can also be detected during the wake-up process to reduce false wake-up.

[0099] As an implementable manner, reference is made to Figure 5 The voice wake-up method provided in the embodiment includes the following steps:

[0100] Step 501, determining a character probability matrix corresponding to a plurality of to-be-processed voice frames, wherein the character probability matrix includes wake-up character probability values corresponding to wake-up characters;

[0101] Step 502, performing sliding window processing on the plurality of to-be-processed voice frames based on a preset sliding window to determine smoothed wake-up character probability values corresponding to the wake-up characters;

[0102] Step 503, determining that a first frame length determined by at least part of the to-be-processed voice frames meets a preset wake-up frame length;

[0103] Step 504, determining that a second frame length determined by at least part of the to-be-processed voice frames after the to-be-processed voice frames currently processed meets a second preset wake-up frame length;

[0104] Step 505, determining that the smoothed wake-up character probability value of each wake-up character meets a preset threshold;

[0105] Step 506, determining a wake-up word probability value of a wake-up word composed of the wake-up characters;

[0106] Step 507, performing a voice wake-up operation in the case where the wake-up word probability value is greater than a preset wake-up word probability value.

[0107] It should be noted that the above steps can be freely combined, the order can be changed or the steps can be executed alone, and they do not need to rely on or depend on a fixed execution order.

[0108] The implementation manners of the steps in the embodiment are the same as those in the above embodiment, and will not be described here again.

[0109] In some embodiments, the determination of the character probability matrix corresponding to the plurality of to-be-processed voice frames includes:

[0110] The FBANK features of the plurality of to-be-processed speech frames are input into a preset acoustic model frame by frame, and probability values of each wake-up character in the plurality of to-be-processed speech frames output by the preset acoustic model are obtained.

[0111] The frequency domain feature (Filter-Bank, FBANK) of the to-be-processed speech frame can be input into the preset acoustic model, and the corresponding prediction character and the probability value thereof are output for each frame, wherein the prediction character includes the wake-up character and other characters.

[0112] In order to speed up the subsequent sliding window processing speed, after the probability values of each prediction character corresponding to each frame are output by using the preset acoustic model, a corresponding character probability matrix can be constructed, for example, for 16-frame FBANK feature data, the preset acoustic model is preset to set 100 characters, and the output matrix is [16, 100]. In actual application, in addition to the wake-up word, other characters output by the preset acoustic model are all blank labels.

[0113] In the embodiment, the preset acoustic model is a model for realizing the mapping of the FBANK feature to the character, for example, a neural network model based on CNN, DFSMN, LSTM or transformer, which is not limited.

[0114] It should be noted that in the process of feature extraction, model prediction and decoding of the speech data, the speech data in a time length is divided into frames in the feature extraction stage, and a plurality of to-be-processed speech frames at time nodes are obtained, for example, the speech data is 160 ms long, which is divided into 16 frames, that is, the first to-be-processed speech frame from the beginning to the 10th ms, the second to-be-processed speech frame from the 11th ms to the 20th ms, and the 16th to-be-processed speech frame from the 159th ms to the 160th ms. In this process, each to-be-processed speech frame is sequentially input into the above-mentioned model, and the probability distribution of each prediction character in the to-be-processed speech frame corresponding to each time node is sequentially output by the above-mentioned model.

[0115] Each horizontal line in the character probability matrix represents the probability value of each preset prediction character in the to-be-processed speech frame at each time node, and each vertical line represents the probability value of each prediction character corresponding to the to-be-processed speech frame at each time node.

[0116] Wherein, since the predicted character result is preset, the number of vertical rows of the character probability matrix in the embodiment is fixed, and since the speech data is endless, the number of horizontal rows of the character probability matrix gradually increases, that is, the first vertical row in the character probability matrix is the probability value of each predicted character corresponding to the to-be-processed speech frame at the first time node, the second vertical row is the probability value of each predicted character corresponding to the to-be-processed speech frame at the second time node, and so on, that is, in the character probability matrix, the left of each to-be-processed speech frame is the to-be-processed speech frame at the past time node corresponding thereto, and the right is the to-be-processed speech frame at the future time node corresponding thereto.

[0117] In some embodiments, in the case of receiving a wake-up parameter adjustment request, the corresponding wake-up parameter is adjusted, wherein the wake-up parameter includes at least one of a preset wake-up frame length, a preset threshold and a preset wake-up word probability value.

[0118] Wherein, the wake-up parameter adjustment request can be input, or generated by other hardware devices or transmitted by software.

[0119] The input, for example, can be input by touch input, voice input, gesture input, visual input, brain-computer input, etc.

[0120] The hardware device generation can be generated by the terminal being set to generate under certain conditions, for example, the terminal automatically generates a wake-up parameter adjustment request when detecting that the number of voice wake-up failures reaches the upper limit.

[0121] In the embodiment, the parameter functions of the above various wake-up condition parameters are consistent with the above embodiments, and will not be described here.

[0122] In the embodiment, during the voice wake-up process, the corresponding wake-up parameters can be flexibly adjusted according to actual needs, so that different voice wake-up effects can be achieved by adjusting the corresponding wake-up parameters without retraining the model, and the decoding process is more flexible.

[0123] The voice wake-up device provided by the present disclosure is described below, and the voice wake-up device described below can be mutually corresponding with the voice wake-up method described above.

[0124] Reference Figure 6 , Figure 6 is a structural schematic diagram of the voice wake-up device provided by the present disclosure, as Figure 6As shown, the voice wake-up apparatus comprises: a character unit 610, configured to determine a character probability matrix corresponding to a plurality of to-be-processed voice frames, wherein the character probability matrix comprises a wake-up character probability value corresponding to a wake-up character; a sliding window unit 620, configured to perform sliding window processing on the plurality of to-be-processed voice frames based on a preset sliding window, and determine a smoothed wake-up character probability value corresponding to the wake-up character; a first wake-up unit 630, configured to determine a wake-up word probability value of a wake-up word composed of the wake-up character based on the fact that a first frame length determined based on at least part of the to-be-processed voice frames meets a preset wake-up frame length, and the smoothed wake-up character probability value of each of the wake-up characters meets a preset threshold; and a second wake-up unit 640, configured to perform a voice wake-up operation in a case where the wake-up word probability value is greater than a preset wake-up word probability value.

[0125] In some embodiments, the first wake-up unit is further configured to determine a second frame length based on at least part of the to-be-processed voice frames after the currently processed to-be-processed voice frame, and the second frame length meets a second preset wake-up frame length.

[0126] In some embodiments, the first wake-up unit is further configured to determine that a sequence of voice frames corresponding to a maximum smoothed wake-up character probability value of each of the wake-up characters meets a preset sequence.

[0127] In some embodiments, the first wake-up unit is further configured to determine the wake-up word probability value of the wake-up word composed of the wake-up character according to the maximum smoothed wake-up character probability value of the wake-up character.

[0128] In some embodiments, the character unit is further configured to input FBANK features of the plurality of to-be-processed voice frames into a preset acoustic model frame by frame, and obtain probability values of each wake-up character output by the preset acoustic model.

[0129] In some embodiments, the character unit is further configured to obtain a plurality of initial voice frames, perform frame splicing processing on the plurality of initial voice frames to obtain a plurality of spliced voice frames, and perform frame skipping processing on the plurality of spliced voice frames to obtain the plurality of to-be-processed voice frames.

[0130] In some embodiments, a size of the preset sliding window is equal to N times of the to-be-processed voice frame, and a sliding window step of the preset sliding window is a positive integer greater than or equal to 1, wherein N is a positive integer.

[0131] In some embodiments, the second wake-up unit is further configured to adjust corresponding wake-up parameters in a case where a wake-up parameter adjustment request is received, wherein the wake-up parameters comprise at least one of a preset wake-up frame length, a preset threshold, and a preset wake-up word probability value.

[0132] The voice wake-up device provided by the embodiment reduces the influence of subsequent calculation amount and interference factors such as burrs on the wake-up result by using a preset sliding window, and performs wake-up decoding in the subsequent decoding process by setting a preset wake-up frame length, a preset threshold and a preset wake-up word probability value. Since the above setting values can be dynamically adjusted, the model does not need to be retrained during wake-up decoding, thereby increasing the decoding flexibility.

[0133] Figure 7 An example of an entity structure diagram of an electronic device is shown in Figure 7 As shown, the electronic device can include a processor 710, a communications interface 720, a memory 730 and a communications bus 740, wherein the processor 710, the communications interface 720 and the memory 730 complete mutual communication through the communications bus 740. The processor 710 can invoke a logical instruction in the memory 730 to execute a voice wake-up method, which includes determining a character probability matrix corresponding to a plurality of to-be-processed voice frames, wherein the character probability matrix includes a wake-up character probability value corresponding to a wake-up character; performing sliding window processing on the plurality of to-be-processed voice frames based on a preset sliding window to determine a smoothed wake-up character probability value corresponding to the wake-up character; determining a wake-up word probability value of a wake-up word composed of the wake-up character based on a first frame length determined based on at least part of the to-be-processed voice frames meeting a preset wake-up frame length and the smoothed wake-up character probability value of each of the wake-up characters meeting a preset threshold; and performing a voice wake-up operation in a case where the wake-up word probability value is greater than a preset wake-up word probability value.

[0134] In addition, the logical instruction in the memory 730 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present disclosure essentially or partly contribute to the prior art, or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, includes several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present disclosure. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0135] In another aspect, the present disclosure also provides a computer program product comprising a computer program, which can be stored in a non-transitory computer-readable storage medium, and the computer program is executable by a processor to enable the computer to perform the voice wake-up method provided by the above method, which comprises: determining a character probability matrix corresponding to a plurality of to-be-processed voice frames, wherein the character probability matrix comprises a wake-up character probability value corresponding to a wake-up character; performing sliding window processing on the plurality of to-be-processed voice frames based on a preset sliding window to determine a smoothed wake-up character probability value corresponding to the wake-up character; determining a wake-up word probability value of a wake-up word composed of the wake-up character based on that a first frame length determined based on at least part of the to-be-processed voice frames meets a preset wake-up frame length, and the smoothed wake-up character probability value of each of the wake-up characters meets a preset threshold; and performing a voice wake-up operation in a case where the wake-up word probability value is greater than a preset wake-up word probability value.

[0136] In yet another aspect, the present disclosure also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is executable by a processor to implement the voice wake-up method provided by the above method, which comprises: determining a character probability matrix corresponding to a plurality of to-be-processed voice frames, wherein the character probability matrix comprises a wake-up character probability value corresponding to a wake-up character; performing sliding window processing on the plurality of to-be-processed voice frames based on a preset sliding window to determine a smoothed wake-up character probability value corresponding to the wake-up character; determining a wake-up word probability value of a wake-up word composed of the wake-up character based on that a first frame length determined based on at least part of the to-be-processed voice frames meets a preset wake-up frame length, and the smoothed wake-up character probability value of each of the wake-up characters meets a preset threshold; and performing a voice wake-up operation in a case where the wake-up word probability value is greater than a preset wake-up word probability value.

[0137] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the present embodiment scheme according to actual needs. Those skilled in the art can understand and implement it without creative labor.

[0138] Those skilled in the art can clearly understand the implementation of the various embodiments by means of software and the necessary general hardware platform from the above description of the embodiments, and of course, the embodiments can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part of the prior art that makes a contribution, can be embodied in the form of a software product. The computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, and the like, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0139] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present disclosure, and not to limit them; although the present disclosure has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present disclosure.

Claims

1. A voice wake-up method, characterized by, The method comprises the following steps: determining a character probability matrix corresponding to a plurality of to-be-processed speech frames, wherein the character probability matrix comprises a wake-up character probability value corresponding to a wake-up character; performing sliding window processing on the plurality of to-be-processed speech frames based on a preset sliding window to determine a smoothed wake-up character probability value corresponding to the wake-up character; determining a wake-up word probability value of a wake-up word composed of the wake-up character based on the fact that a first frame length determined based on at least part of the to-be-processed speech frames meets a preset wake-up frame length and the smoothed wake-up character probability value of each of the wake-up characters meets a preset threshold; performing a speech wake-up operation when the wake-up word probability value is greater than a preset wake-up word probability value; wherein the order of the speech frames corresponding to the maximum smoothed wake-up character probability value of each of the wake-up characters meets a preset word order.

2. The voice wake-up method of claim 1, wherein, After the first frame length determined based on at least part of the to-be-processed speech frames meets the preset wake-up frame length, the method further comprises the following steps: determining a second frame length based on at least part of the to-be-processed speech frames after the currently processed to-be-processed speech frames, wherein the second frame length meets a second preset wake-up frame length.

3. The voice wake-up method of claim 1, wherein, The method further comprises the following steps for determining the wake-up word probability value of the wake-up word composed of the wake-up character: determining the wake-up word probability value of the wake-up word composed of the wake-up character based on the maximum smoothed wake-up character probability value of the wake-up character.

4. The voice wake-up method of claim 1, wherein, The method further comprises the following steps for determining the character probability matrix corresponding to the plurality of to-be-processed speech frames: inputting the FBANK features of the plurality of to-be-processed speech frames into a preset acoustic model frame by frame to obtain the probability values of each wake-up character output by the preset acoustic model.

5. The voice wake-up method of claim 1, wherein, The method further comprises the following steps for determining the character probability matrix corresponding to the plurality of to-be-processed speech frames: obtaining a plurality of initial speech frames; performing frame splicing processing on the plurality of initial speech frames to obtain a plurality of spliced speech frames; performing frame skipping processing on the plurality of spliced speech frames to obtain a plurality of to-be-processed speech frames.

6. The voice wake-up method of claim 1, wherein, The size of the preset sliding window is N times the frame length of the to-be-processed speech frames, and the sliding window step of the preset sliding window is a positive integer greater than or equal to 1, wherein N is a positive integer.

7. The voice wake-up method of any one of claims 1 to 6, wherein, The method further comprises the following steps: adjusting the corresponding wake-up parameters when a wake-up parameter adjustment request is received, wherein the wake-up parameters comprise at least one of the preset wake-up frame length, the preset threshold, and the preset wake-up word probability value.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the speech wake-up method according to any one of claims 1 to 7. 9.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the speech wake-up method according to any one of claims 1 to 7.

Images