Interactive Voice Recognition Method, Recognition Chip, Interactive Device and Medium

Abstract

The present invention provides an interactive voice recognition method, a recognition chip, an interactive device and a medium. The method includes: dividing audio data corresponding to interactive voice information into multiple frames of voice signals, and based on a CTC acoustic model, determining probability values corresponding to each frame of voice signal on multiple tags; judging whether the probability value of the voice signal on the blank tag is greater than or equal to a preset threshold; if so, identifying the probability value of the currently judged voice signal on the blank tag as its final probability value; if not, updating the probability value of the currently judged voice signal on the current non-blank tag by the probability value of the previous voice signal on the current non-blank tag, and based on the updated probability values, determining the final probability value of the currently judged voice signal; forming a voice tag sequence with the tags corresponding to the final probability values of each voice signal, and generating a recognition result of the voice data from the voice tag sequence. In this way, accurate and efficient recognition of interactive voice information is achieved.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and in particular to an interactive speech recognition method, a recognition chip, an interactive device and a medium. Background Art

[0002] With the development of science and technology, the popularity of intelligent interactive devices is increasing. In intelligent interactive devices, interactive speech recognition is one of the key ways to achieve human-computer interaction; a speech is initiated to an intelligent interactive device, and the intelligent interactive device recognizes and responds to the speech to achieve human-computer interaction. At present, interactive speech recognition is usually implemented based on CTC (Connectionist Temporal Classification) acoustic model decoding; and the decoding algorithms of the CTC model mainly include greedy search, beam search and prefix search.

[0003] Among them, the greedy search selects the output value with the highest probability at each step in the calculation until a terminal symbol or the maximum sentence length appears, and the final decoding sequence is obtained. Beam search is a heuristic graph search algorithm. When each step of depth expansion is performed, some nodes with relatively poor quality are removed, and nodes with higher quality are retained to obtain a series of decoding paths. Then, the paths with the same decoding label sequence in a series of decoding paths are merged, and the label sequence with the highest probability is selected as the final decoding sequence. Prefix beam search is an improved algorithm based on beam search. Compared with beam search, more useful nodes are retained when cutting off nodes with poor quality to merge and obtain the final decoding sequence.

[0004] However, for greedy search, each step selects the maximum value of the output probability, which only involves a single path and only uses the information of the current step, resulting in poor decoding results and inaccurate speech recognition. Although beam search and prefix beam search can find several paths, the algorithm has a large amount of computation and takes a long time, resulting in low speech recognition efficiency. At the same time, during the training process of the CTC ascending model, the training set usually contains a large number of blank labels, so that no matter whether the decoding adopts greedy search, beam search or prefix search, the output probability will be biased towards blank labels, so that the blank labels obscure the real labels, resulting in inaccurate decoding results.

[0005] Therefore, how to perform interactive speech recognition accurately and efficiently is a technical problem that needs to be solved urgently. Summary of the invention

[0006] Based on the above situation, the main purpose of the present invention is to provide an interactive speech recognition method, a recognition chip, an interactive device and a medium to accurately and efficiently perform interactive speech recognition.

[0007] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0008] An interactive speech recognition method based on a CTC acoustic model, the interactive speech recognition method comprising the following steps:

[0009] S100, when detecting interactive speech information, dividing the audio data corresponding to the speech information into multiple frames of speech signals having a time sequence relationship;

[0010] S200, based on the CTC acoustic model, determining the probability values respectively corresponding to multiple labels in the CTC acoustic model for each frame of the speech signal, wherein the multiple labels include a blank label and multiple non - blank labels;

[0011] S300, according to the time sequence relationship, determining one by one whether the probability value of the speech signal on the blank label is greater than or equal to a preset threshold:

[0012] If the probability value of the currently judged speech signal on the blank label is greater than or equal to the preset threshold, then execute step S400; if the probability value of the currently judged speech signal on the blank label is less than the preset threshold, then execute step S500;

[0013] S400, identifying the probability value on the blank label as the final probability value of the currently judged speech signal;

[0014] S500, judging whether the probability value of the previous speech signal in time sequence on the current non - blank label is the final probability value of the previous speech signal, and updating the probability value of the currently judged speech signal on the current non - blank label according to the judgment result;

[0015] S600, based on the magnitude relationship between the updated probability values of the currently judged speech signal on each non - blank label, determining the final probability value of the currently judged speech signal;

[0016] S700, searching for the labels respectively corresponding to the final probability values of each frame of the speech signal, generating the searched labels into a speech label sequence, and generating the recognition result of the audio data based on the semantics corresponding to each label.

[0017] Preferably, step S500 includes:

[0018] Judging whether the probability value of the previous speech signal in time sequence on the current non - blank label is the final probability value of the previous speech signal:

[0019] If it is the final probability value of the previous speech signal, then keep the probability value of the currently judged speech signal on the current non - blank label unchanged;

[0020] If it is not the final probability value of the previous voice signal, then perform an addition operation on the probability value of the previous voice signal on the current non-blank label and the probability value of the currently judged voice signal on the current non-blank label, and use the operation result of the addition operation to update the probability value of the currently judged voice signal on the current non-blank label.

[0021] Preferably, before step S500, it includes:

[0022] Determine whether the currently judged voice signal is the voice signal with the earliest time;

[0023] If it is the voice signal with the earliest time, then determine the final probability value of the currently judged voice signal according to the probability values respectively corresponding to the currently judged voice signal on each label;

[0024] If the currently judged voice signal is not the voice signal with the earliest time, then execute step S500.

[0025] Preferably, the step S600 includes:

[0026] Compare the updated probability values of the currently judged voice signal on each non-blank label to determine the target probability value with the largest value;

[0027] Identify the target probability value as the final probability value of the currently judged voice signal.

[0028] Preferably, after the step S600, it includes:

[0029] Determine the voice signal in the next order after the currently judged voice signal according to the time sequence relationship;

[0030] Set the currently judged voice signal as the previous voice signal with an earlier time, and set the voice signal in the next order as the new currently judged voice signal, and execute the step of judging whether the probability value of the voice signal on the blank label is greater than or equal to the preset threshold.

[0031] Preferably, in step S700, the step of generating the found label into a voice label sequence includes:

[0032] Determine the identifier corresponding to the found label, and arrange the identifiers according to the time sequence relationship to generate the voice label sequence.

[0033] Preferably, after the step S700, it includes:

[0034] Determine the control instruction corresponding to the recognition result and execute the control instruction.

[0035] Preferably, before the step S100, it includes:

[0036] When receiving voice information, it is recognized whether a wake-up word is included in the voice information;

[0037] If a wake-up word is included, it is determined that interactive voice information is detected.

[0038] Preferably, before the step S200, it includes:

[0039] Receive voice training samples, and train an initial CTC acoustic model based on the voice training samples to generate the CTC acoustic model.

[0040] The present invention also provides a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned interactive voice recognition method is implemented.

[0041] The present invention also provides an interactive voice recognition chip, which can implement the above-mentioned interactive voice recognition method when performing interactive voice recognition.

[0042] The present invention also provides an intelligent interaction device, which has the above-mentioned chip.

[0043] Preferably, the intelligent interaction device is a smart speaker or earphone.

[0044]

Beneficial effects

[0045] When detecting an interactive voice message, which indicates a need for voice interaction, first divide the voice data carried in the voice message into multiple frames of voice signals with a time sequence; then, through a CTC acoustic model, determine the probability values corresponding to each frame of voice signal on multiple tags in the CTC acoustic model, where the tags are semantic identifiers, including blank tags and non-blank tags. The probability values of each frame of voice signal on each preset tag reflect the likelihood of each frame of voice signal expressing various semantics; then, according to the time sequence, successively determine whether the probability value of the voice signal on the blank tag is greater than or equal to a preset threshold; if the probability value of the currently judged voice signal on the blank tag is greater than or equal to the preset threshold, then identify the probability value on the blank tag as the final probability value of the currently judged voice signal; conversely, if the probability value of the currently judged voice signal on the blank tag is less than the preset threshold, then determine whether the probability value of the previous voice signal with an earlier time on the current non-blank tag is the final probability value of the previous voice signal, and update the probability value of the currently judged voice signal on the current non-blank tag according to the judgment result; thereafter, based on the magnitude relationship between the updated probability values of the currently judged voice signal on each non-blank tag, determine the final probability value of the currently judged voice signal, and find the tags corresponding to the final probability values of each frame of voice signal respectively, generate a voice tag sequence, and then, based on the semantics corresponding to each tag, generate the recognition result of the audio data from the voice tag sequence, so as to realize the recognition of the interactive voice message.

[0046] In this way, through the magnitude relationship between the probability value of the voice signal on the blank tag and the preset threshold, and the probability value of the previous voice signal on the non-blank tag, the probability value of the currently judged voice signal on the non-blank tag is updated to obtain the final probability value of each frame of signal and generate a voice tag sequence. By judging the preset threshold on the blank tag, the blank tag and the non-blank tag can be accurately distinguished, avoiding the output of false blank tags that obscure the true tags, which is beneficial to the accuracy of the decoding result. At the same time, the probability value of the currently judged voice signal on the non-blank tag is related to the probability value of the previous voice signal, avoiding the inaccuracy of the voice recognition result caused by the independence of a single path and the information of each step; and only updating the probability value of the currently judged voice signal by combining the probability value of the previous voice signal avoids the calculation of multiple paths and improves the calculation efficiency. Therefore, by using the simplified operation of a single path to determine the voice tag sequence, the effect of improving the recognition accuracy by means of the probability information of multiple paths is achieved, and accurate and efficient voice recognition is realized through the CTC acoustic model.

[0047] Other beneficial effects of the present invention will be described in the specific implementation manners through the introduction of specific technical features and technical solutions. Those skilled in the art should be able to understand the beneficial technical effects brought by the technical features and technical solutions through the introduction of these technical features and technical solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The preferred embodiments of the speech recognition according to the present invention will be described below with reference to the drawings. In the drawings:

[0049] Figure 1 is a flowchart of an embodiment of an interactive speech recognition method based on the CTC acoustic model of the present invention;

[0050] Figure 2 is a schematic diagram of the numerical comparison before and after the update of the probability values of each frame of speech signal on each preset label in the embodiment of the present invention;

[0051] Figure 3 is Figure 2 a schematic diagram of the probability value identification symbols of each frame of speech signal on each preset label in. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0052] It should be noted that in the present invention, step numbers (letter or number numbers) are used to refer to certain specific method steps, only for the purpose of convenient and concise description, and by no means to limit the order of these method steps with letters or numbers. Those skilled in the art can understand that the order of the relevant method steps should be determined by the technology itself and should not be unduly restricted by the existence of step numbers.

[0053] Figure 1 is a flowchart of an embodiment of an interactive speech recognition method based on the CTC acoustic model of the present invention, and the method includes the following steps.

[0054] Step S100, when detecting interactive speech information, divide the audio data corresponding to the speech information into multiple frames of speech signals having a time sequence relationship.

[0055] The interactive speech recognition method based on the CTC acoustic model in this embodiment is applied to an intelligent interaction device. The intelligent interaction device is a device that recognizes speech commands in audio data and executes the speech commands, such as an intelligent speaker that recognizes speech commands such as pause play or shutdown and executes such commands, or a headset that recognizes speech commands such as previous or next and executes such commands. Specifically, when the intelligent interaction device detects interactive speech information in its environment, it processes the audio data carried in the interactive speech information and divides the audio data into multiple frames of speech signals having a time sequence relationship. Among them, each frame of speech signal is an audio frame, and the time sequence relationship between each frame of audio signal reflects the time sequence of the generation of the audio data in the speech information.

[0056] Considering that there may be noises emitted by non - humans in the environment where the intelligent interaction device is located, or the voice in its environment is not emitted for the intelligent interaction device; therefore, in this embodiment, a screening and judgment mechanism is set for whether the detected voice information is valid interactive voice information. Specifically, before step S100, it includes:

[0057] Step a1, when receiving voice information, identify whether the voice information contains a wake - up word;

[0058] Step a2, if it contains a wake - up word, then determine that interactive voice information is detected.

[0059] Furthermore, when the intelligent interaction device detects that there is voice information in its environment, based on the difference between the human voice frequency range and the non - human voice frequency range, determine whether the voice information is emitted by a human. If it is not emitted by a human, then determine that the voice information is invalid and do not process it. If it is emitted by a human, then continue to judge whether the voice information contains a wake - up word. Among them, the wake - up word is a word pre - set to wake up the intelligent interaction device, which can be the name, model number, serial number, etc. of the intelligent interaction device. If the wake - up word is identified in the voice information, it indicates that the voice information is emitted for the intelligent interaction device; thus, determine that the voice information is valid, determine the voice information as the detected interactive voice information, process the audio data carried therein, and divide the audio data into multiple frames of voice signals with a time - sequence relationship. On the contrary, if the wake - up word is not identified in the voice information, it indicates that the voice information is not emitted for the intelligent interaction device; thus, determine that the voice information is invalid and do not process it.

[0060] Step S200, based on the CTC acoustic model, determine the probability values respectively corresponding to each frame of the voice signal on multiple labels in the CTC acoustic model, where the multiple labels include a blank label and multiple non - blank labels.

[0061] Furthermore, the CTC acoustic model is pre - trained from an initial CTC acoustic model through a large number of received voice training samples, and contains multiple labels. The multiple preset labels are divided into a blank label and multiple categories of non - blank labels, and different semantics are represented by the multiple categories of non - blank labels. After dividing the audio data into multiple frames of voice signals, call the CTC acoustic model to predict each frame of voice signal and determine the probability values respectively corresponding to each frame of voice signal on its multiple labels. Through the probability values of each frame of voice signal on each label, the possibility of each frame of voice signal expressing each semantics is reflected.

[0062] Please refer to Figure 2 , Figure 2The blank label, the first label, the second label, and the third label are labels in the CTC acoustic model; the blank label is a blank label, and the first label, the second label, and the third label are non - blank labels. The first frame, the second frame, the third frame, the fourth frame, and the fifth frame are five frames of speech signals obtained by dividing the audio data. For the first frame, after prediction by the CTC acoustic model, the probability values on the blank label, the first label, the second label, and the third label are 85%, 15%, 0, and 0 respectively. For the second frame, the third frame, the fourth frame, and the fifth frame, predictions are also made according to the CTC acoustic model to obtain the probability values on each label. The probability values on each label reflect the likelihood of the audio signal falling into each label, and the semantics expressed by each frame of the audio signal are reflected by the likelihoods of each label.

[0063] Step S300, according to the chronological relationship, determine one by one whether the probability value of the speech signal on the blank label is greater than or equal to a preset threshold:

[0064] If the probability value of the currently judged speech signal on the blank label is greater than or equal to the preset threshold, then execute step S400; if the probability value of the currently judged speech signal on the blank label is less than the preset threshold, then execute step S500;

[0065] Furthermore, after obtaining the probability values of each frame of speech signal on multiple labels, the probability values of each frame of speech signal on the blank label are judged one by one according to the chronological relationship to determine whether the probability value of each frame of speech signal on the blank label is greater than or equal to the preset threshold. Among them, the preset threshold is set in advance and represents the likelihood of the semantics expressed by the speech signal falling into the blank label. First, compare the probability value of the speech signal with an earlier time sequence on the blank label with the preset threshold to judge whether its probability value on the blank label is greater than or equal to the preset threshold. In this way, after obtaining the judgment result and processing the probability values of the speech signal with an earlier time sequence on non - blank labels, then based on the judgment and processing results of the speech signal with an earlier time sequence, make a comparison judgment on the probability value of the speech signal with a later time sequence on the blank label, and so on in a loop until all frames of speech signals are processed.

[0066] It should be noted that in the process of judging the probability values of speech signals on the blank label one by one according to the chronological relationship, the speech signal being judged at the current time is determined as the currently judged speech signal. If the probability value of the currently judged speech signal on the blank label is compared with the preset threshold and it is determined that the probability value of the currently judged speech signal on the blank label is greater than or equal to the preset threshold, then execute step S400; conversely, if it is determined that the probability value of the currently judged speech signal on the blank label is less than the preset threshold, then execute step S500.

[0067] Step S400, identify the probability value on the blank label as the final probability value of the currently judged speech signal.

[0068] Furthermore, if it is determined that the probability value of the currently judged speech signal on the blank label is greater than or equal to the preset threshold, it indicates that the semantics expressed by the currently judged speech signal are more likely to fall into the blank label, and the speech signal is a blank signal without expressing semantics; therefore, identify the probability value on the blank label as the final probability value of the currently judged speech signal, and stop processing the probability value of the currently judged speech signal on non-blank labels. Among them, the final probability value characterizes the label type into which the speech signal is most likely to fall. Identifying the probability value of the speech signal on the blank label as the final probability value indicates that the speech signal is most likely to fall into the blank label and is a blank signal without representing semantics.

[0069] Step S500, determine whether the probability value of the previous speech signal with an earlier time on the current non-blank label is the final probability value of the previous speech signal, and update the probability value of the currently judged speech signal on the current non-blank label according to the judgment result.

[0070] Even further, if it is determined that the probability value of the currently judged speech signal on the blank label is less than the preset threshold, it indicates that the semantics expressed by the currently judged speech signal are less likely to fall into the blank label, and the semantics it expresses should fall into one of the non-empty labels. At this time, process and update each non-blank label of the currently judged speech signal one by one, and set the non-blank label processed at the current moment as the current non-blank label. Specifically, update the probability value of the currently judged speech signal on the current non-blank label according to whether the probability value of the speech signal with an earlier time on the current non-empty label is its final probability value. And, the speech signal with an earlier time is the previous speech signal having an adjacent relationship with the currently judged speech signal, and there is the same preset label between the updated probability values; that is, for the probability value of the currently judged speech signal on a certain non-blank label, update it according to whether the probability value of the adjacent previous speech signal on the same non-blank label is the final probability value of the previous speech signal.

[0071] Furthermore, step S500 includes:

[0072] Step b1, determine whether the probability value of the previous speech signal with an earlier time on the current non-blank label is the final probability value of the previous speech signal;

[0073] Step b2, if it is the final probability value of the previous speech signal, then keep the probability value of the currently judged speech signal on the current non-blank label unchanged;

[0074] Step b3: If it is not the final probability value of the previous voice signal, perform an addition operation on the probability value of the previous voice signal on the current non-blank label and the probability value of the currently judged voice signal on the current non-blank label, and use the operation result of the addition operation to update the probability value of the currently judged voice signal on the current non-blank label.

[0075] Furthermore, when updating the probability value of the currently judged voice signal on the current non-blank label, first determine whether the probability value of the previous voice signal on the current non-blank label that is earlier in time is the final probability value of the previous voice signal. If it is the final probability value, keep the probability value of the currently judged voice signal unchanged and do not update the probability value of the currently judged voice signal on the current non-blank label through the probability value of the previous voice signal on the current non-blank label. Among them, the final probability value of the previous voice signal is the maximum value among the probability values of each non-blank label after the probability values of this frame of voice signal on each non-blank label are updated.

[0076] Further, if it is determined that the probability value of the previous voice signal on the current non-blank label is not its final probability value, add the initial probability value of the previous voice signal on the current non-blank label and the probability value of the currently judged voice signal on the current non-blank label to obtain the operation result of their addition. And use this operation result to replace the probability value of the currently judged voice signal on the current non-blank label, so as to update the probability value of the currently judged voice signal on the current non-blank label. The probability values of the currently judged voice signal on each non-blank label are updated in this way to achieve the update of the probability values of the currently judged voice signal on each non-blank label.

[0077] It should be noted that the probability value of the previous voice signal on the current non-blank label used for update is its initial probability value before update, that is, the initial probability value predicted by the CTC acoustic model. For example, for Figure 2 the adjacent fourth frame of voice signal and the fifth frame of voice signal, when using the probability value of the fourth frame on the non-blank label to update the probability value of the fifth frame on the same non-blank label, the probability value of the fourth frame on this non-blank label is the initial probability value predicted by the CTC acoustic model, rather than the probability value after being updated by the probability value of the third frame on this non-blank label. And use the initial probability values of the fourth frame on the first label, the second label, and the third label respectively to update the probability values of the fifth frame on the first label, the second label, and the third label.

[0078] Further, the probability value of the current judged speech signal on the non-blank label is updated with the probability value of the previous speech signal on the non-blank label. If the probability values of the current judged speech signal on each non-blank label are all updated, then the probability values of the subsequent frame of speech signal on each non-blank label are updated with the probability values of the current judged speech signal on each non-blank label before updating, until the probability values of the speech signals at the end of time on each non-blank label are updated, indicating that the probability values of the speech signals with time sequence are all updated.

[0079] Specifically Figure 2 for the five frames of speech signals in Figure 3 shown. If the preset threshold is set to 80%, the probability values of the first frame of speech signal on the blank label, the first label, the second label, and the third label are represented by P01, P11, P21, and P31 respectively, the probability values of the second frame of speech signal on the blank label, the first label, the second label, and the third label are represented by P02, P12, P22, and P32 respectively, the probability values of the third frame of speech signal on the blank label, the first label, the second label, and the third label are represented by P03, P13, P23, and P33 respectively, the probability values of the fourth frame of speech signal on the blank label, the first label, the second label, and the third label are represented by P04, P14, P24, and P3 respectively, and the probability values of the fifth frame of speech signal on the blank label, the first label, the second label, and the third label are represented by P05, P15, P25, and P35 respectively, as specifically

[0080] Among them, for the first frame of speech signal, since its probability value on the blank label is greater than the preset threshold, its final probability value is determined to be the probability value on the blank label. For the second frame of speech signal, since its probability value on the blank label is less than the preset threshold, based on the probability values of the first frame of speech signal on each non - blank label, the probability values of the second frame of speech signal on each non - blank label are updated. And when updating each non - blank label one by one, first judge whether the probability value of the first frame of speech signal on the current non - blank label is its final probability value. If it is the final probability value, then the probability value of the second frame of speech signal on the current non - blank label is not updated. If it is not the final probability value, then add the probability value of the second frame of speech signal on the current non - blank label and the probability value of the first frame of speech signal on the current non - blank label, and replace the probability value of the second frame of speech signal on the current non - blank label with the added result to achieve the update. Specifically, since the final probability value of the first frame of speech signal is the probability value of the blank label, it can be determined that when updating the probability values of the second frame of speech signal on each non - blank label, the probability values of the first frame of speech signal on each non - blank label are not the final probability values. Therefore, the probability values of the second frame of speech signal on each non - blank label are updated by addition. That is, add the probability values of the first frame of speech signal on each non - blank label and the probability values of the second frame of speech signal on each non - blank label, and update the probability values of the second frame of speech signal with the added results. The specific addition process is as follows:

[0081] First label: P’12 = P11 + P12 = 15% + 20% = 35%;

[0082] Second label: P’22 = P21 + P22 = 0 + 0 = 0;

[0083] Third label: P’32 = P31 + P32 = 0 + 20% = 20%;

[0084] Use the obtained added results 35%, 0, and 20% to replace and update the probability values of the second frame on the first label, the second label, and the third label respectively.

[0085] For the third frame of speech signal, also since its probability value on the blank label is equal to the preset threshold, its final probability value is determined to be the probability value on the blank label. For the fourth frame of speech signal, since its probability value on the blank label is less than the preset threshold, based on the probability values of the third frame of speech signal on each non - blank label, the probability values of the fourth frame of speech signal on each non - blank label are updated. This update process is similar to the update process of the second frame of speech signal and will not be elaborated here. Among them, the calculation process of adding the probability values of the third frame of speech signal on each non - blank label and the probability values of the fourth frame of speech signal on each non - blank label is as follows:

[0086] First label: P’14 = P13 + P14 = 10% + 10% = 20%;

[0087] Second label: P’24 = P23 + P24 = 0 + 5% = 5%;

[0088] Third label: P’34 = P33 + P34 = 10% + 20% = 30%;

[0089] After that, replace and update the probability values of the fourth frame on the first label, second label, and third label with the obtained results 20%, 5%, and 30% respectively.

[0090] For the fifth frame of voice signal, first, since the probability value on the blank label is less than the preset threshold, the probability values of the fifth frame on each non - blank label are updated based on the probability values of the fourth frame of voice signal on each non - blank label. Second, since the probability value of the fourth frame of voice signal on the blank label is less than the preset threshold, its final probability value is not the probability value on the blank label but the maximum value among the updated probability values of each non - blank label. Therefore, by comparing the updated probability values of the fourth frame of voice signal on each non - blank label, it is determined that the maximum value is the probability value of the third label, which is 30%. So when updating the probability values of the fifth frame of voice signal on each non - blank label, the probability value on the third label is not processed, and only the probability values on the first label and the second label are added to the initial probability values of the fourth frame of voice signal on the second label and the second label respectively. The specific addition process is as follows:

[0091] First label: P’15 = P14 + P15 = 10% + 20% = 30%;

[0092] Second label: P’25 = P24 + P25 = 5% + 10% = 15%;

[0093] Third label: P’35 = P35 = 5%;

[0094] After that, replace and update the probability values of the fifth frame on the first label and the second label with the obtained results 30% and 15% respectively. In this way, the updated probability values of each frame of voice signal on each non - blank label are obtained; among them, Figure 2 The values in the brackets are the updated probability values of each frame of voice signal on the non - blank label.

[0095] Step S600, determine the final probability value of the currently judged voice signal based on the magnitude relationship between the updated probability values of the currently judged voice signal on each non - blank label.

[0096] Further, when updating the probability values of each frame of voice signal on non - blank labels in chronological order, for each updated frame of voice signal, after determining its final probability value according to the magnitude relationship of the probability values on each non - blank label, the judgment and update operation is performed on the next frame of voice signal. Specifically, step S600 includes:

[0097] Step c1, compare the updated probability values of the currently judged voice signal on each non - blank label to determine the target probability value with the largest value;

[0098] Step c2, identify the target probability value as the final probability value of the currently judged voice signal.

[0099] Furthermore, after the probability values of the currently judged voice signal on each non - blank preset label are all updated, compare the updated probability values on each non - blank preset label to determine the probability value with the largest value as the target probability value, and then identify the target probability value as the final probability value of the currently judged voice signal. For example, in Figure 2 if the currently judged voice signal is the fourth - frame voice signal, after the probability values on the first label, the second label, and the third label are all updated, it is determined by comparison that the value on the third label is the largest among the probability values of the non - blank labels of the fourth - frame voice signal. Therefore, the probability value on the third label is identified as the final probability value of the fourth - frame voice signal.

[0100] Further, after step S600, it includes:

[0101] Step d1, determine the voice signal in the next order after the currently judged voice signal according to the chronological relationship;

[0102] Step d2, set the currently judged voice signal as the previous voice signal with an earlier time, and set the voice signal in the next order as the new currently judged voice signal, and execute the step of judging whether the probability value of the voice signal on the blank label is greater than or equal to the preset threshold.

[0103] Furthermore, after the probability values of the currently judged voice signal on each non - blank preset label are updated and the final probability value of the currently judged voice signal is determined, according to the chronological order relationship between each voice signal, determine the voice signal in the next order after the currently judged voice signal. Then set the currently judged voice signal as the previous voice signal with an earlier time, and set the voice signal in the next order as the new currently judged voice signal. Specifically, Figure 2For the five-frame speech signal, after determining that the final probability value of the fourth-frame speech signal is the probability value on the third label, set the fifth-frame speech signal in the subsequent order as the new currently judged speech signal, and set itself as the previous speech signal with an earlier time; that is, set the fourth-frame speech signal as the previous speech signal with an earlier time, and set the fifth-frame speech signal as the currently judged speech signal. Based on the probability values of the fourth-frame speech signal on each non-blank preset label, update the probability values of the fifth-frame speech signal on each non-blank preset label, and sequentially update the probability values of the speech signal with a later adjacent time on each non-blank label with the probability values of the previous speech signal with an earlier time on each non-blank label.

[0104] It should be noted that the CTC acoustic model is trained and generated based on the CTC algorithm, and the CTC training algorithm has the characteristic of assuming that each time slice is independent. This makes the probability values of each frame of speech signal on multiple labels determined only based on each frame of speech signal itself, while ignoring the relevant semantic information between each time slice in speech recognition, resulting in inaccurate probability values determined for each frame of speech signal. In this embodiment, for two frames of speech signals with an adjacent relationship, update the probability values of the currently judged speech signal on each non-blank label with the probability values of the previous speech signal with an earlier time on each non-blank label, which is equivalent to correcting the probability of the later frame of speech signal on each non-blank label by means of the semantics represented by the previous frame of speech signal, reducing the misjudgment of the probability of the later frame of speech signal on each non-blank label. It has the effect of achieving the recognition accuracy of multiple paths by operating on one path.

[0105] Step S700, find the labels corresponding to the final probability values of each frame of the speech signal respectively, generate the found labels into a speech label sequence, and generate the recognition result of the audio data based on the semantics corresponding to each label.

[0106] Furthermore, since the labels corresponding to the final probability values of each frame of speech signal reflect the semantics that each frame of speech signal is most likely to express, the label from which the final probability value of each frame of speech signal is derived can be found according to the final probability value marked for each frame of speech signal, and the found labels can be generated into a speech label sequence. Among them, different labels are represented by different identifiers with different semantics, so that the label sequence can be formed through the identifiers corresponding to the found labels respectively, reflecting the semantics of the speech information. Specifically, in step S700, the step of generating the found labels into a speech label sequence includes:

[0107] Step e, determine the identifier corresponding to the retrieved label, and arrange each of the identifiers according to the time sequence relationship to generate the voice label sequence.

[0108] Furthermore, since the final probability value of the voice signal may be the probability value of the voice signal on the blank label or the maximum value among the probability values of each non - blank label; thus, the label retrieved based on the final probability value may be a blank label or a non - blank label. Blank labels and non - blank labels are distinguished by different identifiers, and the identifiers among non - blank labels are also different to reflect different semantics. For example, for Figure 2 the blank label, the first label, the second label, and the third label in, the set identifiers are 0, 1, 2, and 3 respectively.

[0109] Further, according to the correspondence between the label and the identifier, determine the identifiers corresponding to the retrieved labels respectively. The identifiers corresponding to the retrieved labels are the identifiers corresponding to each frame of the voice signal. Then, according to the time sequence relationship of each frame of the voice signal, arrange the identifiers corresponding to each frame of the voice information to obtain the voice label sequence. For example, for Figure 2 the five - frame voice data in, the final probability values of the first frame, the second frame, the third frame, the fourth frame, and the fifth frame of the voice data come from the blank label, the first label, the blank label, the third label, and the first label respectively; thus, determine the retrieved labels for each source, and determine the corresponding identifiers for each type of label as 0, 1, 0, 3, and 1 respectively. Then, arrange the corresponding identifiers according to the time sequence relationship of the first frame, the second frame, the third frame, the fourth frame, and the fifth frame of the voice signal to obtain the voice label sequence 01031.

[0110] Furthermore, different labels map to different semantics, and there is also a correspondence between the label and the identifier. Therefore, different identifiers map to different semantics. After forming the semantic label sequence, the semantics mapped by each identifier in the semantic label sequence can be determined according to the semantics corresponding to each label. Then, the sentences formed by combining each semantics are generated into the recognition result of the semantic data. This recognition result is the decoding result obtained by decoding the audio data. Different decoding results correspond to different control instructions. Therefore, according to the correspondence between the decoding result and the control instruction, determine the control instruction corresponding to the recognition result and execute the corresponding control instruction to complete the recognition response of the audio data in the voice information. For example, if the control instruction corresponding to the recognition result is to shut down, then execute the control instruction to shut down; if the control instruction corresponding to the recognition result is to play the next song, then execute the control instruction to switch songs. In this way, the recognition of the audio data carried by the voice information is realized, and the response of the intelligent interaction device to the voice information is completed.

[0111] In this embodiment, when the interactive voice information is detected, indicating the need for voice interaction, the voice data carried in the voice information is first divided into multiple frames of voice signals with a chronological relationship; then, through the CTC acoustic model, the probability values respectively corresponding to multiple labels in the CTC acoustic model for each frame of voice signal are determined, where the labels are identifiers reflecting semantics, including blank labels and non-blank labels, and the probability values of each frame of voice signal on each preset label reflect the likelihood of each frame of voice signal expressing various semantics; furthermore, according to the chronological relationship, it is determined one by one whether the probability value of the voice signal on the blank label is greater than or equal to a preset threshold; if the probability value of the currently judged voice signal on the blank label is greater than or equal to the preset threshold, the probability value on the blank label is identified as the final probability value of the currently judged voice signal; conversely, if the probability value of the currently judged voice signal on the blank label is less than the preset threshold, it is judged whether the probability value of the previous voice signal with an earlier time on the current non-blank label is the final probability value of the previous voice signal, and the probability value of the currently judged voice signal on the current non-blank label is updated according to the judgment result; thereafter, based on the magnitude relationship between the updated probability values of the currently judged voice signal on each non-blank label, the final probability value of the currently judged voice signal is determined, and the labels respectively corresponding to the final probability values of each frame of voice signal are searched, and the searched labels are generated into a voice label sequence, and then based on the semantics corresponding to each label, the voice label sequence is generated into the recognition result of the audio data, realizing the recognition of the interactive voice information.

[0112] Thus, through the magnitude relationship between the probability value of the voice signal on the blank label and the preset threshold, and the probability value of the previous voice signal on the non-blank label, the probability value of the currently judged voice signal on the non-blank label is updated, and the final probability value of each frame of signal is obtained, and a voice label sequence is generated. By judging the preset threshold on the blank label, the blank label and the non-blank label are accurately distinguished, avoiding the output of false blank labels from obscuring the true labels, which is beneficial to the accuracy of the decoding result. At the same time, the probability value of the currently judged voice signal on the non-blank label is related to the probability value of the previous voice signal, avoiding the inaccuracy of the voice recognition result caused by the independence of a single path and the information of each step; moreover, only the probability value of the previous voice signal is used to update the probability value of the currently judged voice signal, avoiding the calculation of multiple paths and improving the calculation efficiency. Therefore, the voice label sequence is determined by the simplified operation of one path, achieving the effect of improving the recognition accuracy by means of the probability information of multiple paths, and realizing accurate and efficient voice recognition through the CTC acoustic model.

[0113] Furthermore, a second embodiment of the voice recognition method of the present invention is proposed.

[0114] The difference between the second embodiment of the speech recognition method and the embodiment of the above speech recognition method is that before step S500, it includes:

[0115] Step f1, determining whether the currently judged speech signal is the speech signal with the earliest time;

[0116] Step f2, if it is the speech signal with the earliest time, then determine the final probability value of the currently judged speech signal according to the probability values respectively corresponding to the currently judged speech signal on each label;

[0117] Step f3, if the currently judged speech signal is not the speech signal with the earliest time, then execute step S500.

[0118] This embodiment processes the speech signal with the earliest time among the multiple frames of speech signals obtained by dividing the audio data, that is, the first frame of speech signal obtained by division. Specifically, when updating the probability values of the currently judged speech signal on non - blank labels according to whether the probability value of the previous speech signal with earlier time on non - blank labels is its final probability value, first determine whether the currently judged speech signal is the speech signal with the earliest time. If it is the speech signal with the earliest time, it means that the currently judged speech signal is the first frame of speech signal obtained by division, and there is no other speech signal before it. At this time, based on the probability values of the currently judged speech signal on each label, determine its final probability value. First, compare the probability value of it on the blank label with a preset threshold to judge whether it is greater than or equal to the preset threshold; if it is greater than or equal to the preset threshold, then identify the probability value of the currently judged speech signal on the blank label as its final probability value, indicating that this speech signal is most likely to fall into the blank label and is a blank signal without semantic representation. If it is determined through comparison that the probability value of the currently judged speech signal on the blank label is less than the preset threshold, then compare the probability values of the currently judged speech signal on each non - blank label, and determine the probability value with the largest value as the final probability value of the currently judged speech signal to reflect the semantics expressed by the currently judged speech signal.

[0119] Furthermore, if it is determined that the currently judged speech signal is not the speech signal with the earliest time, it means that there is a previous speech signal with earlier time before the currently judged speech signal. At this time, judge whether the probability values of the previous speech signal with earlier time on each non - blank label are its final probability values, and update the probability values of the currently judged speech signal on each non - blank label according to the judgment result. The specific update process is as described in the above embodiment and will not be elaborated here.

[0120] In this embodiment, it is determined whether the currently judged voice signal is the voice signal with the earliest time, and for the voice signal determined to be the one with the earliest time and the voice signal determined not to be the one with the earliest time, the currently judged voice signal is processed in different ways; for the voice signal with the earliest time, the probability value in each label is used as the final probability value to reflect the semantics expressed by the voice signal with the earliest time. In this way, the accuracy and effectiveness of determining the final probability value of the voice signal with the earliest time are ensured, making the update of the probability value of the subsequent sequential voice signal on each non-blank label more accurate, which is beneficial to improving the overall decoding accuracy of the audio data.

[0121] The present invention also provides an interactive voice recognition chip, and the interactive voice recognition chip can implement the above-mentioned interactive voice recognition method when performing interactive voice recognition.

[0122] The specific implementation manner of the interactive voice recognition chip of the present invention is basically the same as that of each embodiment of the above-mentioned interactive voice recognition method, and will not be repeated here.

[0123] The present invention also provides an intelligent interactive device, which can be an intelligent speaker or an earphone, and has the above-mentioned chip.

[0124] The specific implementation manner of the intelligent interactive device of the present invention is basically the same as that of each embodiment of the above-mentioned interactive voice recognition method, and will not be repeated here.

[0125] The present invention also proposes a readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each step of the above-mentioned interactive voice recognition method is implemented.

[0126] The readable storage medium of the present invention is preferably a computer-readable storage medium, and its specific implementation manner is basically the same as that of each embodiment of the above-mentioned interactive voice recognition method, and will not be repeated here.

[0127] Those skilled in the art can understand that, on the premise of no conflict, the above-mentioned preferred solutions can be freely combined and superimposed.

[0128] It should be understood that the above-mentioned embodiments are only exemplary and not restrictive. Without departing from the basic principle of the present invention, various obvious or equivalent modifications or substitutions made by those skilled in the art to the above details will all be included within the scope of the claims of the present invention.

Claims

1. An interactive speech recognition method based on a CTC acoustic model, characterized in that, the interactive speech recognition method includes the following steps: S100, when detecting interactive speech information, dividing the audio data corresponding to the speech information into multiple frames of speech signals with a time sequence; S200, based on the CTC acoustic model, determining the probability values respectively corresponding to each frame of the speech signal on multiple labels in the CTC acoustic model, where the multiple labels include a blank label and multiple non - blank labels; S300, according to the time sequence, successively judging whether the probability value of the speech signal on the blank label is greater than or equal to a preset threshold: if the probability value of the currently judged speech signal on the blank label is greater than or equal to the preset threshold, then execute step S400; if the probability value of the currently judged speech signal on the blank label is less than the preset threshold, then execute step S500; S400, identifying the probability value on the blank label as the final probability value of the currently judged speech signal; S500, judging whether the probability value of the previous speech signal with an earlier time on the current non - blank label is the final probability value of the previous speech signal, and updating the probability value of the currently judged speech signal on the current non - blank label according to the judgment result; S600, based on the magnitude relationship between the updated probability values of the currently judged speech signal on each non - blank label, determining the final probability value of the currently judged speech signal; S700, searching for the labels respectively corresponding to the final probability values of each frame of the speech signal, generating the searched labels into a speech label sequence, and based on the semantics corresponding to each label, generating the recognition result of the audio data from the speech label sequence.

2. The interactive speech recognition method according to claim 1, characterized in that, step S500 includes: judging whether the probability value of the previous speech signal with an earlier time on the current non - blank label is the final probability value of the previous speech signal: if it is the final probability value of the previous speech signal, then keep the probability value of the currently judged speech signal on the current non - blank label unchanged; if it is not the final probability value of the previous speech signal, then perform an addition operation on the probability value of the previous speech signal on the current non - blank label and the probability value of the currently judged speech signal on the current non - blank label, and use the operation result of the addition operation to update the probability value of the currently judged speech signal on the current non - blank label.

3. The interactive speech recognition method according to claim 1, characterized in that, before step S500 includes: determining whether the currently judged speech signal is the speech signal with the earliest time; if it is the speech signal with the earliest time, then determining the final probability value of the currently judged speech signal according to the probability values respectively corresponding to the currently judged speech signal on each label; if the currently judged speech signal is not the speech signal with the earliest time, then execute step S500.

4. The interactive voice recognition method according to claim 1, wherein, the step S600 includes: Comparing the probability values updated by the current judged speech signal on each non-blank label to determine the target probability value with the largest numerical value; Identifying the target probability value as the final probability value of the current judged speech signal.

5. The interactive voice recognition method according to claim 1, wherein, after the step S600 includes: Determining the speech signal in the subsequent order of the currently judged speech signal according to the time sequence relationship; Setting the currently judged speech signal as the previous speech signal with an earlier time, and setting the speech signal in the subsequent order as the newly currently judged speech signal, and performing the step of judging whether the probability value of the speech signal on the blank label is greater than or equal to a preset threshold.

6. The interactive voice recognition method according to claim 1, wherein, in step S700, the step of generating the found label into a speech label sequence includes: Determining the identifier corresponding to the found label, and arranging the identifiers according to the time sequence relationship to generate the speech label sequence.

7. The interactive voice recognition method according to any one of claims 1-6, wherein, after the step S700 includes: Determining the control instruction corresponding to the recognition result and executing the control instruction.

8. The interactive voice recognition method according to any one of claims 1-6, wherein, before the step S100 includes: When receiving voice information, identifying whether the voice information contains a wake-up word; If it contains a wake-up word, it is determined that interactive voice information is detected.

9. The interactive voice recognition method according to any one of claims 1-6, wherein, before the step S200 includes: Receiving a voice training sample and training an initial CTC acoustic model based on the voice training sample to generate the CTC acoustic model.

10. An interactive voice recognition chip, wherein, the interactive voice recognition chip can implement the interactive voice recognition method according to any one of claims 1-9 when performing interactive voice recognition.

11. A readable storage medium, on which a computer program is stored, wherein, the computer program, when executed by a processor, implements the interactive voice recognition method according to any one of claims 1-9.

12. An intelligent interactive device, wherein, the intelligent interactive device has the chip according to claim 10.

13. The intelligent interactive device according to claim 12, wherein, the intelligent interactive device is an intelligent speaker or an earphone.

Images