Streaming speech recognition method, device, equipment and storage medium

Abstract

The present invention discloses a streaming speech recognition method, apparatus, device and storage medium. The method comprises: receiving a speech stream as a current recognition segment; performing silence detection on the current recognition segment, and judging whether the current recognition segment contains a preset silence portion based on the result of the silence detection; if the current recognition segment contains the preset silence portion, decoding the current recognition segment using a decoder to obtain a corresponding speech recognition result, and initializing corresponding parameters in the decoder after decoding the portion of the current recognition segment located before the preset silence portion; if the current recognition segment does not contain the preset silence portion, decoding the current recognition segment using a decoder to obtain a corresponding speech recognition result, and in the process of decoding the current recognition segment, there is no need to initialize the corresponding parameters in the decoder; the preset silence portion is a portion where the duration of silence is greater than a preset duration. Thus, the speech recognition speed can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of audio detection technology, and more particularly to a streaming speech recognition method, apparatus, device and storage medium. Background Art

[0002] With the continuous advancement of technology, more and more companies and institutions are launching products based on voice interaction technology, such as intelligent outbound call robots, intelligent customer service quality control, and conference subtitles. These products use technologies such as speech recognition and natural language processing to analyze user voice input and complete corresponding operations or tasks.

[0003] To improve user experience during voice interaction, these products must promptly process user voice input to reduce response latency. For example, in intelligent outbound calling scenarios, the product must accurately and quickly convert the voice into text via a voice recognition server. Then, based on natural language processing, it must determine the user's intent and respond accordingly, completing the conversation.

[0004] In traditional solutions, the voice stream is usually continuously input into the voice recognition engine for voice recognition, which may specifically include obtaining the probability matrix of the input voice and decoding the probability matrix. However, due to the long length of the voice, the decoding path will continue to accumulate during the voice recognition process, and the decoding speed will continue to slow down. That is, the method of implementing voice recognition in the existing technology has the problem of slow voice recognition speed. Summary of the Invention

[0005] The object of the present invention is to provide a streaming speech recognition method, apparatus, device and storage medium, which can effectively improve the speech recognition speed.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A streaming speech recognition method, comprising:

[0008] receiving a voice stream, and determining that the currently received voice stream is a currently recognized segment;

[0009] Performing silence detection on the current recognition segment and determining whether the current recognition segment contains a preset silence portion based on the result of the silence detection; wherein the preset silence portion is a portion whose duration is greater than a preset duration;

[0010] If the current recognition segment includes the preset silent portion, decoding the current recognition segment using a decoder to obtain a corresponding speech recognition result, and initializing corresponding parameters in the decoder after decoding a portion of the current recognition segment preceding the preset silent portion;

[0011] If the current recognition segment does not contain the preset silent portion, the decoder is used to decode the current recognition segment to obtain a corresponding speech recognition result, and there is no need to initialize corresponding parameters in the decoder during the decoding of the current recognition segment.

[0012] Preferably, decoding the current identified segment using a decoder comprises:

[0013] Determine whether the current recognition segment is the first recognition segment of the corresponding speech segment;

[0014] If the current recognition segment is the first recognition segment of the speech segment, the probability matrix of the current recognition segment is input into the decoder for decoding; wherein the probability matrix of any recognition segment is obtained by inputting the arbitrary recognition segment into the acoustic model trained by CTC;

[0015] If the current recognition segment is not the first recognition segment of the speech segment to which it belongs, a new probability matrix is ​​calculated using the probability matrices of the two recognition segments before the current recognition segment and the probability matrix of the current recognition segment, and the new probability matrix is ​​input into the decoder for decoding as the probability matrix of the current recognition segment.

[0016] Preferably, the probability matrix of the current recognition segment is input into the decoder for decoding, comprising:

[0017] If the current recognition segment contains the preset silence portion, each of the preset silence portions contained in the current recognition segment is used as a breakpoint, the current recognition segment is divided into a plurality of recognition sub-segments using the breakpoints, and the probability matrix of each recognition sub-segment is input into the decoder for decoding;

[0018] If the currently recognized segment does not contain the preset silent portion, the probability matrix of the currently recognized segment is directly input into the decoder for decoding.

[0019] Preferably, the new probability matrix is ​​calculated using the probability matrices of the two recognition segments before the current recognition segment and the probability matrix of the current recognition segment, including:

[0020] Determine that probability matrices corresponding to the first two recognition segments of the current recognition segment are a first probability matrix and a second probability matrix, respectively; wherein the first probability matrix is ​​obtained earlier than the second probability matrix;

[0021] Calculating a matrix value of the first probability matrix to be a preset ratio, and determining a behavior sub-probability matrix of the preset ratio located at the tail of the second probability matrix;

[0022] The sub-probability matrix and the probability matrix of the current recognition matrix are concatenated to obtain a new probability matrix.

[0023] Preferably, determining whether the current recognition segment is the first recognition segment of the corresponding speech segment includes:

[0024] Check the value of the start flag;

[0025] If the value of the start flag is a first preset value, determining that the current recognition segment is the first recognition segment of the speech segment;

[0026] If the value of the start flag is a second preset value, it is determined that the current recognition segment is not the first recognition segment of the corresponding speech segment; wherein, after receiving the designated recognition segment, the start flag is set to the second preset value, and the rest of the time, the start flag is set to the first preset value, and the designated recognition segment is a recognition segment containing the preset silent part.

[0027] Preferably, judging whether the currently recognized segment contains a preset silence portion based on the result of the silence detection includes:

[0028] Get multiple blank scores contained in the silence detection result;

[0029] If there are any multiple consecutive blank scores that are all greater than the score threshold, it is determined that the current recognition segment contains the preset silent portion, and the portion in the current recognition segment corresponding to the any multiple consecutive blank scores is the preset silent portion;

[0030] If there are not any multiple consecutive blank scores all greater than the score threshold, it is determined that the current recognition segment does not contain the preset silent portion.

[0031] Preferably, before performing silence detection on the current identified segment, the method further includes:

[0032] Check the value of the end flag;

[0033] If the value of the end flag is a third preset value, after decoding the current identification segment using the decoder, initializing corresponding parameters in the decoder;

[0034] If the value of the end flag is the fourth preset value, the step of performing silence detection on the current identification segment is executed; wherein, after receiving the end instruction sent by the management terminal, the end flag is set to the third preset value, and the rest of the time, the end flag is set to the fourth preset value.

[0035] A streaming speech recognition device, comprising:

[0036] The determination module is configured to: receive a voice stream and determine that the currently received voice stream is a currently recognized segment;

[0037] a detection module configured to perform silence detection on the currently identified segment and determine, based on the result of the silence detection, whether the currently identified segment contains a preset silence portion; wherein the preset silence portion is a portion whose duration is greater than a preset duration;

[0038] A first execution module is configured to: if the current recognition segment includes the preset silent portion, decode the current recognition segment using a decoder to obtain a corresponding speech recognition result, and initialize corresponding parameters in the decoder after decoding a portion of the current recognition segment preceding the preset silent portion;

[0039] The second execution module is used to: if the current recognition segment does not contain the preset silent part, use the decoder to decode the current recognition segment to obtain a corresponding speech recognition result, and there is no need to initialize the corresponding parameters in the decoder during the decoding of the current recognition segment.

[0040] A streaming speech recognition device, comprising:

[0041] memory for storing computer programs;

[0042] A processor is configured to implement the steps of any of the above-mentioned streaming speech recognition methods when executing the computer program.

[0043] A computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of any one of the above streaming speech recognition methods.

[0044] The present invention provides a streaming speech recognition method, apparatus, device and storage medium, the method comprising: receiving a speech stream, determining that the currently received speech stream is a current recognition segment; performing silence detection on the current recognition segment, and judging whether the current recognition segment contains a preset silence portion based on the result of the silence detection; if the current recognition segment contains the preset silence portion, decoding the current recognition segment using a decoder to obtain a corresponding speech recognition result, and initializing corresponding parameters in the decoder after decoding the portion of the current recognition segment that is located before the preset silence portion; if the current recognition segment does not contain the preset silence portion, decoding the current recognition segment using the decoder to obtain a corresponding speech recognition result, and there is no need to initialize corresponding parameters in the decoder during the decoding of the current recognition segment; wherein the preset silence portion is a portion whose silence duration is greater than a preset duration.

[0045] Since streaming voice may contain multiple voice segments that have no direct correlation in meaning, and voice segments that have no direct correlation do not need to be considered with each other during decoding, the present application can determine whether the received voice stream contains the last voice part of the voice segment by judging whether the received voice stream has a sufficiently long silent part. If the voice stream has a sufficiently long silent part, it means that the meaningful text part contained in the voice segment before the silent part in the voice stream has been input, that is, the part before the silent part in the voice stream is the last voice part of the voice segment. Taking into account the above silent part in the voice stream, If the decoding of the voice segment to which the part before the silent part belongs has no influence on the decoding of the voice stream after it, the corresponding parameters in the decoder can be initialized to clear the corresponding decoding path. If the voice stream does not have a silent part long enough, the corresponding parameters in the decoder are not initialized in order to consider the influence of the voice stream on other voice streams of the voice segment to which it belongs, but the current decoder is continued to be used. In this way, while ensuring that the received voice stream can be effectively decoded for the corresponding voice recognition, the continuous accumulation of decoding paths and the continuous slowing down of decoding speed due to the long voice length are avoided, which can effectively improve the voice recognition speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0047] Figure 1 A flow chart of a streaming speech recognition method provided by an embodiment of the present invention;

[0048] Figure 2 A flowchart of decoding a current recognition segment by using a decoder in a streaming speech recognition method provided by an embodiment of the present invention;

[0049] Figure 3 A flowchart of calculating a new probability matrix in a streaming speech recognition method provided by an embodiment of the present invention;

[0050] Figure 4 A flowchart of determining whether a current recognition segment is the first recognition segment of a speech segment in a streaming speech recognition method provided by an embodiment of the present invention;

[0051] Figure 5 A flow chart of a streaming speech recognition method according to an embodiment of the present invention that determines whether to perform silence detection based on an end flag;

[0052] Figure 6 A flowchart of determining whether a current recognition segment contains a preset silence portion in a streaming speech recognition method provided by an embodiment of the present invention;

[0053] Figure 7 A schematic structural diagram of a streaming speech recognition device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0055] See also Figure 1 , which shows a flow chart of a streaming speech recognition method provided by an embodiment of the present invention, which may include:

[0056] S11: Receive a voice stream, and determine that the currently received voice stream is a current recognition segment.

[0057] The embodiment of the present invention provides a method for streaming speech recognition, which can be performed by a streaming speech recognition device or apparatus, or referred to as a speech recognition engine. The following description specifically describes the method as being performed by a speech recognition engine. It should be noted that a speech stream can be continuously input into the speech recognition engine. Each time the speech recognition engine receives a segment of the input speech stream, it treats the segment as a recognition segment and subsequently performs speech recognition on the segment.

[0058] S12: Perform silence detection on the current recognition segment, and determine whether the current recognition segment contains a preset silence portion based on the result of the silence detection; wherein the preset silence portion is a portion whose duration is greater than the preset duration.

[0059] Among them, the preset duration can be set according to actual needs, such as 8ms, 10ms, etc. In this application, each time a voice stream is received, it is regarded as a recognition segment, and the currently received recognition segment can be referred to as the current recognition segment. It should be noted that a continuously input voice stream may contain multiple voice segments with no direct connection between their meanings. There is a corresponding silent part at the end of each voice segment. The same as in the prior art, the silent part is a part of the voice that does not have any actual meaning and cannot be converted into text. For example, when the voice stream is the sound of external speech, each segment will be followed by a period of silence before the next segment begins. Therefore, the voice stream of each segment is regarded as a corresponding voice segment. In order to distinguish different voice segments and realize intelligent segmentation of streaming voice, this application can perform silence detection on the current recognition segment. When it is determined based on the result of the silence detection that there is a sufficiently long silent part in the current recognition segment, the part of the current recognition segment before the sufficiently long silent part can be considered as the last voice part of the voice segment to which it belongs, so that the continuously input voice stream can be divided into corresponding voice segments based on this.

[0060] In addition, the current recognition segment may generally only contain one preset silent part, but in special cases it may also contain multiple preset silent parts. At this time, the part before each preset silent part in the current recognition segment is the last part of the speech segment to which it belongs. Therefore, after decoding the undecoded part before each preset silent part in the current recognition segment, it is necessary to initialize the relevant parameters in the decoder to clear the decoding path of the corresponding speech segment.

[0061] S13: If the current recognition segment contains a preset silent portion, the current recognition segment is decoded using a decoder to obtain a corresponding speech recognition result, and after decoding the portion of the current recognition segment before the preset silent portion, the corresponding parameters in the decoder are initialized.

[0062] Among them, the decoder has the same meaning as the decoder used in the prior art to implement speech recognition. The decoder can be used to decode any recognition segment to obtain the result of speech recognition of the arbitrary recognition segment, that is, the speech recognition result; when the decoder is used to decode the recognition segment, if the decoding-related parameters in the decoder are not initialized, these parameters will be affected by the operation of decoding the recognition segment and change accordingly, and the change in these parameters will lead to the accumulation of corresponding decoding paths, that is, continuously using the decoder to decode the recognition segment will lead to the continuous accumulation of corresponding decoding paths, which is consistent with the decoding principle of the decoder in the prior art.

[0063] Since there is a high probability that there is a certain correlation between the meanings of different voice streams in the same voice segment, the present application does not initialize the decoding-related parameters in the decoder during the decoding of the voice streams in the same voice segment, but allows the influence of the voice stream recorded earlier in the same voice segment on the corresponding parameters in the decoder to continue to the voice stream recorded later, thereby achieving effective decoding of different voice streams in the same voice segment; since there is usually no correlation between the meanings of different voice streams in different voice segments, the present application will initialize the decoding-related parameters in the decoder after decoding all the voice streams of any voice segment, so as to clear the decoding paths accumulated during the decoding of the voice stream of the arbitrary voice segment, and avoid the slowdown of the decoding speed due to the accumulation of decoding paths. Among them, initializing the decoding-related parameters in the decoder can be setting the values ​​of the decoding-related parameters in the decoder to default values.

[0064] S14: If the current recognition segment does not contain the preset silent portion, the current recognition segment is decoded by using a decoder to obtain a corresponding speech recognition result, and in the process of decoding the current recognition segment, there is no need to initialize corresponding parameters in the decoder.

[0065] If the current recognition segment does not contain the preset silent part, it means that the last speech part of the speech segment does not exist in the current recognition segment. Therefore, the relevant parameters in the decoder are initialized during the decoding of the current recognition segment, and the current decoder is directly used to realize the decoding of subsequent related speech streams.

[0066] Since streaming voice may contain multiple voice segments that have no direct correlation in meaning, and voice segments that have no direct correlation do not need to be considered with each other during decoding, the present application can determine whether the received voice stream contains the last voice part of the voice segment by judging whether the received voice stream has a sufficiently long silent part. If the voice stream has a sufficiently long silent part, it means that the meaningful text part contained in the voice segment before the silent part in the voice stream has been input, that is, the part before the silent part in the voice stream is the last voice part of the voice segment. Taking into account the above silent part in the voice stream, If the decoding of the voice segment to which the part before the silent part belongs has no influence on the decoding of the voice stream after it, the corresponding parameters in the decoder can be initialized to clear the corresponding decoding path. If the voice stream does not have a silent part long enough, the corresponding parameters in the decoder are not initialized in order to consider the influence of the voice stream on other voice streams of the voice segment to which it belongs, but the current decoder is continued to be used. In this way, while ensuring that the received voice stream can be effectively decoded for the corresponding voice recognition, the continuous accumulation of decoding paths and the continuous slowing down of decoding speed due to the long voice length are avoided, which can effectively improve the voice recognition speed.

[0067] like Figure 2 As shown, an embodiment of the present invention provides a streaming speech recognition method, which uses a decoder to decode the current recognition segment, and may include:

[0068] S21: Determine whether the current recognition segment is the first recognition segment of the corresponding speech segment;

[0069] S22: If the current recognition segment is the first recognition segment of the speech segment, the probability matrix of the current recognition segment is input into the decoder for decoding; wherein the probability matrix of any recognition segment is obtained by inputting the arbitrary recognition segment into the acoustic model trained by CTC;

[0070] S23: If the current recognition segment is not the first recognition segment of the speech segment to which it belongs, a new probability matrix is ​​calculated using the probability matrices of the two recognition segments before the current recognition segment and the probability matrix of the current recognition segment, and the new probability matrix is ​​input into the decoder for decoding as the probability matrix of the current recognition segment.

[0071] Among them, CTC (Connectionist temporal classification, temporal classification based on neural network) is a method of calculating loss used in neural network training to solve the classification problem of temporal data, and the acoustic model trained by CTC has the same meaning as the corresponding concept in the prior art; by inputting any recognition segment into the acoustic model, the probability matrix output by the acoustic model can be obtained, and then the probability matrix is ​​input into the decoder, and the speech recognition result of the arbitrary recognition segment output by the decoder can be obtained. Since there is a high probability of correlation between the meanings of different speech streams belonging to the same speech segment, when decoding any recognition segment, the present application can consider the first two recognition segments of the arbitrary recognition segment, and then comprehensively calculate a new probability matrix based on the probability distance of the arbitrary recognition segment and the probability matrix of the first two recognition segments of the arbitrary recognition segment, and decode the new probability matrix as the probability matrix of the arbitrary recognition segment, thereby effectively improving the decoding accuracy of the recognition segment.

[0072] In addition, since the first recognition segment of any speech segment does not exist in the first two recognition segments, the probability matrix of the first recognition segment of any speech segment can be directly decoded. Since the second recognition segment of any speech segment only has the previous recognition segment, it can also be processed as the first recognition segment, that is, its probability matrix can be decoded; of course, other settings made according to actual needs are also within the scope of protection of the present invention.

[0073] An embodiment of the present invention provides a streaming speech recognition method, which inputs a probability matrix of a current recognition segment into a decoder for decoding, and may include:

[0074] If the current recognition segment contains a preset silent portion, each preset silent portion contained in the current recognition segment is used as a breakpoint, and the current recognition segment is divided into multiple recognition sub-segments using the breakpoints, and the probability matrix of each recognition sub-segment is input into the decoder for decoding;

[0075] If the current recognition segment does not contain the preset silent portion, the probability matrix of the current recognition segment is directly input into the decoder for decoding.

[0076] Since there may be multiple preset silent parts in the same voice stream, if these preset silent parts are located in the middle of the voice stream, the part between two adjacent preset silent parts in the voice stream can be used as an identification sub-segment, the part between the beginning of the voice stream and the first preset silent part in the middle of the voice stream can be used as an identification sub-segment, and the part between the end of the voice stream and the last preset silent part in the middle of the voice stream can be used as an identification sub-segment, so that each preset silent part in the middle of the same voice stream is used as a breakpoint, and the voice stream is divided into multiple identification sub-segments using these breakpoints. Then, the probability matrices corresponding to these identification sub-segments are input into the decoder for decoding in turn, and after decoding the probability matrix of each identification sub-segment, the relevant parameters in the decoder need to be initialized. It can be seen that the present application can divide the video stream into multiple identification sub-segments based on the preset silent parts contained in the video stream, and then realize the decoding of the probability matrices of these identification sub-segments, ensuring that the relevant parameters in the decoder are initialized after the decoding of each voice segment is completed.

[0077] like Figure 3 As shown, an embodiment of the present invention provides a streaming speech recognition method, which uses the probability matrices of the two recognition segments before the current recognition segment and the probability matrix of the current recognition segment to calculate a new probability matrix, which may include:

[0078] S31: Determine that the probability matrices corresponding to the first two recognition segments of the current recognition segment are a first probability matrix and a second probability matrix, respectively; wherein the first probability matrix is ​​obtained earlier than the second probability matrix;

[0079] S32: Calculate the matrix value of the first probability matrix to be a preset ratio, and determine the behavior sub-probability matrix of the preset ratio located at the tail of the second probability matrix;

[0080] S33: Concatenate the sub-probability matrix and the probability matrix of the current recognition matrix to obtain a new probability matrix.

[0081] Among them, the first probability matrix is ​​obtained earlier than the second probability matrix, that is, the recognition segment corresponding to the first probability matrix is ​​received earlier than the recognition segment corresponding to the second probability matrix. It should be noted that when the present application calculates a new probability matrix using the probability matrices corresponding to the current recognition segment and its first two recognition segments respectively, the matrix value of the probability matrix of the recognition segment with the earlier reception time in the first two recognition segments can be calculated, and the matrix value is used as a preset ratio. Then, starting from the last row of the probability matrix of the recognition segment with the later reception time in the first two recognition segments, at least one row is taken upward as a sub-probability matrix. The proportion of the sub-probability matrix to the probability matrix of the recognition segment with the later reception time in the first two recognition segments is a preset ratio. Finally, the sub-probability matrix and the probability matrix of the current recognition segment are spliced ​​together to obtain a new probability matrix. In this way, the current recognition segment can be effectively decoded based on the current recognition segment and its first two recognition segments, thereby improving the decoding accuracy of the current recognition segment.

[0082] like Figure 4 As shown, an embodiment of the present invention provides a streaming speech recognition method, which determines whether the current recognition segment is the first recognition segment of the speech segment, and may include:

[0083] S41: Detect the value of the start flag;

[0084] S42: If the value of the start flag is the first preset value, determining that the current recognition segment is the first recognition segment of the corresponding speech segment;

[0085] S43: If the value of the start flag is the second preset value, it is determined that the current recognition segment is not the first recognition segment of the corresponding speech segment; wherein, after receiving the designated recognition segment, the start flag is set to the second preset value, and the rest of the time, the start flag is set to the first preset value, and the designated recognition segment is a recognition segment containing a preset silent part.

[0086] Among them, the values ​​of the first preset value and the second preset value can be set according to actual needs, such as the first preset value is 1, and the second preset value is 0. In order to facilitate the rapid judgment of whether the current recognition segment is the first recognition segment of the speech segment to which it belongs, the present application can be provided with a start flag, and the default value of the start flag is the first preset value. Specifically, a data stream is received, and the value of the start flag is the first preset value at this time, so it can be determined that the data stream is the first recognition segment of the speech segment to which it belongs, and the value of the start flag is changed to the second preset value. Then, a new data stream is received, and the value of the start flag is the second preset value at this time, so it can be determined that the new data stream received is not the first recognition segment of the speech segment to which it belongs. If the new data stream received contains the last recognition segment of the speech segment to which it belongs, the value of the start flag is changed to the first preset value. Otherwise, the value of the start flag is not changed, and the data stream continues to be received, and so on.

[0087] like Figure 5 As shown, a streaming speech recognition method provided by an embodiment of the present invention may further include, before performing silence detection on the current recognition segment:

[0088] S51: Detect the value of the end flag;

[0089] S52: If the value of the end flag is a third preset value, after decoding the current identified segment using the decoder, initializing corresponding parameters in the decoder;

[0090] S53: If the value of the end flag is the fourth preset value, perform a step of performing silence detection on the current recognition segment; wherein, after receiving the end instruction sent by the management terminal, the end flag is set to the third preset value, and the end flag is set to the fourth preset value at other times.

[0091] Among them, the values ​​of the third preset value and the fourth preset value can be set according to actual needs, such as the third preset value is 1, and the fourth preset value is 0. In the embodiment of the present application, the outside world can send an instruction to end voice recognition, that is, an end instruction, to the voice recognition engine through the management terminal; after receiving the end instruction, the voice recognition engine can respond to the end instruction to stop receiving the voice stream after completing the recognition of the received recognition segment, and set the value of the end flag to the third preset value; if after receiving any recognition segment, the value of the end flag is the third preset value, it means that the arbitrary recognition segment must be the last recognition segment, so there is no need to perform silence detection on it, but directly decode it and initialize the corresponding parameters in the decoder, otherwise, continue to implement silence detection and other steps. It can be seen that the present application can meet the needs of the outside world to end voice recognition, and can no longer perform silence detection on the received voice stream when the outside world needs to end voice recognition, effectively avoiding the waste of corresponding resources.

[0092] like Figure 6 As shown, an embodiment of the present invention provides a streaming speech recognition method, which determines whether a current recognition segment contains a preset silence portion based on the result of silence detection, and may include:

[0093] S61: Obtain multiple blank scores included in the silence detection result;

[0094] S62: If there are any multiple consecutive blank scores that are all greater than the score threshold, it is determined that the current recognition segment contains a silent portion, and the portion of the current recognition segment corresponding to the any multiple consecutive blank scores is a silent portion;

[0095] S63: If there are not any multiple consecutive blank scores all greater than the score threshold, it is determined that the current recognition segment does not contain a silent part.

[0096] Among them, the score threshold can be set according to actual needs. The present application can be implemented based on the blank score when implementing silence detection. The blank score has the same meaning as the corresponding concept in the prior art. Blank represents empty, and the blank score is the probability of the corresponding part being a non-word. This part may be silence or noise, but not one or more words. Based on this feature, it is possible to judge whether the corresponding part in the current recognition segment is a human voice based on the size of the blank score. Specifically, any recognition segment can contain multiple blank scores, and these blank scores correspond to different parts of any recognition segment. For example, a recognition segment is 40ms, and the generated blank score corresponds to one for every 10ms, then a 40ms recognition segment corresponds to 4 blank scores; in a specific implementation method, the blank score of any recognition segment can be the last column of the probability matrix obtained after inputting it into the acoustic model. If any blank score is greater than the corresponding score threshold, it means that the portion of the recognition segment corresponding to the blank score is empty and does not contain any text, that is, it is silent; therefore, when there are multiple consecutive blank scores that are all greater than the score threshold, it can be determined that the portion of the recognition segment corresponding to these multiple consecutive blank scores is the preset silent portion, so that the corresponding preset silent portion can be effectively determined based on the blank score of the recognition segment. In addition, when determining the preset silent portion, the number of consecutive blank scores that need to be greater than the score threshold is set according to actual needs, such as 3; and the total duration of the portion of the recognition segment corresponding to the multiple consecutive blank scores that need to be greater than the score threshold is greater than the preset duration.

[0097] In a specific implementation, the process of implementing silence detection based on the blank score may include:

[0098] Step 1: Initialize three lists, which are used to record the starting position, ending position and blank score of each identified sub-segment after division based on the preset silent part;

[0099] Step 2: Determine whether the end flag is 1 (the third preset value is 1). If so, the start position is the beginning of the current recognition segment and the end position is the end of the current recognition segment; otherwise, execute step 3;

[0100] Step 3: Fill the blank score of the current recognition segment into the blank score list, and each blank score has a corresponding serial number. The starting position list is added to the starting position of the current recognition segment; initialize the start variable and end variable to 0, then loop through the blank score list and execute step 4;

[0101] Step 4: Update the values ​​of the start and end variables according to the blank score. If the blank score is greater than the score threshold, assign the end variable to the sequence number of the current blank score; otherwise, assign the start variable to the sequence number of the current blank score. When the values ​​of the end and start variables are greater than the segmentation threshold set according to actual needs and the next blank score is also greater than the score threshold, add the value of the end variable to the start position list and the end position list, and update the values ​​of the start and end variables to the sequence number of the current blank score.

[0102] In step 5, the list of end positions is added to the end position of the current recognition segment, and finally each preset silent portion and each recognition sub-segment are determined based on the list of start positions and the list of end positions.

[0103] The process of implementing streaming speech recognition can specifically include:

[0104] Step 1: Input a voice stream;

[0105] Step 2: Use the input voice stream as the current recognition segment and execute the CTC blank intelligent segmentation logic (step 3);

[0106] Step 3: Determine whether the start flag is 0 (the second preset value is 0). If so, proceed to step 4. Otherwise, concatenate the probability matrix returned by the previous recognition segment processing with the probability matrix of the current recognition segment for decoding, and return the probability matrix returned by the previous recognition segment processing.

[0107] Step 4: Determine whether the probability matrix returned by the previous recognition segment is empty. If it is empty, directly obtain the segmentation result achieved by the blank score. Otherwise, based on the size of the probability matrix returned by the previous recognition segment, obtain the sub-probability matrix from the starting node of the current recognition segment (the last row of the probability matrix of the previous recognition segment of the current recognition segment) upwards, and concatenate this sub-probability matrix with the probability matrix of the current recognition segment to obtain the final probability matrix of the current recognition segment, and obtain the segmentation result achieved by the blank score.

[0108] Step 5: If the current recognition segment is divided into multiple recognition sub-segments, the probability matrix corresponding to each recognition sub-segment is input into the decoder to obtain the corresponding speech recognition result. After each recognition sub-segment is decoded, the decoding parameters are initialized, and the speech recognition results corresponding to each recognition sub-segment are spliced ​​to obtain the total speech recognition result of the current recognition segment, and the probability matrix of the last recognition sub-segment is returned. If the current recognition segment is not divided into multiple recognition sub-segments, the probability matrix of the current recognition segment is directly input into the decoder to obtain the corresponding speech recognition result, and the probability matrix returned by the previous recognition segment is returned. Among them, the probability matrix returned by any recognition segment and the probability matrix of any recognition segment have different meanings.

[0109] In a specific application scenario, the input voice is the sound of someone speaking from the outside world, and each received data stream is 20ms. After each 20ms data stream is received, the data stream segment is used as the current segment to be recognized to implement streaming voice recognition. In the streaming voice recognition process, the voice segment contains the voice of the outside world continuously speaking a certain paragraph. The preset silent part is the sound of the outside world not speaking, and the rest is the sound of the outside world speaking. The recognition sub-segment division based on the preset silent part can actually be understood as intelligent segmentation. Finally, the relevant parameters of the decoder are initialized through the result of intelligent segmentation to eliminate the corresponding decoding path and avoid the decoding speed being slowed down due to the continuous accumulation of decoding paths.

[0110] In summary, the present application can locate the preset silent part based on the blank score, and then realize the division of the voice stream. It uses the peak characteristics of the acoustic model trained by CTC to effectively segment the voice in environments such as music noise, preventing the decoding speed from slowing down; at the same time, it does not introduce additional computing consumption.

[0111] The embodiment of the present invention also provides a streaming speech recognition device, such as Figure 7 As shown, this may include:

[0112] The determining module 11 is configured to: receive a voice stream and determine that the currently received voice stream is a currently recognized segment;

[0113] The detection module 12 is configured to perform silence detection on the currently identified segment and determine whether the currently identified segment contains a preset silence portion based on the result of the silence detection; wherein the preset silence portion is a portion whose duration is greater than a preset duration;

[0114] The first execution module 13 is configured to: if the current recognition segment contains a preset silent portion, decode the current recognition segment using a decoder to obtain a corresponding speech recognition result, and initialize corresponding parameters in the decoder after decoding the portion of the current recognition segment preceding the preset silent portion;

[0115] The second execution module 14 is used to: if the current recognition segment does not contain the preset silent part, use the decoder to decode the current recognition segment to obtain the corresponding speech recognition result, and there is no need to initialize the corresponding parameters in the decoder during the decoding process of the current recognition segment.

[0116] In a streaming speech recognition device provided by an embodiment of the present invention, the first execution module and the second execution module may each include:

[0117] The decoding module is used to: determine whether the current recognition segment is the first recognition segment of the speech segment to which it belongs; if the current recognition segment is the first recognition segment of the speech segment to which it belongs, then input the probability matrix of the current recognition segment into the decoder for decoding; wherein, the probability matrix of any recognition segment is obtained by inputting the arbitrary recognition segment into the acoustic model trained by CTC; if the current recognition segment is not the first recognition segment of the speech segment to which it belongs, then use the probability matrices of the two recognition segments before the current recognition segment and the probability matrix of the current recognition segment to calculate a new probability matrix, and input the new probability matrix into the decoder for decoding as the probability matrix of the current recognition segment.

[0118] An embodiment of the present invention provides a streaming speech recognition device, wherein the decoding module may include:

[0119] The division unit is used to: if the current recognition segment contains a preset silent part, each preset silent part contained in the current recognition segment is used as a breakpoint, and the current recognition segment is divided into multiple recognition sub-segments using the breakpoints, and the probability matrix of each recognition sub-segment is input into the decoder for decoding; if the current recognition segment does not contain a preset silent part, the probability matrix of the current recognition segment is directly input into the decoder for decoding.

[0120] An embodiment of the present invention provides a streaming speech recognition device, wherein the decoding module may include:

[0121] A calculation unit is used to: determine that the probability matrices corresponding to the first two recognition segments of the current recognition segment are a first probability matrix and a second probability matrix respectively; calculate that the matrix value of the first probability matrix is ​​a preset ratio, and determine a behavior sub-probability matrix of the preset ratio located at the end of the second probability matrix; splice the sub-probability matrix and the probability matrix of the current recognition matrix to obtain a new probability matrix; wherein the first probability matrix is ​​obtained earlier than the second probability matrix.

[0122] An embodiment of the present invention provides a streaming speech recognition device, wherein the decoding module may include:

[0123] A judgment unit is used to: detect the value of the start flag; if the value of the start flag is a first preset value, determine that the current recognition segment is the first recognition segment of the corresponding voice segment; if the value of the start flag is a second preset value, determine that the current recognition segment is not the first recognition segment of the corresponding voice segment; wherein, after receiving the designated recognition segment, the start flag is set to the second preset value, and the rest of the time, the start flag is set to the first preset value, and the designated recognition segment is a recognition segment containing a preset silent part.

[0124] An embodiment of the present invention provides a streaming speech recognition device, wherein the detection module may include:

[0125] A detection unit is used to: obtain multiple blank scores included in the result of silence detection; if there are any multiple consecutive blank scores that are greater than the score threshold, it is determined that the current recognition segment contains a preset silence part, and the part of the current recognition segment corresponding to the any multiple consecutive blank scores is the preset silence part; if there are not any multiple consecutive blank scores that are greater than the score threshold, it is determined that the current recognition segment does not contain the preset silence part.

[0126] An embodiment of the present invention provides a streaming speech recognition device, further comprising:

[0127] The response module is used to: detect the value of the end flag before performing silence detection on the current recognition segment; if the value of the end flag is a third preset value, initialize the corresponding parameters in the decoder after decoding the current recognition segment using the decoder; if the value of the end flag is a fourth preset value, execute the step of performing silence detection on the current recognition segment; wherein, after receiving the end instruction sent by the management terminal, the end flag is set to the third preset value, and the end flag is set to the fourth preset value at other times.

[0128] An embodiment of the present invention further provides a streaming speech recognition device, which may include:

[0129] memory for storing computer programs;

[0130] A processor is configured to implement the steps of any of the above streaming speech recognition methods when executing a computer program.

[0131] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of any of the above streaming speech recognition methods can be implemented.

[0132] It should be noted that for the description of the relevant portions of the streaming speech recognition apparatus, device, and storage medium provided in the embodiments of the present invention, please refer to the detailed description of the corresponding portions in the streaming speech recognition method provided in the embodiments of the present invention, and will not be repeated here. In addition, the portions of the above-mentioned technical solutions provided in the embodiments of the present invention that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive elaboration.

[0133] It should be pointed out that the above is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

[0134] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. Reference can be made to the descriptions of the identical or similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and the relevant parts can be referred to the descriptions of the methods.

[0135] The above is a detailed introduction to the streaming speech recognition method, device, equipment and computer-readable storage medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core ideas. At the same time, for general technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A streaming speech recognition method, characterized in that: include: receiving a voice stream, and determining that the currently received voice stream is a currently recognized segment; Performing silence detection on the current recognition segment and determining whether the current recognition segment contains a preset silence portion based on the result of the silence detection; wherein the preset silence portion is a portion whose duration is greater than a preset duration; If the current recognition segment includes the preset silent portion, decoding the current recognition segment using a decoder to obtain a corresponding speech recognition result, and initializing corresponding parameters in the decoder after decoding a portion of the current recognition segment preceding the preset silent portion; If the current recognition segment does not contain the preset silent portion, the current recognition segment is decoded by the decoder to obtain a corresponding speech recognition result, and the corresponding parameters in the decoder do not need to be initialized during the decoding of the current recognition segment; The decoder is used to decode the current recognition segment, including: Determine whether the current recognition segment is the first recognition segment of the corresponding speech segment; If the current recognition segment is the first recognition segment of the speech segment, the probability matrix of the current recognition segment is input into the decoder for decoding; wherein the probability matrix of any recognition segment is obtained by inputting the arbitrary recognition segment into the acoustic model trained by CTC; If the current recognition segment is not the first recognition segment of the speech segment to which it belongs, a new probability matrix is ​​calculated using the probability matrices of the two recognition segments before the current recognition segment and the probability matrix of the current recognition segment, and the new probability matrix is ​​input into the decoder for decoding as the probability matrix of the current recognition segment.

2. The method according to claim 1, characterized in that Inputting the probability matrix of the current recognition segment into the decoder for decoding includes: If the current recognition segment contains the preset silence portion, each of the preset silence portions contained in the current recognition segment is used as a breakpoint, the current recognition segment is divided into a plurality of recognition sub-segments using the breakpoints, and the probability matrix of each recognition sub-segment is input into the decoder for decoding; If the currently recognized segment does not contain the preset silent portion, the probability matrix of the currently recognized segment is directly input into the decoder for decoding.

3. The method according to claim 1, characterized in that The new probability matrix is ​​calculated using the probability matrices of the two recognition segments before the current recognition segment and the probability matrix of the current recognition segment, including: Determine that probability matrices corresponding to the first two recognition segments of the current recognition segment are a first probability matrix and a second probability matrix, respectively; wherein the first probability matrix is ​​obtained earlier than the second probability matrix; Calculating a matrix value of the first probability matrix to be a preset ratio, and determining a behavior sub-probability matrix of the preset ratio located at the tail of the second probability matrix; The sub-probability matrix and the probability matrix of the current recognition matrix are concatenated to obtain a new probability matrix.

4. The method according to claim 1, wherein Determining whether the current recognition segment is the first recognition segment of the corresponding speech segment includes: Check the value of the start flag; If the value of the start flag is a first preset value, determining that the current recognition segment is the first recognition segment of the speech segment; If the value of the start flag is a second preset value, it is determined that the current recognition segment is not the first recognition segment of the corresponding speech segment; wherein, after receiving the designated recognition segment, the start flag is set to the second preset value, and the rest of the time, the start flag is set to the first preset value, and the designated recognition segment is a recognition segment containing the preset silent part.

5. The method according to claim 2, characterized in that Based on the result of silence detection, it is determined whether the current recognition segment contains a preset silence section, including: Get multiple blank scores contained in the silence detection result; If there are any multiple consecutive blank scores that are all greater than the score threshold, it is determined that the current recognition segment contains the preset silent portion, and the portion in the current recognition segment corresponding to the any multiple consecutive blank scores is the preset silent portion; If there are not any multiple consecutive blank scores all greater than the score threshold, it is determined that the current recognition segment does not contain the preset silent portion.

6. The method according to any one of claims 2 to 5, characterized in that Before performing silence detection on the current recognition segment, the following steps are also included: Check the value of the end flag; If the value of the end flag is a third preset value, after decoding the current identification segment using the decoder, initializing corresponding parameters in the decoder; If the value of the end flag is the fourth preset value, the step of performing silence detection on the current identification segment is executed; wherein, after receiving the end instruction sent by the management terminal, the end flag is set to the third preset value, and the rest of the time, the end flag is set to the fourth preset value.

7. A streaming speech recognition device, characterized in that: include: The determination module is configured to: receive a voice stream and determine that the currently received voice stream is a currently recognized segment; a detection module configured to perform silence detection on the currently identified segment and determine, based on the result of the silence detection, whether the currently identified segment contains a preset silence portion; wherein the preset silence portion is a portion whose duration is greater than a preset duration; A first execution module is configured to: if the current recognition segment includes the preset silent portion, decode the current recognition segment using a decoder to obtain a corresponding speech recognition result, and initialize corresponding parameters in the decoder after decoding a portion of the current recognition segment preceding the preset silent portion; a second execution module, configured to: if the current recognition segment does not contain the preset silence portion, decode the current recognition segment using the decoder to obtain a corresponding speech recognition result, without initializing corresponding parameters in the decoder during the decoding of the current recognition segment; Among them, the first execution module and the second execution module both include a decoding module, which is used to determine whether the current recognition segment is the first recognition segment of the speech segment to which it belongs; if the current recognition segment is the first recognition segment of the speech segment to which it belongs, the probability matrix of the current recognition segment is input into the decoder for decoding; wherein, the probability matrix of any recognition segment is obtained by inputting the arbitrary recognition segment into the acoustic model trained by CTC; if the current recognition segment is not the first recognition segment of the speech segment to which it belongs, the probability matrices of the two recognition segments before the current recognition segment and the probability matrix of the current recognition segment are used to calculate a new probability matrix, and the new probability matrix is ​​input into the decoder for decoding as the probability matrix of the current recognition segment.

8. A streaming speech recognition device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the streaming speech recognition method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the streaming speech recognition method according to any one of claims 1 to 6.

Images