Speech recognition processing method, system, device and storage medium
By setting preset time and rescoring processing in speech recognition technology, the problems of endpoint delay and background noise in speech recognition are solved, and the accuracy and real-timeness of speech recognition are improved.
Patent Information
- Application Number
- CN202111527196.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-14
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2041-12-14
AI Technical Summary
In the existing speech recognition technology, the endpoint delay problem has not been effectively optimized, especially when background noise exists, which can easily lead to inaccurate judgment of speech endpoints and increase the delay in speech recognition processing.
By obtaining voice data for decoding and obtaining intermediate recognition results, the first preset time judgment voice endpoint is set, and rescoring is performed within the second preset time, and the rescoring result is output as the final recognition result to reduce background noise recognition The probability of word is improved to improve the accuracy of pronunciation endpoint judgment.
This reduces the problem of inaccurate judgment of voice endpoints due to background noise, reduces the endpoint delay of voice recognition processing, and improves the accuracy and real-timeness of voice recognition.
Smart Images

Figure CN114267339B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the field of voice interaction, and in particular to a voice recognition processing method and system, device and storage medium. Background Art
[0002] With the development of mobile Internet technology, technologies related to natural language processing, including speech recognition, have flourished. Real-time duplex voice interaction refers to voice interaction in which users and engines can send information streams to each other at any time / have two-way communication, especially long-term human-computer dialogue requests, such as online education.
[0003] For example, one scenario of online education is an AI (Artificial Intelligence) teacher class. In this scenario, voice recognition processing needs to process the user's audio in the background and return the results. Like other human-computer interaction scenarios, AI teachers need to simulate real people to have characteristics such as handling interruptions, responding at a reasonable time, and natural pauses. The indicator to measure the speed of AI teachers' responses can be marked as "end-to-end delay", which usually simulates the natural pauses in normal conversation scenarios at a reasonable pause time interval. If it is too fast, it will be interrupted, and if it is too slow, it will be delayed too fast and the response will be too slow.
[0004] End-to-end latency is the accumulation of a series of partial delays, including speech recognition processing (Automatic Speech Recognition, ASR), speech synthesis (Text to Speech, TTS), dialogue management (Dialog Management, DM), natural language understanding (Natural Language Understanding, NLU), etc. A dynamic and active waiting time is usually left to ensure that the average end-to-end latency is around the expected value. In speech recognition processing, the time difference from the timestamp of the end of the last word in a complete sentence to the time of receiving the final recognition result is defined as endpoint latency.
[0005] However, the endpoint latency of speech recognition still needs to be optimized. Summary of the invention
[0006] The problem solved by the embodiments of the present invention is to provide a speech recognition processing method and system, device and storage medium to improve the endpoint delay of speech recognition.
[0007] To solve the above problems, an embodiment of the present invention provides a speech recognition processing method, including: acquiring speech data; decoding the speech data to obtain an intermediate recognition result corresponding to the current word; when the pause length from the time the intermediate recognition result is obtained is a first preset length, judging that a first speech endpoint needs to be generated; when the pause length from the time the intermediate recognition result is obtained is a second preset length, performing a re-scoring process to obtain a re-scoring result; the second preset length is less than or equal to the first preset length; based on the first speech endpoint, outputting the re-scoring result as the final recognition result.
[0008] Correspondingly, an embodiment of the present invention also provides a speech recognition processing system, including: a speech acquisition module, used to acquire speech data; a decoding module, used to decode the speech data to obtain an intermediate recognition result corresponding to the current word; an endpoint judgment module, used to judge that a first speech endpoint needs to be generated when the pause length from the time the intermediate recognition result is obtained is a first preset length; a re-scoring module, used to perform a re-scoring process to obtain a re-scoring result when the pause length from the time the intermediate recognition result is obtained is a second preset length; the second preset length is less than or equal to the first preset length; a recognition result output module, used to output the re-scoring result as the final recognition result based on the first speech endpoint.
[0009] Correspondingly, an embodiment of the present invention also provides a device, comprising at least one memory and at least one processor, wherein the memory stores one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the speech recognition processing method provided by an embodiment of the present invention.
[0010] Correspondingly, an embodiment of the present invention further provides a storage medium, wherein the storage medium stores one or more computer instructions, and the one or more computer instructions are used to implement the speech recognition processing method provided by the embodiment of the present invention.
[0011] Compared with the prior art, the technical solution of the embodiment of the present invention has the following advantages:
[0012] In the speech recognition processing method provided by the embodiment of the present invention, when the pause time from the time when the intermediate recognition result is obtained is the first preset time, it is judged that the first speech endpoint needs to be generated; when the pause time from the time when the intermediate recognition result is obtained is the second preset time, a re-scoring process is performed to obtain a re-scoring result; wherein, the second preset time is less than or equal to the first preset time. Therefore, the embodiment of the present invention judges whether the current word is the last word of a sentence based on the time interval of continuous word output in the decoding process. Compared with the method of judging whether a speech endpoint needs to be generated based on the silence time of the audio, the decoding process is less sensitive to background noise, and the probability of recognizing background noise as a word is low, thereby reducing the probability of inaccurate speech endpoint judgment caused by background noise, which is correspondingly beneficial to improving the endpoint delay of the speech recognition process.
[0013] In the speech recognition processing system provided by the embodiment of the present invention, the endpoint judgment module is used to judge that the first speech endpoint needs to be generated when the pause time from the time when the intermediate recognition result is obtained is the first preset time; the re-scoring module is used to perform re-scoring processing and obtain a re-scoring result when the pause time from the time when the intermediate recognition result is obtained is the second preset time; the second preset time is less than or equal to the first preset time. Therefore, the embodiment of the present invention judges whether the current word is the last word of a sentence based on the time interval of continuous word output in the decoding process. Compared with the method of judging whether a speech endpoint needs to be generated based on the silence time of the audio, the decoding process is less sensitive to background noise, and the probability of identifying background noise as a word is low, thereby reducing the probability of inaccurate speech endpoint judgment caused by background noise, which is correspondingly beneficial to improving the endpoint delay of the speech recognition process. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] Figure 1 It is a flow chart of a speech recognition processing method;
[0015] Figure 2 is a flow chart of a first embodiment of a speech recognition processing method of the present invention;
[0016] Figure 3 is a flow chart of a second embodiment of a speech recognition processing method of the present invention;
[0017] Figure 4 Schematic diagram of the flow chart of the third embodiment of the speech recognition processing method of the present invention
[0018] Figure 5 is a functional block diagram of an embodiment of a speech recognition processing system of the present invention;
[0019] Figure 6 It is a hardware structure diagram of a device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0020] As can be seen from the background technology, the endpoint delay of current speech recognition still needs to be optimized. Now, in combination with a speech recognition processing method, the reason why the endpoint delay of current speech recognition still needs to be optimized is analyzed.
[0021] like Figure 1 As shown, a flow chart of a speech recognition processing method is shown.
[0022] The speech recognition processing method comprises:
[0023] Step s100: obtaining voice data;
[0024] Step s200: Taking the end time of the current word in the voice data as the starting point, when the silence time of the audio reaches the silence of a preset duration, a voice endpoint is generated;
[0025] Step s300: After the voice endpoint is generated, re-scoring is performed and the final recognition result is output.
[0026] The speech recognition processing method uses the end time of the current word in the speech data as the starting point to detect when the silence time of the audio reaches a preset length of silence. If background noise is mixed in, it is easy for the silence timing to start after the background noise ends, which may easily lead to inaccurate judgment of the speech endpoint and increase the endpoint delay of the speech recognition processing.
[0027] In order to solve the above technical problem, an embodiment of the present invention provides a speech recognition processing method. Figure 2 , shows a flow chart of an embodiment of a speech recognition processing method of the present invention. In this embodiment, the speech recognition processing method includes the following basic steps:
[0028] Step S1a: Acquire voice data;
[0029] Step S2a: Decode the speech data to obtain an intermediate recognition result corresponding to the current word;
[0030] Step S3a: when the pause time from when the intermediate recognition result is obtained is a first preset time, determining that a first voice endpoint needs to be generated;
[0031] Step S4a: When the pause time from the time when the intermediate recognition result is obtained is a second preset time, re-scoring is performed to obtain a re-scoring result; the second preset time is less than or equal to the first preset time;
[0032] Step S5a: Based on the first voice endpoint, output the re-scoring result as the final recognition result.
[0033] In the speech recognition processing method provided by the embodiment of the present invention, a judgment is made as to whether the current word is the last word of a sentence based on the time interval between consecutive words output in the decoding processing. Compared with the method of judging whether a speech endpoint needs to be generated based on the silent time of the audio, the decoding processing is less sensitive to background noise, and the probability of recognizing background noise as a word is low, thereby reducing the probability of inaccurate speech endpoint judgment caused by background noise, which is correspondingly beneficial to reducing the endpoint delay of the speech recognition processing.
[0034] In order to make the above-mentioned purposes, features and advantages of the embodiments of the present invention more obvious and understandable, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.
[0035] refer to Figure 2 , Step S1a: Acquire voice data so as to perform recognition processing on the voice data.
[0036] As an embodiment, the application scenario of the speech recognition processing method can be an AI teacher class in online education, so the user's voice data is obtained in real time to process the user's audio throughout the process and return the result.
[0037] Specifically, the voice data is voice stream data input in real time, or file stream data in an audio file. The voice data can be data directly received and processed, or data obtained from other terminals.
[0038] Among them, voice stream data can usually be obtained by real-time recording of voice through hardware with real-time recording function such as microphones and sound cards. File stream data can usually be obtained by reading audio files that store recorded audio data. The format of the audio file can be arbitrary, including but not limited to .WAV, .MP3, etc.
[0039] Continue to refer Figure 2 , step S2a: decoding the speech data to obtain an intermediate recognition result (partial result) corresponding to the current word.
[0040] In the scenario of real-time speech recognition processing, the intermediate results of speech recognition are obtained during the real-time decoding process of speech data, and the intermediate results are thrown to the user so that they can be quickly displayed on the screen, so that the user can feel that subtitles are popping up during the conversation, thereby optimizing the user's real-time experience. In addition, by obtaining the intermediate recognition results, the intermediate recognition results can also be passed to the downstream modules during the real-time decoding process, so that the downstream modules can perform corresponding processing in advance, thereby saving the calculation time of the downstream modules and reducing the delay of the downstream modules.
[0041] In this embodiment, decoding the speech data includes: decoding the speech data with a fixed decoding window length as a basic unit. In a specific implementation, the speech data may be decoded by a decoding model to generate an intermediate result of speech recognition. As an embodiment, the decoding model may be composed of an acoustic model, a dictionary, and a language model.
[0042] It should be noted that the decoding method is not limited to this. In other embodiments, other methods can be used to decode the voice data.
[0043] Among them, the mapping between speech features and phonemes in speech data can be established through the acoustic model; the mapping between phonemes and words can be established through the dictionary; and the mapping between words and words, and words and sentences can be established through the language model. According to the mapping established by the acoustic model, dictionary and language model, the speech data is decoded to generate the corresponding speech recognition intermediate result.
[0044] Specifically, the acoustic model is a knowledge representation of differences in acoustics, phonetics, environmental variables, speaker gender, accent, etc.; the language model is a knowledge representation of a set of word sequences; and the dictionary is a set of phoneme indexes corresponding to words.
[0045] In a specific implementation, the intermediate recognition result includes the text corresponding to the speech and text-related information. The text may be Chinese text, English text, or both, and may include text in other languages; the text-related information may include the timestamp corresponding to the text in the speech data, that is, the start time and end time of the audio corresponding to the text.
[0046] The text-related information may also generally include various information including confidence, word attributes, etc., which will not be described in detail in this embodiment.
[0047] Specifically, in the process of real-time decoding of speech recognition, the intermediate results are output one by one, and each intermediate result and the output order of the intermediate results are recorded accordingly. Taking the user speaking a complete sentence as an example, there are usually multiple intermediate results of speech recognition.
[0048] As an example, in the voice data, the user said "I have a dream", and the intermediate result set usually returned is:
[0049] Partial result: "I";
[0050] Partial result: "I have";
[0051] Partial result: "I have a";
[0052] Partial result: "I have a dream".
[0053] In the above example, if the current voice data is a complete sentence spoken by the user, the current word is "dream", and the corresponding intermediate recognition result is "I have a dream".
[0054] Step S3a: When the pause duration from when the intermediate recognition result is obtained is equal to a first preset duration, it is determined that a first voice endpoint needs to be generated.
[0055] When the pause time from the time the intermediate result is obtained is the first preset time, it means that the user has finished a complete sentence, and then it is determined that a first voice endpoint needs to be generated, so as to serve as a basis for determining whether the re-scoring result needs to be output as the final recognition result.
[0056] In this embodiment, the time when the intermediate recognition result is obtained is used as the starting point of the pause duration to determine whether it is necessary to generate the first voice endpoint, so as to determine whether the current word is the last word of a complete sentence based on the time interval of continuous word output in the decoding process. Compared with the method of determining whether a voice endpoint needs to be generated based on the audio-based silence time, the decoding process is less sensitive to background noise, and the probability of recognizing background noise as a word is low, thereby reducing the probability of inaccurate voice endpoint judgment caused by background noise, which is correspondingly beneficial to reducing the endpoint delay of the voice recognition process.
[0057] In a specific implementation, the first preset duration is set based on the natural pause time of the user speaking.
[0058] Accordingly, the first preset time length should not be too small or too large. If the first preset time length is too small, it is easy to make inaccurate judgments on the first voice endpoint, resulting in the judgment that the first voice endpoint needs to be generated before the user has finished a complete sentence; if the first preset time length is too large, it is easy to increase the endpoint delay time, and it is easy to result in too few words that meet the requirement that the pause time length from the time the intermediate result is obtained is the first preset time length, which in turn easily leads to missed judgments on the first voice endpoint. As an example, the first preset time length is 0.9S to 1.1S. For example: the first preset time length can be set to 1S, which is closer to the natural pause time of the user's speech.
[0059] In other embodiments, based on actual application scenarios and requirements, the first preset duration may also be set to other appropriate durations.
[0060] In a specific implementation, the intermediate recognition result includes text-related information, for example: the timestamp corresponding to the text in the voice data. Accordingly, from the time when the intermediate recognition result is obtained, by judging whether the time stamp difference between adjacent words meets the first preset time length, it is judged whether the pause time from the time when the intermediate recognition result is obtained is the first preset time length.
[0061] Continue to refer Figure 2 , step S4a: when the pause time from the time when the intermediate recognition result is obtained is a second preset time, re-scoring is performed to obtain a re-scoring result; wherein the second preset time is less than or equal to the first preset time.
[0062] Re-score the intermediate recognition results, so as to use the language model with better performance, re-evaluate the candidate intermediate recognition results, and update the final output result ranking to give more accurate recognition results, thereby improving the accuracy of speech recognition processing.
[0063] The re-scoring process is performed to obtain a re-scoring result, so that when it is determined that a voice endpoint needs to be generated, the re-scoring result can be output as the final recognition result. In other words, the re-scoring result is used as a candidate for the final recognition result.
[0064] In this embodiment, starting from the time when the intermediate recognition result corresponding to the current word is generated, the pause time of the recognized word is calculated to the second preset time length, and then a re-scoring process is performed, so that the time interval between the intermediate recognition results of adjacent words can be judged from the perspective of decoding processing.
[0065] As an example, an RNN (Recurrent Neural Networks) language model is used to re-score the intermediate recognition results. In other embodiments, other language models (such as Large N-gram, etc.) can also be used to re-score the intermediate recognition results.
[0066] The second preset time length is less than or equal to the first preset time length, that is, compared with the second preset time length, the first preset time length is longer, that is to say, the pause time for judging whether the user has finished a complete sentence is longer, thereby improving the accuracy of judging whether the user has finished a complete sentence, and correspondingly improving the accuracy of judging the first voice endpoint, thereby improving the accuracy of the final recognition result output.
[0067] In this embodiment, the second preset time length is equal to the first preset time length as an example for description.
[0068] Accordingly, if Figure 2As shown, in this embodiment, in step S3a: after determining that the first voice endpoint needs to be generated, step S4a is executed: re-scoring is performed to obtain a re-scoring result.
[0069] It should be noted that, in this embodiment, the second preset duration is equal to the first preset duration as an example for description. In other embodiments, the second preset duration may also be less than the second preset duration, and accordingly, the order of generating the first voice endpoint and performing the re-scoring process may also be swapped.
[0070] Continue to refer Figure 2 , Step S5a: Based on the first voice endpoint, output the re-scoring result as the final recognition result. When it is determined that the first voice endpoint needs to be generated, it is determined that the user has finished speaking a complete sentence, and then the re-scoring result can be returned as the final recognition result.
[0071] The final recognition result is the final recognition result of a complete sentence spoken by the user, and is returned to the downstream module for processing. For example, the final recognition result is returned to the natural language understanding module for corresponding processing.
[0072] As an example, in the voice data, the user says "I have a dream", and the final recognition result returned is: "I have a dream".
[0073] In the field of speech recognition processing, the time difference between the timestamp of the last word of a complete sentence and the time when the final recognition result is received is defined as endpoint latency.
[0074] In this embodiment, the time for obtaining the intermediate recognition result is used as the starting point of the pause time to judge the voice endpoint, so as to judge whether the current word is the last word of a sentence based on the time interval of continuous word output in the decoding process, thereby reducing the problem of inaccurate voice endpoint judgment caused by background noise, which is correspondingly beneficial to reducing the endpoint delay of the voice recognition process.
[0075] In this embodiment, the second preset time length is equal to the first preset time length. Therefore, based on the first voice endpoint, outputting the re-scoring result as the final recognition result includes: when obtaining the re-scoring result, outputting the re-scoring result as the final recognition result.
[0076] It should be noted that the reference Figure 2 In this embodiment, the speech recognition processing method further includes:
[0077] Step S6a: Taking the end time of the current word as the starting point, when it is determined that the silence time of the voice data reaches the first preset duration, a second voice endpoint is generated; there is an endpoint interval between the time when the second voice endpoint is generated and the end time of the current word.
[0078] In a specific implementation, a voice activity detection (VAD) module may be used to determine whether the silence time reaches a second preset duration to determine whether a second voice endpoint needs to be generated.
[0079] In the field of speech recognition processing, generally speaking, the sound activity detection module is accurate in judging the silence time of speech data. Therefore, by also judging whether the silence time of speech data reaches the second preset duration to judge whether the second speech endpoint is generated, it is possible to use existing technology to judge the speech endpoints in most cases, which is conducive to simplifying the calculation.
[0080] However, in the field of speech recognition, since the sound activity detection module judges the silent time from the audio perspective, when background noise is mixed in the user's voice data, it is easy to start judging the silent time when the background noise ends, which can easily lead to inaccurate judgment of the voice endpoint and cause an unexpected increase in endpoint delay.
[0081] Therefore, in this embodiment, the speech recognition processing method further includes: Step S7a: determining whether the endpoint interval is greater than the first preset time length.
[0082] When the endpoint interval is greater than the first preset time length, executing step S5a: outputting a re-scoring result as a final recognition result based on the first voice endpoint;
[0083] When the endpoint interval is less than or equal to the first preset time length, execute step S8a: perform re-scoring processing based on the second voice endpoint, and output the re-scoring result as the final recognition result.
[0084] Therefore, in this embodiment, the time when the current word ends is used as the starting point, and whether the second voice endpoint needs to be generated is determined by judging whether the silence time of the voice data reaches the second preset duration. In addition, the pause duration is detected from the time when the intermediate recognition result is obtained as the starting point to judge whether the first voice endpoint needs to be generated. In this way, the two judgment methods can be turned on at the same time and can complement each other. By comparing the endpoint interval with the first preset duration, it is determined whether to generate an endpoint in a way that satisfies the endpoint generation condition earlier. Accordingly, while ensuring the accuracy of endpoint judgment, it is also possible to prevent the increase of endpoint delay.
[0085] It should be noted that for the scenario where background noise is mixed in the voice data, by pausing for a first preset time from the time the intermediate recognition result is obtained, it is determined that a first voice endpoint needs to be generated. When the method of determining the second voice endpoint by determining the silence time of the voice data fails, the voice endpoint can be accurately determined, which is correspondingly beneficial to reducing the variance of the endpoint delay.
[0086] Figure 3 The second embodiment of the speech recognition processing method of the present invention is a flow chart of the second embodiment of the speech recognition processing method of the present invention. The similarities between the speech recognition processing method of the present embodiment and the above embodiment are not repeated here. The difference between the speech recognition processing method of the present embodiment and the above embodiment is that the second preset time length is less than the first preset time length.
[0087] The following combination Figure 3 , the steps of the speech recognition processing method of this embodiment are described in detail.
[0088] Step S1b: Acquire voice data.
[0089] Step S2b: Decode the speech data to obtain an intermediate recognition result corresponding to the current word.
[0090] Step S3b: When the pause duration from the time when the intermediate recognition result is obtained is equal to the first preset duration, it is determined that a first voice endpoint needs to be generated.
[0091] Step S4b: When the pause time from when the intermediate recognition result is obtained is a second preset time, a re-scoring process is performed to obtain a re-scoring result.
[0092] In this embodiment, the second preset time length is smaller than the first preset time length. Accordingly, the triggering timing for performing the re-scoring process does not depend on the generation of the first voice endpoint. Instead, the re-scoring process can be started before the generation of the first voice endpoint, thereby advancing the time for performing the re-scoring process. Furthermore, the steps of determining whether it is necessary to generate the first voice endpoint and performing the re-scoring process are performed in parallel. Thus, the re-scoring process is performed within the waiting time of the first preset time length, and the time required for the re-scoring process can occupy part of the first preset time length, which is correspondingly beneficial to further reduce endpoint delay.
[0093] It should be noted that the second preset time length should not be too small or too large. If the second preset time length is too small, the time interval between the decoding process and the adjacent word recognition results is too small, which may easily lead to too high a frequency of triggering the re-scoring process, and thus increase the amount of calculation; if the second preset time length is too large, it may easily lead to too short a time interval between the start of the re-scoring process and the generation of the first voice endpoint, which may easily lead to the further improvement of the endpoint delay being not obvious. For this reason, in this embodiment, the second preset time length is 40% to 90% of the first preset time length.
[0094] As an example, the first preset time length is 1S, and the second preset time length can be set to 0.6S, so that the variance of endpoint delay can be significantly improved while the frequency of triggering the re-scoring process is controlled within a reasonable range.
[0095] As an embodiment, the time required for the re-scoring process is less than or equal to the difference between the second preset time length and the first preset time length, so that the re-scoring result is obtained before or when the first voice endpoint is generated. Accordingly, the sum of the second preset time length and the time required for the re-scoring process can be controlled within the range of the first preset time length. Accordingly, after the first voice endpoint is generated, no additional time is required for the re-scoring process, which is conducive to further improving endpoint delay and correspondingly improving the variance of endpoint delay.
[0096] Step S5b: Based on the first voice endpoint, output the re-scoring result as the final recognition result.
[0097] Among them, when the time required for the re-scoring process is less than or equal to the difference between the second preset time length and the first preset time length, the re-scoring result can be output as the final recognition result when the first voice endpoint is generated.
[0098] Figure 4 It is a flow chart of the third embodiment of the speech recognition processing method of the present invention. The similarities between the speech recognition processing method of this embodiment and the second embodiment are not repeated here. The difference between the speech recognition processing method of this embodiment and the aforementioned embodiment is that: a target endpoint delay time is set between the time point of outputting the final recognition result and the time point of the end of the current word.
[0099] Step S1c: Acquire voice data.
[0100] Step S2c: Decode the speech data to obtain an intermediate recognition result corresponding to the current word. Decoding the speech data includes: decoding the speech data with a fixed decoding window length (Decoder chunk size) as a basic unit.
[0101] Step S3c: When the pause duration from when the intermediate recognition result is obtained is equal to a first preset duration, it is determined that a first voice endpoint needs to be generated.
[0102] Step S4c: When the pause time from the time when the intermediate recognition result is obtained is a second preset time, a re-scoring process is performed to obtain a re-scoring result; the second preset time is shorter than the first preset time.
[0103] The time interval between the time point of obtaining the re-scoring result and the end time point of the current word is defined as the re-scoring time interval.
[0104] In this embodiment, the second preset time length is the fixed decoding window length, so that the fixed decoding window length is used as a basic unit. From the time of the intermediate recognition result, if the recognition result is not updated within the fixed decoding window length, it is determined that no word is recognized within the fixed decoding window length. In other embodiments, the second preset time length can also be set to other values.
[0105] It should be noted that the fixed decoding window length should not be too small or too large. If the fixed decoding window length is too small, it is easy to trigger re-scoring frequently; if the fixed decoding window length is too large, it is easy to cause the re-scoring time interval to be too large, and then it is easy to cause the time interval between the time point of outputting the final recognition result and the time point of the end of the current word to be too large, that is, the endpoint latency is difficult to meet the target endpoint latency requirement. For this reason, in this embodiment, the fixed decoding window length is 30% to 50% of the target endpoint latency. .
[0106] As an embodiment, the fixed decoding window length is 400 mS. In other embodiments, the fixed decoding window length may also be set to other values.
[0107] In this embodiment, the speech recognition processing method further includes:
[0108] Step S6c: Determine whether the re-scoring time interval is less than the target endpoint delay time.
[0109] If yes, execute step S7c: when the re-scoring result is obtained, generate a first voice endpoint with a dynamic window length; the dynamic window length is the difference between the target endpoint delay time and the re-scoring time interval.
[0110] By determining that the re-scoring time interval is less than the target endpoint delay duration, a first voice endpoint is generated based on the interval dynamic window length starting from the time the re-scoring result is obtained, thereby limiting the endpoint delay to the target endpoint delay duration, and correspondingly further improving the endpoint delay and the variance of the endpoint delay.
[0111] Step S5c: Based on the first voice endpoint, output the re-scoring result as the final recognition result. In this embodiment, based on the first voice endpoint, outputting the re-scoring result as the final recognition result includes: when the first voice endpoint is generated, outputting the re-scoring result as the final recognition result.
[0112] In order to solve the above problem, an embodiment of the present invention further provides a speech recognition processing system. Figure 5 , showing a functional block diagram of an embodiment of a speech recognition processing system of the present invention.
[0113] In this embodiment, the speech recognition processing system includes: a speech acquisition module 101, which is used to acquire speech data; a decoding module 102, which is used to decode the speech data to obtain an intermediate recognition result corresponding to the current word; an endpoint judgment module 103, which is used to judge that a first speech endpoint needs to be generated when the pause length from the time when the intermediate recognition result is obtained is a first preset length; a re-scoring module 104, which is used to perform a re-scoring process to obtain a re-scoring result when the pause length from the time when the intermediate recognition result is obtained is a second preset length; the second preset length is less than or equal to the first preset length; a recognition result output module 105, which is used to output the re-scoring result as a final recognition result based on the first speech endpoint.
[0114] In the speech recognition processing system, the endpoint judgment module 103 is used to judge that the first speech endpoint needs to be generated when the pause time from the time when the intermediate recognition result is obtained is the first preset time; the re-scoring module 104 is used to perform re-scoring processing and obtain a re-scoring result when the pause time from the time when the intermediate recognition result is obtained is the second preset time; the second preset time is less than or equal to the first preset time. Therefore, this embodiment judges whether the current word is the last word of a sentence based on the time interval of continuous word output in the decoding process. Compared with the method of judging whether a speech endpoint needs to be generated based on the silent time of the audio, the decoding process is less sensitive to background noise, and the probability of recognizing background noise as a word is low, thereby reducing the problem of inaccurate speech endpoint judgment caused by background noise, which is correspondingly conducive to improving the endpoint delay of the speech recognition process.
[0115] The voice acquisition module 101 acquires the user's voice data so as to perform recognition processing on the voice data.
[0116] As an embodiment, the application scenario of the speech recognition processing system can be an AI teacher class in online education, so the user's voice data is obtained in real time to process the user's audio throughout the process and return the result.
[0117] Specifically, the voice data is voice stream data input in real time, or file stream data in an audio file. The voice data can be data directly received and processed, or data obtained from other terminals.
[0118] Among them, voice stream data can usually be obtained by real-time recording of voice through hardware with real-time recording function such as microphones and sound cards. File stream data can usually be obtained by reading audio files that store recorded audio data. The format of the audio file can be arbitrary, including but not limited to .WAV, .MP3, etc.
[0119] The decoding module 102 is used to decode the voice data to obtain an intermediate recognition result corresponding to the current word. In the scenario of real-time voice recognition processing, the decoding module 102 obtains the intermediate result of voice recognition during the real-time decoding process of the voice data, and throws the intermediate result to the user so that it can be quickly displayed on the screen, so that the user can feel that subtitles are popping up during the conversation, thereby optimizing the user's real-time experience in voice recognition processing.
[0120] Moreover, by obtaining intermediate recognition results, the intermediate recognition results can be passed to downstream modules during real-time decoding processing, so that the downstream modules can perform corresponding processing in advance, thereby saving the calculation time of the downstream modules and reducing the delay of the downstream modules.
[0121] The decoding module 102 decodes the speech data, including: decoding the speech data with a fixed decoding window length as a basic unit. In a specific implementation, the decoding module 102 can decode the speech data through a decoding model to generate an intermediate result of speech recognition. As an embodiment, the decoding model can be composed of an acoustic model, a dictionary, and a language model.
[0122] It should be noted that the decoding module is not limited to the above method. In other embodiments, other methods may be used to decode the voice data.
[0123] Among them, the mapping between speech features and phonemes in speech data can be established through the acoustic model; the mapping between phonemes and words can be established through the dictionary; and the mapping between words and words, and words and sentences can be established through the language model. According to the mapping established by the acoustic model, dictionary and language model, the speech data is decoded to generate the corresponding speech recognition intermediate result.
[0124] Specifically, the acoustic model is a knowledge representation of differences in acoustics, phonetics, environmental variables, speaker gender, accent, etc.; the language model is a knowledge representation of a set of word sequences; and the dictionary is a set of phoneme indexes corresponding to words.
[0125] In a specific implementation, the intermediate recognition result includes the text corresponding to the speech and text-related information. The text may be Chinese text, English text, or both, and may include text in other languages; the text-related information may include the timestamp corresponding to the text in the speech data, that is, the start time and end time of the audio corresponding to the text.
[0126] The text-related information may also generally include various information including confidence, word attributes, etc., which will not be described in detail in this embodiment.
[0127] Specifically, during the real-time decoding process of the decoding module 102, the intermediate results are output one by one, and each intermediate result and the output order of the intermediate results are recorded accordingly. Taking the user speaking a complete sentence as an example, there are usually multiple intermediate results of speech recognition.
[0128] As an example, in the voice data, the user said "I have a dream", and the intermediate result set usually returned is:
[0129] Partial result: "I";
[0130] Partial result: "I have";
[0131] Partial result: "I have a";
[0132] Partial result: "I have a dream".
[0133] In the above example, if the current voice data is a complete sentence spoken by the user, the current word is "dream", and the corresponding intermediate recognition result is "I have a dream".
[0134] The endpoint judgment module 103 is used to determine whether the pause time from the time the intermediate result is obtained is a first preset time, which means that the user has finished a complete sentence, and then determine whether a first voice endpoint needs to be generated, so as to serve as a judgment basis for whether the re-scoring result needs to be output as the final recognition result.
[0135] In this embodiment, the endpoint judgment module 103 uses the time when the intermediate recognition result is obtained as the starting point of the pause duration to determine whether the first voice endpoint needs to be generated, thereby judging whether the current word is the last word of a complete sentence based on the time interval of continuous word output by the decoding module 102 for decoding processing. Compared with the method of judging whether a voice endpoint is generated based on the audio-based silence time, the decoding process is less sensitive to background noise, and the probability of recognizing background noise as a word is low, thereby improving the problem of inaccurate voice endpoint judgment caused by background noise, and correspondingly reducing the endpoint delay of the voice recognition process.
[0136] In a specific implementation, the first preset duration is set based on the natural pause time of the user speaking.
[0137] Accordingly, the first preset time length should not be too small or too large. If the first preset time length is too small, it is easy to make inaccurate judgments on the first voice endpoint, resulting in the judgment that the first voice endpoint needs to be generated before the user has finished a complete sentence; if the first preset time length is too large, it is easy to increase the endpoint delay time, and it is easy to result in too few words that meet the requirement that the pause time length from the time the intermediate result is obtained is the first preset time length, which in turn easily leads to missed judgments on the first voice endpoint. As an example, the first preset time length is 0.9S to 1.1S. For example: the first preset time length can be set to 1S, which is closer to the natural pause time of the user's speech.
[0138] In other embodiments, based on actual application scenarios and requirements, the first preset duration may also be set to other appropriate durations.
[0139] In a specific implementation, the intermediate recognition result includes text-related information, for example: the timestamp corresponding to the text in the voice data. Accordingly, from the time when the intermediate recognition result is obtained, by judging whether the time stamp difference between adjacent words meets the first preset time length, it is judged whether the pause time from the time when the intermediate recognition result is obtained is the first preset time length.
[0140] The re-scoring module 104 re-scores the intermediate recognition results, thereby using a language model with better performance, re-evaluating the candidate intermediate recognition results, and updating the final output result ranking to provide a more accurate recognition result, thereby improving the accuracy of the speech recognition process.
[0141] The re-scoring module 104 performs re-scoring processing to obtain a re-scoring result, so that when it is determined that a voice endpoint needs to be generated, the re-scoring result can be output as a final recognition result. In other words, the re-scoring result is used as a candidate for the final recognition result.
[0142] In this embodiment, the re-scoring module 104 calculates the pause time of the recognized word from the time when the intermediate recognition result corresponding to the current word is generated to the second preset time length, and then performs a re-scoring process, so that the time interval between the intermediate recognition results of adjacent words can be judged from the perspective of decoding processing.
[0143] As an example, the rescoring module 104 may use an RNN language model to rescore the intermediate recognition result. In other embodiments, the rescoring module may also use other language models (eg, Large N-gram, etc.) to rescore the intermediate recognition result.
[0144] The second preset time length is less than or equal to the first preset time length, that is, compared with the second preset time length, the first preset time length is longer, that is to say, the pause time for judging whether the user has finished a complete sentence is longer, thereby improving the accuracy of judging whether the user has finished a complete sentence, and correspondingly improving the accuracy of judging the first voice endpoint, thereby improving the accuracy of the final recognition result output.
[0145] In this embodiment, the second preset time length is equal to the first preset time length as an example for description.
[0146] Accordingly, in this embodiment, after the endpoint determination module 103 determines that the first voice endpoint needs to be generated, the re-scoring module 104 is triggered to perform a re-scoring process to obtain a re-scoring result.
[0147] It should be noted that, in this embodiment, the second preset duration is equal to the first preset duration as an example for description. In other embodiments, the second preset duration may also be less than the second preset duration, and accordingly, the order of generating the first voice endpoint and performing the re-scoring process may also be swapped.
[0148] The recognition result output module 105, when determining that the first voice endpoint needs to be generated, accordingly determines that the user has finished speaking a complete sentence, and then can return the re-scoring result as the final recognition result.
[0149] The final recognition result is the final recognition result of a complete sentence spoken by the user, and is returned to the downstream module for processing. For example, the final recognition result is returned to the natural language understanding module for corresponding processing.
[0150] As an example, if the user says "I have a dream", the final recognition result (Final result) returned is: "I have a dream".
[0151] In the field of speech recognition processing, the time difference between the timestamp of the last word of a complete sentence and the time when the final recognition result is received is defined as endpoint latency.
[0152] In this embodiment, the endpoint judgment module 103 uses the time of obtaining the intermediate recognition result as the starting point of the pause time to judge the speech endpoint, and thus judges whether the current word is the last word of a sentence based on the time interval of continuous word output in the decoding process, thereby reducing the problem of inaccurate speech endpoint judgment caused by background noise, which is correspondingly beneficial to reducing the endpoint delay of the speech recognition process.
[0153] In this embodiment, the second preset time length is equal to the first preset time length. Therefore, when the recognition result output module 105 obtains the re-scoring result, it outputs the re-scoring result as the final recognition result.
[0154] It should be noted that, in this embodiment, the speech recognition processing system also includes: a voice activity detection (VAD) module 106, which is used to take the time when the current word ends as the starting point, and when the silence time of the voice data reaches a first preset duration, generate a second voice endpoint; there is an endpoint interval between the time when the second voice endpoint is generated and the time when the current word ends.
[0155] In the field of speech recognition processing, generally speaking, the sound activity detection module 106 is accurate in judging the silence time of speech data. Therefore, by also judging whether the silence time of speech data reaches the second preset time length to judge whether the second speech endpoint is generated, the existing technology can be used to judge the speech endpoints in most cases, which is conducive to simplifying the calculation.
[0156] However, in the field of speech recognition, since the sound activity detection module 106 judges the silent time from the perspective of audio, when background noise is mixed in the user's voice data, it is easy to start judging the silent time when the background noise ends, which can easily lead to inaccurate judgment of the voice endpoint and cause an unexpected increase in endpoint delay.
[0157] Therefore, the speech recognition processing system further includes: a comparison module 107, which is used to compare the relative size of the endpoint interval and the first preset time length.
[0158] When the endpoint interval is greater than the first preset time length, the recognition result output module 105 outputs the re-scoring result as the final recognition result based on the first voice endpoint.
[0159] When the endpoint interval is less than or equal to the first preset duration, the re-scoring module 104 is used to perform a re-scoring process based on the second voice endpoint; the recognition result output module 105 outputs the re-scoring result as the final recognition result.
[0160] Therefore, in this embodiment, not only does the sound activity detection module 106 use the time when the current word ends as the starting point to determine whether it is necessary to generate a second voice endpoint by determining whether the silence time of the voice data reaches a second preset duration, but the endpoint judgment module 103 also uses the time when the intermediate recognition result is obtained as the starting point to detect the pause duration to determine whether it is necessary to generate a first voice endpoint, so that the two judgment methods can be turned on at the same time and complement each other. The comparison module 107 determines whether to generate an endpoint in a way that satisfies the endpoint generation condition earlier by comparing the relative size of the endpoint interval with the first preset duration. Accordingly, while ensuring the accuracy of the endpoint judgment, it can also prevent the increase of endpoint delay.
[0161] It should be noted that for the scenario where background noise is mixed in the voice data, the endpoint judgment module 103 judges that a first voice endpoint needs to be generated when the pause time from the time the intermediate recognition result is obtained is a first preset time. When the method of judging the second voice endpoint by judging the silence time of the voice data fails, the voice endpoint can be accurately judged, which is correspondingly beneficial to reduce the variance of the endpoint delay.
[0162] It should be noted that, in this embodiment, the second preset time length is equal to the first preset time length as an example for description.
[0163] In another embodiment of the speech recognition processing system provided by the present invention, the second preset time length may also be smaller than the first preset time length.
[0164] The second preset time duration is smaller than the first preset time duration. Accordingly, the triggering timing of the re-scoring module for re-scoring processing does not depend on the generation of the first voice endpoint, but the re-scoring processing can be started before the first voice endpoint is generated, thereby advancing the time for the re-scoring processing, and then the steps of the endpoint judgment module judging whether it is necessary to generate the first voice endpoint and the re-scoring module performing the re-scoring processing are performed in parallel, so that the re-scoring processing operation is performed within the waiting time of the first preset time duration, and the time required for the re-scoring processing can occupy part of the first preset time duration, which is correspondingly beneficial to further reduce the endpoint delay.
[0165] It should be noted that the second preset time length should not be too small or too large. If the second preset time length is too small, the time interval between the decoding module's decoding processing and the adjacent word recognition results is too small, which may easily lead to too high a frequency of triggering the re-scoring process, and thus easily increase the amount of calculation; if the second preset time length is too large, it may easily lead to too short a time interval between the start of the re-scoring process and the generation of the first voice endpoint, which may easily lead to the further improvement of the endpoint delay being not obvious. For this reason, as an embodiment, the second preset time length is 40% to 90% of the first preset time length.
[0166] As an example, the first preset time length is 1S, and the second preset time length can be set to 0.6S, so that the variance of endpoint delay can be significantly improved while the frequency of triggering the re-scoring process is controlled within a reasonable range.
[0167] As one embodiment, the time required for the re-scoring module to perform the re-scoring process is less than or equal to the difference between the second preset time length and the first preset time length, so that the re-scoring result is obtained before the endpoint generation module generates the first voice endpoint or when the first voice endpoint is generated, so that the recognition result output module outputs the re-scoring result as the final recognition result when the first endpoint is generated. Accordingly, the sum of the second preset time length and the time required for the re-scoring process can be controlled within the range of the first preset time length. Accordingly, after the endpoint judgment module generates the first voice endpoint, no additional time is required for the re-scoring process, which is conducive to further improving the endpoint delay and correspondingly improving the variance of the endpoint delay.
[0168] In another embodiment of the speech recognition processing system provided by the present invention, when the second preset time length is less than the first preset time length, the speech recognition system further sets a target endpoint delay time length between the time point of outputting the final recognition result and the time point when the current word ends, and defines the time interval between the time point when the re-scoring result is obtained and the time point when the current word ends as the re-scoring time interval.
[0169] As an embodiment, the second preset time length is the fixed decoding window length, so that the fixed decoding window length is used as a basic unit, and from the time of the intermediate recognition result, if the recognition result is not updated within the fixed decoding window length, it is determined that no word is recognized within the fixed decoding window length. In other embodiments, the second preset time length can also be set to other values.
[0170] It should be noted that the fixed decoding window length should not be too small or too large. If the fixed decoding window length is too small, it is easy to trigger re-scoring frequently; if the fixed decoding window length is too large, it is easy to cause the re-scoring time interval to be too large, and then it is easy to cause the time interval between the time point of outputting the final recognition result and the time point of the end of the current word to be too large, that is, the endpoint delay is difficult to meet the target endpoint delay time requirement. For this reason, in this embodiment, the fixed decoding window length is 30% to 50% of the target endpoint delay time.
[0171] As an embodiment, the fixed decoding window length is 400 mS. In other embodiments, the fixed decoding window length may also be set to other values.
[0172] The speech recognition processing system also includes: an interval judgment module, used to judge whether the re-scoring time interval is less than the target endpoint delay duration; a dynamic window length insertion module, used to generate the first speech endpoint with an interval dynamic window length starting from the time the re-scoring result is obtained when the re-scoring time interval is less than the target endpoint delay duration; wherein the dynamic window length is: the difference between the target endpoint delay duration and the re-scoring time interval.
[0173] The dynamic window length insertion module generates a first voice endpoint with an interval dynamic window length when the re-scoring time interval is less than the target endpoint delay duration when the re-scoring result is obtained, thereby limiting the endpoint delay to the target endpoint delay duration, and correspondingly further improving the endpoint delay and the variance of the endpoint delay.
[0174] Correspondingly, the recognition structure output module outputs the re-scoring result as the final recognition result when generating the first voice endpoint.
[0175] The embodiment of the present invention further provides a device, which can implement the speech recognition processing method provided by the embodiment of the present invention by loading the above-mentioned speech recognition processing method in the form of a program. An optional hardware structure of the device provided by the embodiment of the present invention is as follows: Figure 6 As shown, it includes: at least one processor 01, at least one communication interface 02, at least one memory 03 and at least one communication bus 04.
[0176] In this embodiment, the processor 01 , the communication interface 02 , and the memory 03 communicate with each other via the communication bus 04 .
[0177] Optionally, the communication interface 02 may be an interface of a communication module for network communication, such as an interface of a GSM module.
[0178] Optionally, the processor 01 may be a central processing unit CPU, or an application specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention.
[0179] Optionally, the memory 03 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.
[0180] The memory 03 stores one or more computer instructions, and the one or more computer instructions are executed by the processor 01 to implement the speech recognition processing method provided in the embodiment of the present invention.
[0181] It should be noted that the above-mentioned terminal device may also include other devices (not shown) that may not be necessary for understanding the contents disclosed in the embodiments of the present invention; given that these other devices may not be necessary for understanding the contents disclosed in the embodiments of the present invention, the embodiments of the present invention will not introduce them one by one.
[0182] Correspondingly, an embodiment of the present invention further provides a storage medium, wherein the storage medium stores one or more computer instructions, and the one or more computer instructions are used to implement the speech recognition processing method described in the embodiment of the present invention.
[0183] The storage medium is a computer-readable storage medium, and the storage medium can be a read-only memory (ROM), a random access memory (RAM), a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk, or other medium that can store program codes.
[0184] The above-mentioned embodiments of the present invention are combinations of elements and features of the present invention. Unless otherwise mentioned, the elements or features may be considered as optional. Each element or feature may be put into practice without being combined with other elements or features. In addition, the embodiments of the present invention may be constructed by combining some elements and / or features. The order of operations described in the embodiments of the present invention may be rearranged. Some constructions of any one embodiment may be included in another embodiment, and may be replaced by the corresponding construction of another embodiment. It is obvious to those skilled in the art that claims that do not have a clear reference relationship to each other in the attached claims may be combined into embodiments of the present invention, or may be included as new claims in the amendment after submitting this application.
[0185] The embodiments of the present invention can be implemented by various means such as hardware, firmware, software or a combination thereof. In a hardware configuration, the method according to the exemplary embodiment of the present invention can be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, etc.
[0186] In a firmware or software configuration, the embodiments of the present invention may be implemented in the form of modules, procedures, functions, etc. The software code may be stored in a memory unit and executed by a processor. The memory unit is located inside or outside the processor and may send data to and receive data from the processor via various known means.
[0187] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the scope defined by the claims.
Claims
1. A speech recognition processing method, It is characterized in that include: Get voice data; Decoding the speech data to obtain an intermediate recognition result corresponding to the current word; When the pause time from when the intermediate recognition result is obtained is a first preset time, determining that a first voice endpoint needs to be generated; When the pause time from the time when the intermediate recognition result is obtained is a second preset time, re-scoring is performed to obtain a re-scoring result; the second preset time is less than or equal to the first preset time; Taking the end time of the current word as the starting point, when it is determined that the silence time of the voice data reaches the second preset time length, a second voice endpoint is generated; There is an endpoint interval between the time when the second speech endpoint is generated and the time when the current word ends; Determine whether the endpoint interval is greater than the first preset time length; When the endpoint interval is greater than the first preset time length, based on the first voice endpoint, outputting the re-scoring result as a final recognition result; When the endpoint interval is less than or equal to the first preset duration, a re-scoring process is performed based on the second voice endpoint, and the re-scoring result is output as the final recognition result.
2. The speech recognition processing method according to claim 1, It is characterized in that The second preset duration is equal to the first preset duration; after the first voice endpoint is generated, a re-scoring process is performed to obtain a re-scoring result; Based on the first voice endpoint, outputting the re-scoring result as a final recognition result includes: when the re-scoring result is obtained, outputting the re-scoring result as a final recognition result.
3. The speech recognition processing method according to claim 1, It is characterized in that The second preset time length is shorter than the first preset time length.
4. The speech recognition processing method according to claim 3, It is characterized in that The second preset time duration is 40% to 90% of the first preset time duration.
5. The speech recognition processing method according to claim 3, It is characterized in that The time required for the re-scoring process is less than or equal to the difference between the second preset time length and the first preset time length.
6. The speech recognition processing method according to claim 3, It is characterized in that Set the target endpoint delay time between the time point of outputting the final recognition result and the time point of the end of the current word; The time interval between the time point of obtaining the re-scoring result and the end time point of the current word is the re-scoring time interval; The speech recognition processing method further includes: determining whether the re-scoring time interval is less than the target endpoint delay time length; If yes, then when the re-scoring result is obtained, the first voice endpoint is generated at the interval of the dynamic window length; wherein the dynamic window length is: the difference between the delay time length of the target endpoint and the re-scoring time interval; Based on the first voice endpoint, outputting the re-scoring result as a final recognition result includes: when the first voice endpoint is generated, outputting the re-scoring result as the final recognition result.
7. The speech recognition processing method according to claim 6, It is characterized in that The decoding process of the speech data includes: decoding the speech data with a fixed decoding window length as a basic unit; The second preset duration is the fixed decoding window length.
8. The speech recognition processing method according to claim 7, It is characterized in that The fixed decoding window length is 30% to 50% of the target endpoint delay duration.
9. A speech recognition processing system, It is characterized in that include: A voice acquisition module, used to acquire voice data; A decoding module, used to decode the speech data to obtain an intermediate recognition result corresponding to the current word; An endpoint judgment module, used to judge that a first voice endpoint needs to be generated when the pause time from the time when the intermediate recognition result is obtained is a first preset time; and used to judge that a second voice endpoint needs to be generated when the silence time of the voice data reaches a second preset time with the time when the current word ends as the starting point; There is an endpoint interval between the time when the second speech endpoint is generated and the time when the current word ends; A re-scoring module, configured to perform a re-scoring process to obtain a re-scoring result when a pause time from when the intermediate recognition result is obtained is a second preset time, wherein the second preset time is less than or equal to the first preset time; An identification result output module, used to determine whether the endpoint interval is greater than the first preset time length; When the endpoint interval is greater than the first preset time length, based on the first voice endpoint, outputting the re-scoring result as a final recognition result; When the endpoint interval is less than or equal to the first preset duration, a re-scoring process is performed based on the second voice endpoint, and the re-scoring result is output as the final recognition result.
10. A device comprising at least one memory and at least one processor, wherein the memory stores one or more computer instructions, in, The one or more computer instructions are executed by the processor to implement the speech recognition processing method as described in any one of claims 1-8.
11. A storage medium, It is characterized in that The storage medium stores one or more computer instructions, and the one or more computer instructions are used to implement the speech recognition processing method as described in any one of claims 1-8.
