Speech recognition method and device, speech recognition model training method and device

Abstract

The application provides a speech recognition method and device, a training method and device of a speech recognition model. The method is applied to a speech recognition model and includes: obtaining speech data, the speech data including information of a to-be-recognized word; sequentially extracting speech features corresponding to each audio frame from the speech data; in response to the speech features accumulating to a preset time length, determining a speech state corresponding to each audio frame within the preset time length, wherein the speech state is used to represent a position of the audio frame in a text; and determining a recognition result according to the speech state, the recognition result including a single character in the to-be-recognized word. The technical solution of the application can recognize a single character with extremely low delay.

Description

Technical Field

[0001] The present application relates to the field of speech recognition technology, and in particular to a speech recognition method and device, and a speech recognition model training method and device. Background Art

[0002] Streaming speech recognition involves streaming audio recognition of spoken words, returning the corresponding text upon completion. This means that text recognition results are output during the user's voice input. However, when using neural networks for streaming speech recognition, when the speech involves long words (e.g., words consisting of three or more characters), the entire word must be spoken before it can be recognized. This results in significant delays and lags during speech recognition, which reduces the user experience.

[0003] In view of this, how to improve the text response speed of streaming speech recognition has become a technical problem that needs to be solved urgently. Summary of the Invention

[0004] In view of this, the embodiments of the present application provide a speech recognition method and device, and a speech recognition model training method and device, which can improve the response speed of text recognition.

[0005] In the first aspect, an embodiment of the present application provides a speech recognition method, which is applied to a speech recognition model, including: obtaining speech data, the speech data including information of words to be recognized; extracting speech features corresponding to each audio frame from the speech data in sequence; in response to the speech features accumulating to a preset time length, determining the speech state corresponding to each audio frame within the preset time length, wherein the speech state is used to indicate the position of the audio frame in the text; determining the recognition result based on the speech state, the recognition result including a single text in the word to be recognized.

[0006] In some embodiments of the present application, determining a recognition result according to a voice state includes: determining a recognition result when the number of consecutive times the voice state is detected meets a preset condition.

[0007] In certain embodiments of the present application, the speech state includes a tail-ending state, wherein when the number of consecutive detections of the speech state meets a preset condition, the recognition result is determined, including: when the number of frames in which the tail-ending state is detected continuously exceeds a first preset threshold, obtaining multiple phonemes corresponding to the currently accumulated multiple audio frames; performing word segmentation processing on the speech data to obtain multiple single characters, and corresponding the multiple phonemes to one of the multiple single characters to obtain a recognition result, wherein the recognition result includes a single character.

[0008] In certain embodiments of the present application, the voice state also includes an inactive voice state, and the recognition result also includes end recognition, wherein when the consecutive number of detected voice states meets a preset condition, the recognition result is determined, including: when the number of frames in which the inactive voice state and / or the word end state is continuously detected exceeds a second preset threshold, the recognition result is obtained, wherein the second preset threshold is greater than the first preset threshold.

[0009] In certain embodiments of the present application, the voice status includes: a tail-end status, which is used to indicate that the audio frame is within a first preset frame range corresponding to the tail of the word in the text; or a word-active voice status, which is used to indicate that the audio frame is within a second preset frame range in the text that is not the first preset frame range, wherein the first preset frame number and the second preset frame number are the total frame number of the text; or an inactive voice status, which is used to indicate silence or noise.

[0010] On the second aspect, an embodiment of the present application provides a training method for a speech recognition model, including: obtaining speech sample data with text annotation information, the speech sample data including sample words to be recognized; extracting speech sample features corresponding to each audio frame from the speech sample data in sequence; in response to the speech sample features accumulating to a preset time length, determining the speech state corresponding to each audio frame within the preset time length, wherein the speech state is used to indicate the position of the audio frame in the text; determining a prediction result based on the speech state, the prediction result including a single text in the sample words to be recognized; based on the prediction result, using a loss function to train the speech recognition model.

[0011] In some embodiments of the present application, determining a prediction result according to a voice state includes: determining a prediction result when the number of consecutive detections of a voice state meets a preset condition.

[0012] In certain embodiments of the present application, the speech state includes a tail-end state, and when the number of consecutive detections of the speech state meets a preset condition, a prediction result is determined, including: when the number of frames in which the tail-end state is detected continuously exceeds a first preset threshold, obtaining multiple phonemes corresponding to the currently accumulated multiple audio frames; performing word segmentation processing on the speech sample data to obtain multiple single characters, and corresponding the multiple phonemes to one of the multiple single characters to obtain a prediction result, wherein the prediction result includes a single character.

[0013] In certain embodiments of the present application, speech sample data is segmented to obtain multiple single characters, including: splitting text annotation information in the speech sample data into speech blocks of preset length, the speech blocks including multiple characters; and splitting the multiple characters to obtain multiple single characters.

[0014] In certain embodiments of the present application, the voice state also includes an inactive voice state, and the prediction result also includes end recognition, wherein when the consecutive number of detected voice states meets a preset condition, the prediction result is determined, including: when the number of frames in which the inactive voice state and / or the word tail end state is continuously detected exceeds a second preset threshold, the prediction result is obtained, wherein the second preset threshold is greater than the first preset threshold.

[0015] In certain embodiments of the present application, the method further includes: using the last preset number of frames of the final phoneme of each word included in the voice sample data as special audio frames; and training the voice recognition module based on the voice state corresponding to the special audio frame.

[0016] In a third aspect, an embodiment of the present application provides a speech recognition device, comprising: a first acquisition module for acquiring speech data, the speech data including information of words to be recognized; a first extraction module for extracting speech features corresponding to each audio frame from the speech data in sequence; a first determination module for determining the speech state corresponding to each audio frame within a preset time length in response to the accumulation of speech features to a preset time length, wherein the speech state is used to indicate the position of the audio frame in the text; a second determination module for determining the recognition result based on the speech state, the recognition result including a single text in the words to be recognized.

[0017] In a fourth aspect, an embodiment of the present application provides a training device for a speech recognition model, comprising: a second acquisition module for acquiring speech sample data with text annotation information, the speech sample data including sample words to be recognized; a second extraction module for extracting speech sample features corresponding to each audio frame from the speech sample data in sequence; a third determination module for determining the speech state corresponding to each audio frame within a preset time length in response to the accumulation of speech sample features to a preset time length, wherein the speech state is used to indicate the position of the audio frame in the text; a fourth determination module for determining a prediction result based on the speech state, the prediction result including a single text in the sample words to be recognized; a training module for training the speech recognition model using a loss function based on the prediction result.

[0018] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program is used to execute the speech recognition method described in the first aspect above, and / or to execute the training method of the speech recognition model described in the second aspect above.

[0019] In a sixth aspect, an embodiment of the present application provides an electronic device comprising: a processor; and a memory for storing processor-executable instructions, wherein the processor is used to execute the speech recognition method described in the first aspect above, and / or to execute the training method of the speech recognition model described in the second aspect above.

[0020] The embodiments of the present application provide a speech recognition method and device, and a speech recognition model training method and device. By determining the speech state corresponding to each audio frame and determining the recognition result of a single character in the speech data based on the speech state, the user does not need to wait for the recognition time of the entire word (such as a long word) in the speech data, and the speech recognition model can output single words with a lower delay, thereby improving the recognition response speed of the speech recognition model. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flowchart of a speech recognition method provided by an exemplary embodiment of the present application.

[0022] Figure 2 It is a flowchart of a speech recognition method provided by another exemplary embodiment of the present application.

[0023] Figure 3 It is a flowchart of a method for training a speech recognition model provided by an exemplary embodiment of the present application.

[0024] Figure 4 It is a flowchart of a method for training a speech recognition model provided by another exemplary embodiment of the present application.

[0025] Figure 5 It is a schematic diagram of splitting a plurality of single characters by a language module provided by an exemplary embodiment of the present application.

[0026] Figure 6 It is a flowchart of a method for training a speech recognition model provided by another exemplary embodiment of the present application.

[0027] Figure 7 It is a structural diagram of a speech recognition device provided by an exemplary embodiment of the present application.

[0028] Figure 8 It is a structural diagram of a training device for a speech recognition model provided by an exemplary embodiment of the present application.

[0029] Figure 9 It is a block diagram of an electronic device for training a speech recognition model or speech recognition provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0031] As one of the most common modes of communication, voice has been the subject of focused research as a universal method of human-computer interaction since the 1960s, with significant progress over the past decade. Simultaneously, with the rapid advancement of hardware performance and the increasing maturity of artificial intelligence technologies, a wide variety of smart devices have emerged. Voice interaction (such as voice wake-up and voice recognition) has become a highly popular and frequently used method, offering hands-free interaction.

[0032] While voice interaction demonstrates enormous demand and market potential, it also places higher demands on various scenarios. This has led to the emergence of streaming speech recognition, which has been highly praised in various voice interaction scenarios for its ability to recognize user input in real time, enhancing the user experience.

[0033] However, in the process of applying neural networks to streaming speech recognition, in order to ensure the recognition accuracy of speech recognition, the neural network is usually required to have some future vision, that is, the neural network needs to combine the content of the previous and next sentences to comprehensively determine the text result corresponding to the speech, which makes streaming speech recognition have a certain delay.

[0034] To solve the above technical problems, the embodiments of the present application provide the following speech recognition method.

[0035] Figure 1 It is a flowchart of a speech recognition method provided by an exemplary embodiment of the present application. Figure 1 The method is performed by a computing device, such as a server. Figure 1 As shown, the speech recognition method includes the following contents.

[0036] In one embodiment, the speech recognition method may be applied to a speech recognition model, that is, the execution subject of the following method may be a speech recognition model on a server.

[0037] The speech recognition model may include an acoustic module (also called an acoustic model) and a language module (also called a language model). The acoustic module may include a delayed streaming speech recognition branch and a non-delayed current frame speech state detection branch. The non-delayed current frame speech state detection branch is used to determine the speech state and phoneme corresponding to each audio frame; the delayed streaming speech recognition branch is used to combine with the language module to determine the recognition result of the current single text (also called a single word or a character, etc.).

[0038] It should be noted that since the delay-free current frame speech state detection branch included in the acoustic module does not need to be combined with the previous and next fields of view when performing speech state detection, that is, it does not need to be combined with the previous and next context for detection, the current frame speech state detection branch is a delay-free branch.

[0039] 110: Acquire voice data, where the voice data includes information about words to be recognized.

[0040] Specifically, the voice data may be collected by a terminal device, where the terminal device may be a terminal device having a sound pickup device such as a microphone, such as a mobile phone, a computer, a remote control, or a speaker.

[0041] In one embodiment, voice data may be obtained through direct user input. For example, when a user interacts with a terminal device via voice, the user speaks directly into a sound pickup device on the terminal device, and the terminal device captures the user's spoken words through the sound pickup device. The user's spoken words constitute the voice data.

[0042] The speech data may include information about words to be recognized. The words to be recognized may be proper nouns, such as "Great Wall." Furthermore, the speech data may include one or more words to be recognized, and a word to be recognized may be two or more characters. This embodiment of the present application does not specifically limit the words to be recognized and the speech data.

[0043] 120: Extract the speech features corresponding to each audio frame from the speech data in sequence.

[0044] Specifically, the speech data input to the speech recognition model is first pre-emphasized to highlight the high-frequency information, and then the spectral features corresponding to each audio frame are extracted through framing, windowing, Fourier transform, etc., and the spectral features are used as the speech features of each audio frame.

[0045] The speech feature may be a spectral feature, such as a Mel Frequency Cepstrum Coefficient (MFCC) feature or a Perceptual Linear Predictive (PLP) feature, etc. The embodiment of the present application does not specifically limit the speech feature.

[0046] The order of extracting the speech features of each audio frame may be in accordance with the order in which the speech data is input, that is, the first words spoken by the user are placed in front, and the later words spoken are placed in the back.

[0047] 130: In response to the voice features being accumulated to a preset time length, determining a voice state corresponding to each audio frame within the preset time length.

[0048] In one embodiment, the speech state is used to indicate the position of the audio frame in the text.

[0049] Specifically, the voice data may include multiple characters, each character may include one or more phonemes, each phoneme may correspond to multiple audio frames, and each audio frame may correspond to a certain voice feature.

[0050] The buffer is established in the speech recognition model, and the input audio frame corresponding speech features are accumulated in the buffer, and when the speech features accumulate to a preset time length, the current frame speech state detection branch in the acoustic module can determine the phoneme and speech state judgment value (such as posterior probability) of each audio frame corresponding to the speech features in the preset time length; and then the speech state judgment value is compared with the preset speech state threshold to determine the speech state corresponding to the audio frame.

[0051] It should be noted that the execution mode of step 130 can be loop execution, that is, the speech recognition model responds to the time length accumulation of the speech features, that is, when the speech recognition model detects that the speech features in the buffer accumulate to a preset time length, the subsequent steps will be executed.

[0052] The preset time length can be in milliseconds, such as 240ms or 500ms, etc. It should be noted that the preset time length can be greater than or equal to the time length of a character to ensure that each refresh obtains a single character recognition result, and the value of the preset time length is not limited in the embodiment of the application.

[0053] It should be noted that the speech state threshold corresponds to the speech state one by one, and when the speech state is multiple, the speech state threshold is also multiple, and each speech state threshold is different from each other. When the speech state judgment value is close to the first speech state threshold (that is, the threshold corresponding to the first speech state), the speech state corresponding to the current audio frame can be determined as the first speech state, wherein the first speech state is any one of the multiple speech states.

[0054] The speech state can be determined according to the position of the audio frame in the character, and the speech state can include one or more, and the type of speech state is not limited in the embodiment of the application.

[0055] In an embodiment, the number of speech states can be three, which are word active speech state, word tail end state and non-active speech state. That is, the current frame speech state detection branch without delay is used to determine the speech state of each audio frame as one of the above three speech states.

[0056] Among them, the word tail end state is used to indicate that the audio frame is located in the first preset frame number range corresponding to the word tail in the character; the word active speech state is used to indicate that the audio frame is located in the second preset frame number range which is not in the first preset frame number range in the character, wherein the first preset frame number and the second preset frame number are the total frame number of the character; the non-active speech state is used to indicate that it is a mute or noise.

[0057] It should be noted that if the voice state threshold is not reached for a long time, the subsequent refresh and end methods will still output in the form of whole words, which can effectively prevent the abnormal phenomenon of no refresh and no end for a long time in some complex, harsh, and high-noise environments.

[0058] 140: Determine a recognition result according to the speech state, where the recognition result includes a single character in the word to be recognized.

[0059] Specifically, the current frame voice state detection branch without delay can determine the recognition result of a single character when the number of consecutive detections of the voice state meets a preset condition, wherein the single character can be a single character in the word to be recognized.

[0060] In one embodiment, when the number of frames in which the voice state detection branch of the current frame without delay continuously detects the word ending state exceeds a first preset threshold, the recognition result of the current word is refreshed.

[0061] It should be noted that for the detailed description of this step, please refer to the description of the following embodiment.

[0062] It can be seen from this that the embodiment of the present application determines the voice state corresponding to each audio frame and determines the recognition result of a single character in the voice data based on the voice state, so that the user does not need to wait for the recognition time of the entire word (such as a long word) in the voice data, and thus enables the voice recognition model to output single words with a lower delay, thereby improving the recognition response speed of the voice recognition model.

[0063] In one embodiment of the present application, determining the recognition result according to the voice state includes: determining the recognition result when the number of consecutive times the voice state is detected meets a preset condition.

[0064] Specifically, the preset condition may be that the end of the word state is detected for a consecutive number of frames of a first preset threshold; the preset condition may also be that the end of the word state is detected for a consecutive number of frames of a second preset threshold; the preset condition may also be that the inactive voice state is detected for a consecutive number of frames of a second preset threshold. The embodiment of the present application does not specifically limit the preset conditions.

[0065] In one embodiment, when the number of frames in the word ending state continuously detected by the current frame voice state detection branch without delay exceeds a first preset threshold, the recognition result of the current word is refreshed. At this time, the delayed streaming speech recognition branch is combined with the language module to match the phonemes corresponding to each audio frame accumulated in the buffer with the text separated by the language module, thereby determining the current single text and outputting the single text as the recognition result. The first preset threshold can be 3, 5, or 7, and the value of the first preset threshold in the embodiment of the present application is not specifically limited.

[0066] In another embodiment, when the number of frames in which the voice state detection branch of the current frame continuously detects an inactive voice state and / or a word end state exceeds a second preset threshold, the current voice recognition is terminated. The second preset threshold is greater than the first preset threshold, and the value of the second preset threshold can be 7, 8, or 9. The embodiment of the present application does not specifically limit the value of the second preset threshold.

[0067] It can be seen from this that the embodiment of the present application determines whether to refresh the current result or whether to end the recognition by judging the current voice state, thereby providing a guarantee for reducing the delay in the recognition result of the single word in the future.

[0068] Figure 2 It is a flowchart of a speech recognition method provided by another exemplary embodiment of the present application. Figure 2 The embodiment is Figure 1 The examples of the embodiments will not be repeated here, and the differences will be described in detail. Figure 2 As shown, the speech recognition method includes the following contents.

[0069] In one embodiment, the speech state may include a word ending state, where the word ending state is used to indicate that the audio frame is within a first preset frame number range corresponding to the word ending in the text.

[0070] 210: When the number of frames in which the word tail end state is continuously detected exceeds a first preset threshold, obtain multiple phonemes corresponding to the multiple currently accumulated audio frames.

[0071] Specifically, when the number of frames in which the voice state detection branch of the current frame without delay continuously detects the end-of-word state exceeds a first preset threshold, that is, when the end-of-word state is detected for a number of consecutive frames exceeding the first preset threshold, multiple phonemes corresponding to multiple audio frames stored in the buffer are obtained. It should be noted that since the embodiment of the present application recognizes a single character, the multiple phonemes can be understood as the phonemes of a single character.

[0072] For example, taking the recognition of the word "中" as an example, when the end state of the word tail is detected for three consecutive frames, that is, when the audio frame corresponding to the phoneme "g" is detected for three consecutive frames, the "zh", "o" and "n" phonemes corresponding to the current accumulated multiple audio frames are obtained to proceed to the subsequent step 220.

[0073] 220: Perform word segmentation processing on the voice data to obtain multiple single characters, and correspond the multiple phonemes to one of the multiple single characters to obtain a recognition result.

[0074] In one embodiment, the recognition result includes a single character.

[0075] Specifically, multiple phonemes corresponding to multiple audio frames accumulated by the delayed streaming speech recognition branch in the acoustic module are mapped to one of the multiple single characters in the speech data obtained by the language module, that is, a mapping of one or more phonemes to a single character (that is, any character in the speech data) is obtained, thereby determining the single character corresponding to the phoneme.

[0076] The recognition result can be understood as a single character corresponding to the current audio frame, or can be understood as a word with the phoneme corresponding to the audio frame as the final phoneme.

[0077] For example, when the current frame speech state detection branch without delay continuously detects that the number of frames in the word ending state exceeds the first preset threshold, multiple phonemes corresponding to the currently accumulated multiple audio frames are obtained; then the streaming speech recognition branch with delay is combined with the language module to match the one or more phonemes corresponding to the multiple audio frames with the multiple single characters of the speech data split by the language module, thereby taking the determined single character as the recognition result.

[0078] It should be noted that the process of matching the multiple phonemes corresponding to the multiple currently accumulated audio frames with the single text determined by the language module to obtain the recognition result can be understood as refreshing the recognition result of the current text.

[0079] In one embodiment, the speech data is segmented to obtain multiple single characters based on a language module. The language module can first use an N-gram model to split the text information included in the speech data into units of one character and one word according to the input order of the speech data to obtain multiple speech blocks of preset lengths.

[0080] Furthermore, the words included in the speech blocks of preset length are split into multiple single characters, and in the process of splitting into multiple single characters, in order to avoid the loss of effect caused by cropping during the decoding process, the split multiple single characters can be connected by independent arcs.

[0081] Furthermore, the delayed streaming speech recognition branch is combined with the language module to match one or more phonemes corresponding to the accumulated multiple audio frames with any single word of the speech data split by the language module, thereby determining the single word corresponding to the one or more phonemes.

[0082] As can be seen, the embodiment of the present application refreshes the recognition result of the current word when the end of the word is detected for a preset number of frames in a row, eliminating the need for users to wait for the full utterance of a long word to obtain the corresponding text recognition result, thereby improving the response speed of the speech recognition model for single-word recognition. At the same time, the embodiment of the present application uses a language module to ensure that the recognition effect of the split single word is intact, thus solving the problem of achieving both high recognition response speed and excellent recognition effect in the speech recognition model.

[0083] In one embodiment of the present application, the voice state also includes an inactive voice state, and the recognition result also includes end recognition, wherein when the consecutive number of detected voice states meets a preset condition, the recognition result is determined, including: when the tail end state and / or the number of frames of the tail end state are continuously detected exceeds a second preset threshold, the recognition result is obtained, wherein the second preset threshold is greater than the first preset threshold.

[0084] In one embodiment, the inactive speech state is used to indicate a state where the audio frame is silent or noisy, such as a pause between words in speech data, or the end of speech.

[0085] Specifically, when the number of frames in which the current frame voice state detection branch detects an inactive voice state continuously exceeds a second preset threshold value without delay, or when the number of frames in which the tail end state is detected continuously exceeds a second preset threshold value, that is, when the tail end state or the inactive voice state is detected for a continuous number of frames of the second preset threshold value, the recognition result is obtained, and the recognition result may include ending this voice recognition.

[0086] For example, when the end of a word state or an inactive voice state is detected in five consecutive frames, the current voice recognition is terminated.

[0087] The second preset threshold value can be 7, 8, or 9. The numerical value of the second preset threshold value in the embodiment of the present application is not specifically limited. The second preset threshold value should be greater than the first preset threshold value. For example, the first preset threshold value is 3 and the second preset threshold value is 5. The embodiment of the present application does not specifically limit the values ​​of the first preset threshold value and the second preset threshold value.

[0088] It should be noted that, since during the detection process, the speech recognition model will first detect the end-of-word state and then detect the inactive voice state, the second preset threshold should be greater than the first preset threshold. If the first preset threshold is equal to the second preset threshold, the speech recognition model will refresh the recognition of the text corresponding to the current audio frame and will not stop recognizing the voice data. In other words, the second preset threshold should be the first preset threshold plus the number of frames of the end-of-word state or the inactive voice state.

[0089] It can be seen from this that the embodiment of the present application ends the speech recognition when an inactive speech state is detected for a number of consecutive preset threshold frames, so that the speech recognition model can be stopped in time during the detection process to avoid generating additional calculations.

[0090] In one embodiment of the present application, the voice status includes: a tail-end status, which is used to indicate that the audio frame is within a first preset frame range corresponding to the tail of the word in the text; or a word-active voice status, which is used to indicate that the audio frame is within a second preset frame range in the text that is not the first preset frame range, wherein the first preset frame number and the second preset frame number are the total frame number of the text; or an inactive voice status, which is used to indicate silence or noise.

[0091] Specifically, the voice state can be determined based on the position of the audio frame in the text, and the voice state can include one or more types. The embodiment of the present application does not specifically limit the type of voice state.

[0092] The following describes the three speech states using the character “中” as an example.

[0093] For example, the total number of frames of the character "中" is 50 frames, of which the phoneme "zh" corresponds to 20 frames; the phoneme "o" corresponds to 10 frames; the phoneme "n" corresponds to 10 frames; and the phoneme "g" corresponds to 10 frames.

[0094] At this time, the first preset frame number range in the word ending state can be the number of frames corresponding to the phoneme near the end of the character (i.e., the character "中"). For example, the first preset frame number range is 10 frames corresponding to "g", or the first preset frame number range is 10 frames corresponding to "g" and 5 frames corresponding to "n", for a total of 15 frames. The first preset frame number range is not specifically limited in the embodiment of the present application.

[0095] When the current frame voice state detection branch without delay detects that the current audio frame falls within the first preset frame number range, it is determined that the voice state corresponding to the current audio frame is the end-of-word state.

[0096] The second preset frame range in the active voice state of a character may be a range of audio frames within the total number of frames of the character (i.e., the character "中"), excluding the first preset frame range. For example, if the first preset frame range is 10 frames corresponding to "g", the second preset frame range is 40 frames corresponding to the remaining phonemes.

[0097] When the current frame voice state detection branch without delay detects that the current audio frame falls within the second preset frame number range, it is determined that the voice state corresponding to the current audio frame is the word-active voice state.

[0098] It should be noted that the total number of text frames can be understood as the sum of the first preset frame number range and the second preset frame number range.

[0099] The inactive speech state can be a state when the audio frame is silent or noisy, for example, the inactive speech state can be a pause between speeches, or the end of speech, etc.

[0100] It should be noted that the first preset frame number range and the second preset frame number range can be flexibly set according to actual conditions, and the embodiments of the present application do not make specific limitations. In special cases, the word active speech state can be used to represent that the phoneme corresponding to the audio frame is any phoneme except the last phoneme in a word; the word tail end state can be used to represent that the phoneme corresponding to the audio frame is the last phoneme in a word; and the inactive speech state can be used to represent silence or noise.

[0101] Therefore, the embodiments of the present application divide the speech state into the word active speech state, the word tail end state and the inactive speech state, which provides protection for refreshing the current word or ending the speech recognition according to the detected specified speech state in the process of speech recognition.

[0102] The following takes the text corresponding to the speech data as "People's Republic of China" as an example to describe the speech recognition method of the present application.

[0103] When the speech state of the "China" word audio frame is detected as the word tail end state in the non-delayed current frame speech state detection branch in the acoustic module, and the frame number of the audio frame corresponding to the word tail end state satisfies the preset condition, the recognition result of the current word is refreshed. At this time, the streaming speech recognition branch with delay combines the language module to map the accumulated "zh", "o" and "n" to a single word recognized by the language module, thereby determining the "China" word.

[0104] Similarly, when the speech state of the "China" word audio frame is detected as the word tail end state in the non-delayed current frame speech state detection branch, and the frame number of the audio frame corresponding to the word tail end state satisfies the preset condition, the recognition result of the current word is refreshed. At this time, the streaming speech recognition branch with delay combines the language module to map the accumulated "zh", "o" and "n" to a single word recognized by the language module, thereby determining the "China" word. By analogy, after the "country" word is refreshed, when the inactive speech state is detected in the non-delayed current frame speech state detection branch, the streaming speech recognition is ended.

[0105] Figure 3 is a flowchart of a training method of a speech recognition model provided by an exemplary embodiment of the present application. Figure 3 The method of is performed by a computing device, for example, a server. As shown in Figure 3 The training method of the speech recognition model includes the following contents.

[0106] 310: Acquire speech sample data with text annotation information, where the speech sample data includes sample words to be recognized.

[0107] Specifically, the speech sample data may be speech data with text annotation information, and the speech sample data may also be speech data in a speech sample training set, wherein the speech sample data may include sample words (ie, words) to be recognized.

[0108] The voice sample data can be collected by a terminal device, where the terminal device can be a terminal device with a microphone or other sound pickup device, such as a mobile phone, computer, or remote control. The voice sample data can also be collected by a microphone or other sound pickup device and obtained after noise reduction processing. The voice sample data can also include application scenarios, such as television home, car, and natural outdoor use. The embodiments of this application do not specifically limit the voice sample data.

[0109] The speech sample data may also be prepared by first preparing a large amount of speech data with text annotation information to construct a speech sample training set; and then selecting any speech data in the speech sample training set as the speech sample data.

[0110] It should be noted that the amount of speech sample data used to train the speech recognition model application can be determined according to actual needs, and the embodiments of the present application do not specifically limit the number of samples required for training.

[0111] 320: Extracting speech sample features corresponding to each audio frame from the speech sample data in sequence.

[0112] 330: In response to the voice sample features being accumulated to a preset time length, determining a voice state corresponding to each audio frame within the preset time length.

[0113] In one embodiment, the speech state is used to indicate the position of the audio frame in the text.

[0114] 340: Determine a prediction result according to the speech state, where the prediction result includes a single character in the sample word to be recognized.

[0115] It should be noted that steps 320 to 340 are substantially the same as those described in the above embodiment. For details, please refer to the description of the above embodiment.

[0116] 350:Based on the prediction results, the speech recognition model is trained using the loss function.

[0117] Specifically, the loss function is used to calculate the loss value of the prediction result of the speech recognition model and the text annotation information, and based on the loss value, the speech recognition model is repeatedly trained by backpropagation until the desired speech recognition model is obtained.

[0118] The loss function can be a cross-entropy function, a mean square error loss function, etc. The type of the loss function is not limited in the embodiments of the present application.

[0119] In an embodiment, the speech recognition model comprises an acoustic module, the current frame speech state detection branch without delay included in the acoustic module can be obtained by determining each audio frame based on the speech sample features of the speech sample data, and determining the speech state of each audio frame respectively, and the streaming speech recognition branch with delay included in the acoustic module can be obtained by training based on the speech sample data.

[0120] In an embodiment, the speech recognition model further comprises a language module, the language module is obtained by training based on mapping the phonemes corresponding to each audio frame by splitting the speech sample data into single characters.

[0121] The speech sample training set can be a training set collecting a large amount of recognition corpus text, and the speech sample data can or can not be included in the speech sample training set. The embodiments of the present application do not limit the speech sample training set.

[0122] It should be noted that the N-gram model can be understood as an unsupervised model obtained by statistical analysis of a large amount of corpus text.

[0123] Therefore, by training the acoustic module in the speech recognition model, the embodiments of the present application determine the speech state corresponding to each audio frame, which provides a guarantee for the speech recognition model to output single characters with low delay in the speech recognition process.

[0124] In an embodiment of the present application, determining the prediction result according to the speech state comprises: determining the prediction result when the number of consecutive detections of the speech state meets a preset condition.

[0125] In an embodiment of the present application, the speech state comprises an ending state of a character tail, and determining the prediction result when the number of consecutive detections of the speech state meets a preset condition comprises: when the number of frames of the ending state of the character tail detected consecutively exceeds a first preset threshold, obtaining a plurality of phonemes corresponding to a plurality of audio frames accumulated currently; performing word segmentation processing on the speech sample data to obtain a plurality of single characters, and corresponding the plurality of phonemes to one of the plurality of single characters to obtain the prediction result, wherein the prediction result comprises a single character.

[0126] It should be noted that the embodiments of the present application are basically the same as the above-mentioned process of processing speech data, and details can be referred to the description of the above-mentioned embodiments.

[0127] Figure 4It is a flowchart of a method for training a speech recognition model provided by another exemplary embodiment of the present application. Figure 4 The embodiment is Figure 1 The examples of the embodiments will not be repeated here, and the differences will be described in detail. Figure 4 As shown, the training method of the speech recognition model includes the following contents.

[0128] 410: Split the text annotation information in the voice sample data into voice blocks of a preset length, where the voice blocks include multiple characters.

[0129] Specifically, before splitting the text information included in the voice sample data into voice blocks of a preset length, it also includes: based on a large amount of collected corpus text, generating an N-gram model through preprocessing operations, where the preprocessing operations may include filtering, word segmentation, conversion and other operations; the N-gram model can be understood as an algorithm based on a statistical language module, that is, performing a sliding window operation of a preset length on the content in the text to form a sequence of a preset length, where the preset length can be 1, 2 or 3. The embodiment of the present application does not specifically limit the value of the preset length.

[0130] In one embodiment, the text information included in the voice sample data is split according to a specified path using an N-gram model to obtain multiple voice blocks of preset lengths, where the specified path can be the input order of the voice sample data, that is, the order of user voice input.

[0131] Among them, when the preset length is 1, the N-gram model will be split into one character and / or one word as a unit, for example, I love China, which is split into I, love, and China; when the preset length is 2, the N-gram model will be split into two characters, two words, or a combination of one character and one word as a unit, for example, I love China, which is split into I love and China.

[0132] In one embodiment, see Figure 5 When the preset length is 1, a plurality of speech blocks 510 of preset length are obtained, wherein the plurality of speech blocks 510 of preset length include "I", "love" and "People's Republic of China", wherein the plurality of characters included in the speech blocks may be words (i.e., sample words to be recognized), and the words may be long words greater than or equal to two characters.

[0133] It should be noted that when a speech block of a preset length includes multiple characters, the speech block of the preset length can be understood as a high-order gram.

[0134] 420: Split multiple characters into multiple single characters.

[0135] Specifically, multiple characters (such as sample words to be recognized) included in a speech block of a preset length are split into single words. In the process of splitting into single words, in order to ensure that the split single words are independent of each other, the split single words can be connected with independent arcs; at the same time, in order to avoid the effect loss caused by clipping during the decoding process, the score on the independent arc can be evenly distributed to each independent arc after the split, or the score on the independent arc (such as Figure 5 1.0 or 7.0 in the example), and concentrate on the first independent arc after the split. It should be noted that the score distribution method can be flexibly set according to actual needs, and the embodiment of the present application does not make specific limitations on this.

[0136] For example, see Figure 5 , splitting the multiple characters (eg, People's Republic of China) included in the speech block of a preset length into single characters, such as the split speech block 520.

[0137] It should be noted that if the speech block does not include multiple characters, that is, if the speech block contains only single words (or single characters), step 420 can be omitted.

[0138] It should be noted that the embodiment of the present application (ie, step 410 and step 420) may be training for a language module.

[0139] In one embodiment, after splitting multiple characters into multiple single characters, the method may further include mapping the phonemes corresponding to each audio frame with one of the multiple single characters to obtain the single character corresponding to the phoneme, thereby realizing training of the language module.

[0140] Specifically, the one or more phonemes corresponding to each audio frame accumulated by the delayed streaming speech recognition branch in the acoustic module are mapped to one of the multiple individual characters split by the language module. This mapping of one or more phonemes to individual characters (i.e., words) is obtained, thereby determining the individual characters corresponding to the one or more phonemes. Subsequently, weight shifting, determinization, minimization, and serialization are performed to package the resulting data into a language module for training.

[0141] As can be seen from this, the embodiment of the present application, by setting up a language module, splits the voice sample data and then maps the phonemes to single words. This allows the speech recognition model to recognize long words as single words during the recognition process, thereby improving the recognition response speed of the speech recognition model for single words. At the same time, the language module uses independent arcs to split multiple characters to ensure that the recognition effect of the split single words is intact, thus solving the problem of not being able to achieve both recognition response speed and excellent recognition effect in the speech recognition model.

[0142] In one embodiment of the present application, the voice state also includes an inactive voice state, and the prediction result also includes end recognition, wherein when the consecutive number of detected voice states meets the preset conditions, the prediction result is determined, including: when the number of frames in which the inactive voice state and / or the word tail end state is continuously detected exceeds a second preset threshold, the prediction result is obtained, wherein the second preset threshold is greater than the first preset threshold.

[0143] Figure 6 It is a flowchart of a method for training a speech recognition model provided by another exemplary embodiment of the present application. Figure 6 The embodiment is Figure 1 The examples of the embodiments will not be repeated here, and the differences will be described in detail. Figure 6 As shown, the training method of the speech recognition model includes the following contents.

[0144] 610: The last preset number of frames of the final phoneme of each word included in the voice sample data is used as a special audio frame.

[0145] Specifically, the speech sample data may include multiple characters, each of which may include one or more phonemes, and each phoneme may correspond to multiple audio frames. The last preset number of frames may be the last three or five frames corresponding to the last phoneme. This embodiment of the application does not specifically limit the value of the last preset number of frames.

[0146] In one embodiment, the last three frames of the final phoneme of each word are used as special audio frames.

[0147] It should be noted that in order to enable the undelayed current frame speech state detection branch in the acoustic module to better learn the speech state corresponding to the audio frame (such as the end state of the word tail), so that the acoustic module can refresh and recognize each text according to preset conditions, the embodiment of the present application uses the last three frames of each word tail phoneme as special audio frames when training the acoustic module.

[0148] 620: Training the speech recognition module based on the speech state corresponding to the special audio frame.

[0149] In one embodiment, the word-ending state is used to indicate that the phoneme corresponding to the audio frame is the last phoneme in a word.

[0150] Specifically, the special audio frame is the last preset number of audio frames corresponding to the final phoneme of each word, that is, the voice state corresponding to the special audio frame is the final end state.

[0151] It should be noted that, since the technical solution of the present application will refresh the recognition of the current single character in the process of speech recognition only when the number of frames in which the end state of the word tail is continuously detected exceeds the first preset threshold, during the training process of the acoustic module, it is necessary to pay attention to the acoustic module's learning of the end state of the word tail to ensure that the acoustic module can refresh the single character in a timely manner.

[0152] It can be seen from this that the embodiment of the present application sets special audio frames to strengthen the speech recognition model's learning of specified speech states, thereby providing a guarantee for the subsequent speech recognition model to output the recognition results of single words in a timely manner.

[0153] Figure 7 This is a schematic diagram of the structure of a speech recognition device provided by an exemplary embodiment of the present application. Figure 7 As shown, the speech recognition device 700 includes: a first acquisition module 710 , a first extraction module 710 , a first determination module 730 and a second determination module 740 .

[0154] The first acquisition module 710 is used to acquire voice data, which includes information about the words to be recognized; the first extraction module 710 is used to extract the voice features corresponding to each audio frame from the voice data in sequence; the first determination module 730 is used to determine the voice state corresponding to each audio frame within the preset time length in response to the accumulation of voice features to a preset time length, wherein the voice state is used to indicate the position of the audio frame in the text; the second determination module 740 is used to determine the recognition result based on the voice state, and the recognition result includes a single character in the words to be recognized.

[0155] An embodiment of the present application provides a speech recognition device that determines the speech state corresponding to each audio frame and determines the recognition result of a single character in the speech data based on the speech state, so that the user does not need to wait for the recognition time of the entire word (such as a long word) in the speech data, and thus enables the speech recognition model to output single words with a lower delay, thereby improving the recognition response speed of the speech recognition model.

[0156] According to an embodiment of the present application, the second determination module 740 is configured to determine a recognition result when the number of consecutive detections of the voice state satisfies a preset condition.

[0157] According to one embodiment of the present application, the speech state includes a tail end state, and the second determination module 740 is used to obtain multiple phonemes corresponding to the current accumulated multiple audio frames when the number of frames in which the tail end state is continuously detected exceeds a first preset threshold; perform word segmentation processing on the speech data to obtain multiple single characters, and correspond the multiple phonemes to one of the multiple single characters to obtain a recognition result, wherein the recognition result includes a single character.

[0158] According to one embodiment of the present application, the voice state also includes an inactive voice state, and the recognition result also includes end recognition. The second determination module 740 is used to obtain the recognition result when the number of frames in which the inactive voice state and / or the word end state is continuously detected exceeds a second preset threshold, wherein the second preset threshold is greater than the first preset threshold.

[0159] According to one embodiment of the present application, the voice status includes: a tail-end status, which is used to indicate that the audio frame is within a first preset frame number range corresponding to the tail of the word in the text; or a word-active voice status, which is used to indicate that the audio frame is within a second preset frame number range in the text that is not the first preset frame number range, wherein the first preset frame number and the second preset frame number are the total frame number of the text; or an inactive voice status, which is used to indicate silence or noise.

[0160] It should be understood that the specific working processes and functions of the first acquisition module 710, the first extraction module 710, the first determination module 730 and the second determination module 740 in the above embodiment can refer to the above Figure 1 or Figure 2 To avoid repetition, the description of the speech recognition method provided in the embodiment will not be repeated here.

[0161] Figure 8 Schematic diagram of the structure of a training device for a speech recognition model provided by an exemplary embodiment of the present application. Figure 8 As shown, the speech recognition model training device 800 includes: a second acquisition module 810, a second extraction module 820, a third determination module 830, a fourth determination module 840 and a training module 850.

[0162] The second acquisition module 810 is used to obtain speech sample data with text annotation information, and the speech sample data includes sample words to be recognized; the second extraction module 820 is used to extract the speech sample features corresponding to each audio frame from the speech sample data in sequence; the third determination module 830 is used to determine the speech state corresponding to each audio frame within the preset time length in response to the accumulation of speech sample features to a preset time length, wherein the speech state is used to indicate the position of the audio frame in the text; the fourth determination module 840 is used to determine the prediction result according to the speech state, and the prediction result includes a single text in the sample words to be recognized; the training module 850 is used to train the speech recognition model based on the prediction result using a loss function.

[0163] An embodiment of the present application provides a training device for a speech recognition model. By setting an acoustic module, the speech state corresponding to each audio frame is determined, thereby ensuring that the speech recognition model can output single words with low delay during the speech recognition process.

[0164] According to an embodiment of the present application, the fourth determination module 840 is configured to determine a prediction result when the number of consecutive detections of the voice state satisfies a preset condition.

[0165] According to one embodiment of the present application, the speech state includes a tail end state, and the fourth determination module 840 is used to obtain multiple phonemes corresponding to the current accumulated multiple audio frames when the number of frames in which the tail end state is continuously detected exceeds a first preset threshold; perform word segmentation processing on the speech sample data to obtain multiple single characters, and correspond the multiple phonemes to one of the multiple single characters to obtain a prediction result, wherein the prediction result includes a single character.

[0166] According to an embodiment of the present application, the fourth determination module 840 is used to split the text annotation information in the voice sample data into voice blocks of a preset length, where the voice blocks include multiple characters; and split the multiple characters to obtain multiple single characters.

[0167] According to one embodiment of the present application, the voice state also includes an inactive voice state, and the recognition result also includes end recognition. The fourth determination module 840 is used to obtain the prediction result when the number of frames in which the inactive voice state and / or the word end state is continuously detected exceeds a second preset threshold, wherein the second preset threshold is greater than the first preset threshold.

[0168] According to an embodiment of the present application, the third determination module 830 is configured to use the last preset number of frames of the final phoneme of each word included in the voice sample data as special audio frames; and train the voice recognition module based on the voice state corresponding to the special audio frames.

[0169] It should be understood that the specific working processes and functions of the second acquisition module 810, the second extraction module 820, the third determination module 830, the fourth determination module 840 and the training module 850 in the above embodiment can refer to the above Figures 3 to 6 To avoid repetition, the description of the training method of the speech recognition model provided in the embodiment will not be repeated here.

[0170] Figure 9 1 is a block diagram of an electronic device 900 for training a speech recognition model or speech recognition provided by an exemplary embodiment of the present application.

[0171] Reference Figure 9 The electronic device 900 includes a processing component 910, which further includes one or more processors, and a memory resource represented by a memory 920 for storing instructions executable by the processing component 910, such as an application. The application stored in the memory 920 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 910 is configured to execute the instructions to perform the above-mentioned speech recognition model training method or speech recognition method.

[0172] The electronic device 900 may further include a power supply component configured to perform power management of the electronic device 900, a wired or wireless network interface configured to connect the electronic device 900 to a network, and an input / output (I / O) interface. The electronic device 900 may be operated based on an operating system stored in the memory 920, such as Windows Server 2008. TM , Mac OSX TM , Unix TM , Linux TM , FreeBSD TM or similar.

[0173] A non-temporary computer-readable storage medium, when the instructions in the storage medium are executed by the processor of the above-mentioned electronic device 900, enables the above-mentioned electronic device 900 to execute a training method for a speech recognition model, including: obtaining speech sample data with text annotation information, the speech sample data including sample words to be recognized; extracting speech sample features corresponding to each audio frame from the speech sample data in sequence; in response to the speech sample features accumulating to a preset time length, determining the speech state corresponding to each audio frame within the preset time length, wherein the speech state is used to indicate the position of the audio frame in the text; determining a prediction result based on the speech state, the prediction result including a single word in the sample words to be recognized; based on the prediction result, training the speech recognition model using a loss function.

[0174] Alternatively, a speech recognition method includes: obtaining speech data, the speech data including information of a word to be recognized; extracting speech features corresponding to each audio frame from the speech data in sequence; in response to the speech features accumulating to a preset time length, determining the speech state corresponding to each audio frame within the preset time length, wherein the speech state is used to indicate the position of the audio frame in the text; and determining a recognition result based on the speech state, the recognition result including a single character in the word to be recognized.

[0175] All of the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, and will not be described in detail here.

[0176] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0177] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0178] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0179] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0180] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0181] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program check codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0182] It should be noted that, in the description of this application, the terms "first," "second," "third," etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, "plurality" means two or more.

[0183] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A speech recognition method, characterized in that: Applied to speech recognition models, including: Acquiring voice data, wherein the voice data includes information about words to be recognized; Extract the speech features corresponding to each audio frame from the speech data in sequence: In response to the voice features being accumulated for a preset time length, determining a voice state corresponding to each audio frame within the preset time length, wherein the voice state is used to represent the position of the audio frame in the text, and the voice state includes a word ending state; When the number of consecutive detections of the voice state satisfies a preset condition, a recognition result is determined, wherein the recognition result includes a single character in the word to be recognized. Wherein, when the number of consecutive detections of the voice state meets a preset condition, determining the recognition result includes: When the number of frames in which the word ending state is continuously detected exceeds a first preset threshold, obtaining a plurality of phonemes corresponding to the plurality of currently accumulated audio frames; The speech data is segmented to obtain a plurality of single characters, and the plurality of phonemes are matched with one of the plurality of single characters to obtain the recognition result, wherein the recognition result includes the single character.

2. The speech recognition method according to claim 1, wherein: The voice state also includes an inactive voice state, and the recognition result also includes end of recognition. The step of determining the recognition result when the number of consecutive detections of the voice state meets a preset condition includes: When the number of frames in which the inactive voice state and / or the word ending state is continuously detected exceeds a second preset threshold, the recognition result is obtained, wherein the second preset threshold is greater than the first preset threshold.

3. The speech recognition method according to claim 1 or 2, characterized in that: The voice status includes: The suffix end state is used to indicate that the audio frame is within a first preset frame number range corresponding to the suffix in the text; or The active voice state of the character is used to indicate that the audio frame is within a second preset frame number range of the character that is not within the first preset frame number range, wherein the first preset frame number and the second preset frame number are the total frame number of the character; or Inactive speech state, used to represent silence or noise.

4. A method for training a speech recognition model, characterized in that: include: Acquire speech sample data with text annotation information, wherein the speech sample data includes sample words to be recognized; Extracting speech sample features corresponding to each audio frame from the speech sample data in sequence; In response to the voice sample features being accumulated to a preset duration, determining a voice state corresponding to each audio frame within the preset duration, wherein the voice state is used to represent the position of the audio frame in the text, and the voice state includes a word ending state; When it is detected that the number of consecutive occurrences of the voice state satisfies a preset condition, determining a prediction result, wherein the prediction result includes a single character in the sample word to be recognized; Based on the prediction results, the speech recognition model is trained using a loss function. Wherein, when the number of consecutive detections of the voice state satisfies a preset condition, determining the prediction result includes: When the number of frames in which the word ending state is continuously detected exceeds a first preset threshold, obtaining a plurality of phonemes corresponding to the plurality of currently accumulated audio frames; The speech sample data is segmented to obtain a plurality of single characters, and the plurality of phonemes are matched with one of the plurality of single characters to obtain the prediction result, wherein the prediction result includes the single character.

5. The method for training a speech recognition model according to claim 4, wherein: The word segmentation processing of the voice sample data to obtain multiple single characters includes: Splitting the text annotation information in the voice sample data into voice blocks of a preset length, wherein the voice blocks include a plurality of characters; The multiple characters are split to obtain the multiple single characters.

6. The method for training a speech recognition model according to claim 4, wherein: The speech state also includes an inactive speech state, and the prediction result also includes end recognition. The step of determining a prediction result when the number of consecutive detections of the voice state satisfies a preset condition includes: When the number of frames in which the inactive voice state and / or the word ending state is continuously detected exceeds a second preset threshold, the prediction result is obtained, wherein the second preset threshold is greater than the first preset threshold.

7. The method for training a speech recognition model according to claim 4, wherein: The method further comprises: The last preset number of frames of the final phoneme of each word included in the voice sample data is used as a special audio frame; The speech recognition module is trained based on the speech state corresponding to the special audio frame.

8. A speech recognition device, characterized in that: include: A first acquisition module is used to acquire voice data, wherein the voice data includes information about words to be recognized; A first extraction module is used to extract the speech features corresponding to each audio frame from the speech data in sequence; A first determining module is configured to determine, in response to the voice features being accumulated for a preset time period, a voice state corresponding to each audio frame within the preset time period, wherein the voice state is used to represent the position of the audio frame in the text, and the voice state includes a word ending state; The second determining module is configured to determine a recognition result when the number of consecutive detections of the voice state satisfies a preset condition, wherein the recognition result includes a single character in the word to be recognized. The second determination module is also used to obtain multiple phonemes corresponding to the current accumulated multiple audio frames when the number of frames in which the word ending state is continuously detected exceeds a first preset threshold; perform word segmentation processing on the voice data to obtain multiple single characters, and correspond the multiple phonemes to one of the multiple single characters to obtain the recognition result, wherein the recognition result includes the single character.

9. A training device for a speech recognition model, characterized in that: include: A second acquisition module is used to acquire speech sample data with text annotation information, wherein the speech sample data includes sample words to be recognized; A second extraction module is used to extract the speech sample features corresponding to each audio frame from the speech sample data in sequence; a third determining module configured to determine, in response to the accumulation of the speech sample features to a preset duration, a speech state corresponding to each audio frame within the preset duration, wherein the speech state is used to represent the position of the audio frame in the text, and the speech state includes a word ending state; a fourth determining module, configured to determine a prediction result when the number of consecutive detections of the voice state satisfies a preset condition, wherein the prediction result includes a single character in the sample word to be recognized; A training module is used to train the speech recognition model using a loss function based on the prediction results. Among them, the fourth determination module is also used to obtain multiple phonemes corresponding to the current accumulated multiple audio frames when the number of frames in which the word ending state is continuously detected exceeds a first preset threshold; perform word segmentation processing on the voice sample data to obtain multiple single characters, and correspond the multiple phonemes to one of the multiple single characters to obtain the prediction result, wherein the prediction result includes the single character.

10. A computer-readable storage medium, characterized in that The storage medium stores a computer program, which is used to execute the speech recognition method described in any one of claims 1 to 3 and / or the training method of the speech recognition model described in any one of claims 4 to 7.

11. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor, The processor is used to execute the speech recognition method described in any one of claims 1 to 3, and / or to execute the training method of the speech recognition model described in any one of claims 4 to 7.

Images