A speech recognition method, apparatus, storage medium, and device

Abstract

The application discloses a speech recognition method and device, a storage medium and equipment, and the method comprises the following steps: firstly, obtaining a target speech to be recognized; then, detecting the starting frame and the ending frame of the target speech by means of real-time voice activity endpoint detection, and determining the effective audio segment in the target speech according to the starting frame and the ending frame; then, extracting the audio features of the effective audio segment; and decoding the target speech according to the audio features to obtain a preliminary recognition result; further, adaptively and dynamically adjusting a preset clipping threshold according to the preliminary recognition result; and clipping the target speech and the preliminary recognition result according to the adjusted clipping threshold to obtain a final recognition result corresponding to the target speech. It can be seen that the starting frame and the ending frame of the target speech are more accurately detected by means of real-time voice activity endpoint detection, and the preset clipping threshold is adaptively and dynamically adjusted, so that the false triggering of noise clipping is reduced, and the accuracy of the recognition result is improved.

Description

Technical Field

[0001] This application relates to the field of speech processing technology, and in particular to a speech recognition method, apparatus, storage medium and device. Background Technology

[0002] With the continuous breakthroughs in artificial intelligence technology and the increasing popularity of various smart terminal devices, human-computer interaction is occurring more and more frequently in people's daily work and life. Voice interaction, as the next generation of human-computer interaction, can bring great convenience to people's lives, and among its key aspects is speech recognition technology.

[0003] Traditional speech recognition methods typically use Voice Activity Detection (VAD) to identify the start and end frames of audio. After feature extraction via a neural network, the average number of valid states per frame (VAP) is calculated. During decoding, a pre-set threshold is applied; if the VAP exceeds this threshold, the audio is rejected, and the decoding path (MaxPath) is set to empty, indicating noisy audio and a blank recognition result. However, this method has two problems: first, the start and end frames detected by VaD include margins, failing to reflect the true start and end positions of the audio; second, the VAP threshold is fixed during decoding and cannot be dynamically adjusted, leading to some audio being mistakenly truncated as noise. This results in inaccurate final speech recognition results. Summary of the Invention

[0004] The main objective of this application is to provide a speech recognition method, apparatus, storage medium, and device that can effectively improve the accuracy of recognition results during speech recognition.

[0005] This application provides a speech recognition method, including:

[0006] Obtain the target speech to be recognized;

[0007] The start and end frames of the target speech are detected by real-time speech activity endpoint detection, and the effective audio segments in the target speech are determined based on the start and end frames.

[0008] Extract the audio features of the effective audio segments in the target speech; and decode the target speech based on the audio features to obtain preliminary recognition results;

[0009] Based on the preliminary recognition results, the preset cropping threshold is adaptively and dynamically adjusted; and based on the adjusted cropping threshold, the target speech and the preliminary recognition results are cropped to obtain the final recognition result corresponding to the target speech.

[0010] In one possible implementation, extracting audio features from effective audio segments of the target speech includes:

[0011] The effective audio segments in the target speech are input into a preset neural network to extract the audio features of the effective audio segments in the target speech.

[0012] In one possible implementation, the preset cropping thresholds include a first preset cropping threshold, a second preset cropping threshold, and a third preset cropping threshold; the first preset cropping threshold is greater than the second preset cropping threshold; and the second preset cropping threshold is greater than the third preset cropping threshold.

[0013] In one possible implementation, the step of adaptively and dynamically adjusting a preset pruning threshold based on the preliminary recognition result, and pruning the target speech and the preliminary decoding result according to the adjusted pruning threshold to obtain the final recognition result corresponding to the target speech, includes:

[0014] Determine whether the preliminary identification results contain words from a preset whitelist;

[0015] If so, the preset cropping threshold is adjusted to the first preset cropping threshold;

[0016] Calculate the frame average number of valid states for valid audio segments in the target speech, and then trim the valid audio segments and their preliminary recognition results that have a frame average number of valid states that is not less than the first preset trimming threshold to obtain the final recognition result corresponding to the target speech.

[0017] In one possible implementation, the step of adaptively and dynamically adjusting a preset pruning threshold based on the preliminary recognition result, and pruning the target speech and the preliminary decoding result according to the adjusted pruning threshold to obtain the final recognition result corresponding to the target speech, includes:

[0018] Determine whether the preliminary identification results contain words from a preset blacklist;

[0019] If so, the preset cropping threshold is adjusted to the third preset cropping threshold;

[0020] Calculate the frame average number of valid states for the valid audio segments in the target speech, and then trim the valid audio segments and their preliminary recognition results that have a frame average number of valid states that is not less than the third preset trimming threshold to obtain the final recognition result corresponding to the target speech.

[0021] In one possible implementation, the step of adaptively and dynamically adjusting a preset pruning threshold based on the preliminary recognition result, and pruning the target speech and the preliminary decoding result according to the adjusted pruning threshold to obtain the final recognition result corresponding to the target speech, includes:

[0022] When it is determined that the preliminary identification result does not contain words from the preset whitelist or the preset blacklist, the preset pruning threshold is adjusted to the second preset pruning threshold.

[0023] Calculate the frame average number of valid states for the valid audio segments in the target speech, and then trim the valid audio segments and their preliminary recognition results corresponding to the frame average number of valid states that is not less than the second preset trimming threshold to obtain the final recognition result corresponding to the target speech.

[0024] In one possible implementation, the first preset cropping threshold is 2000; the second preset cropping threshold is 1600; and the third preset cropping threshold is 1130.

[0025] This application also provides a voice recognition device, including:

[0026] The acquisition unit is used to acquire the target speech to be recognized;

[0027] The detection unit is used to detect the start frame and end frame of the target speech by means of real-time speech activity endpoint detection, and to determine the effective audio segment in the target speech based on the start frame and end frame.

[0028] An extraction unit is used to extract audio features of effective audio segments from the target speech; and to decode the target speech based on the audio features to obtain preliminary recognition results;

[0029] The cropping unit is used to adaptively and dynamically adjust a preset cropping threshold based on the preliminary recognition result; and to crop the target speech and the preliminary recognition result based on the adjusted cropping threshold to obtain the final recognition result corresponding to the target speech.

[0030] In one possible implementation, the extraction unit is specifically used for:

[0031] The effective audio segments in the target speech are input into a preset neural network to extract the audio features of the effective audio segments in the target speech.

[0032] In one possible implementation, the preset cropping thresholds include a first preset cropping threshold, a second preset cropping threshold, and a third preset cropping threshold; the first preset cropping threshold is greater than the second preset cropping threshold; and the second preset cropping threshold is greater than the third preset cropping threshold.

[0033] In one possible implementation, the trimming unit includes:

[0034] The first judgment subunit is used to determine whether the preliminary identification result contains words from a preset whitelist;

[0035] The first adjustment subunit is used to adjust the preset cropping threshold to the first preset cropping threshold if it is determined that the preliminary identification result contains words in the preset whitelist.

[0036] The first trimming subunit is used to calculate the frame average number of valid states of the valid audio segments in the target speech, and trim the valid audio segments and their preliminary recognition results corresponding to the frame average number of valid states that are not less than the first preset trimming threshold to obtain the final recognition result corresponding to the target speech.

[0037] In one possible implementation, the trimming unit includes:

[0038] The second judgment subunit is used to determine whether the preliminary identification result contains words from a preset blacklist.

[0039] The second adjustment subunit is used to adjust the preset cropping threshold to the third preset cropping threshold if it is determined that the preliminary identification result contains words from the preset blacklist.

[0040] The second trimming subunit is used to calculate the frame average number of valid states of the valid audio segments in the target speech, and trim the valid audio segments and their preliminary recognition results corresponding to the frame average number of valid states that are not less than the third preset trimming threshold to obtain the final recognition result corresponding to the target speech.

[0041] In one possible implementation, the trimming unit includes:

[0042] The third adjustment subunit is used to adjust the preset cropping threshold to the second preset cropping threshold when it is determined that the preliminary identification result does not contain words in the preset whitelist or the preset blacklist.

[0043] The third trimming subunit is used to calculate the frame average number of valid states of the valid audio segments in the target speech, and trim the valid audio segments and their preliminary recognition results corresponding to the frame average number of valid states that are not less than the second preset trimming threshold, so as to obtain the final recognition result corresponding to the target speech.

[0044] In one possible implementation, the first preset cropping threshold is 2000; the second preset cropping threshold is 1600; and the third preset cropping threshold is 1130.

[0045] This application also provides a voice recognition device, including: a processor, a memory, and a system bus;

[0046] The processor and the memory are connected via the system bus;

[0047] The memory is used to store one or more programs, the one or more programs including instructions, which, when executed by the processor, cause the processor to perform any of the above-described implementations of the speech recognition method.

[0048] This application also provides a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform any of the above-described implementations of the speech recognition method.

[0049] This application also provides a computer program product, which, when run on a terminal device, causes the terminal device to execute any of the above-described implementations of the speech recognition method.

[0050] This application provides a speech recognition method, apparatus, storage medium, and device. First, it acquires the target speech to be recognized. Then, it detects the start and end frames of the target speech using real-time speech activity endpoint detection, and determines the effective audio segments in the target speech based on the start and end frames. Next, it extracts the audio features of the effective audio segments in the target speech. Then, it decodes the target speech based on the audio features to obtain a preliminary recognition result. Furthermore, it adaptively and dynamically adjusts a preset pruning threshold based on the preliminary recognition result. Finally, it prunes the target speech and the preliminary recognition result based on the adjusted pruning threshold to obtain the final recognition result corresponding to the target speech.

[0051] As can be seen, this application first uses real-time speech activity endpoint detection to more accurately detect the start and end frames of the target speech, and then adaptively and dynamically adjusts the preset pruning threshold based on the preliminary audio recognition results. This not only prunes the truly effective audio segments in the target speech, but also reduces false triggering of noise pruning, thereby effectively improving the accuracy of the recognition results and enhancing the user experience and recognition effect. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 A flowchart illustrating a speech recognition method provided in an embodiment of this application;

[0054] Figure 2 This is a schematic diagram illustrating the detection of the start and end frames of target speech using real-time speech activity endpoint detection, as provided in an embodiment of this application.

[0055] Figure 3 This is a schematic diagram illustrating the composition of a speech recognition device provided in an embodiment of this application. Detailed Implementation

[0056] Currently, commonly used speech recognition methods generally involve four steps: the first step is to use Vad to detect active endpoints and segment out valid audio; the second step is to use an acoustic neural network to extract audio features from the valid audio; the third step is to send the audio features to a decoder to decode and obtain the recognition result; and the fourth step is to send the recognition result to post-processing for normalization and then give the final recognition result.

[0057] Vad is a crucial step in speech recognition. In everyday conversations, what sounds like a continuous speech signal is actually a series of silent and spoken segments. Typically, the duration of a speech segment rarely exceeds 40% of the total speaking time. In practical applications, it's necessary to analyze the system's input signal and accurately identify the start and end points of the speech signal to acquire accurate speech data, thereby reducing the amount of data and computation.

[0058] While a standard Vad model can detect the start and end frames of an audio file and feed the valid audio data into a neural network for feature extraction, and then decode and post-process the audio features to complete the speech recognition process and output the corresponding recognition result, it may produce meaningless recognition results such as "uh-huh" or "ah-ah" in noisy audio scenarios. These results need to be trimmed during the decoding process to obtain the correct recognition result.

[0059] It is evident that the existing speech recognition methods described above have two problems: First, the start and end frames detected by Vad include margins, failing to reflect the true start and end frame positions of the audio; second, the threshold for the average number of valid states per frame is fixed during decoding and cannot be dynamically adjusted, causing some audio to be mistakenly cut off as noise. This results in inaccurate final speech recognition results.

[0060] Therefore, how to effectively reduce the probability of erroneous pruning in the speech recognition process and remove real noise audio to improve decoding efficiency and real-time speech recognition accuracy is a technical problem that urgently needs to be solved.

[0061] To address the aforementioned shortcomings, this application provides a speech recognition method. First, the target speech to be recognized is acquired. Then, the start and end frames of the target speech are detected using real-time speech activity endpoint detection. Based on the start and end frames, valid audio segments in the target speech are determined. Next, audio features of the valid audio segments in the target speech are extracted. The target speech is then decoded based on these audio features to obtain a preliminary recognition result. Furthermore, a preset pruning threshold is adaptively and dynamically adjusted based on the preliminary recognition result. Finally, the target speech and the preliminary recognition result are pruned according to the adjusted pruning threshold to obtain the final recognition result corresponding to the target speech.

[0062] As can be seen, this application first uses real-time speech activity endpoint detection to more accurately detect the start and end frames of the target speech, and then adaptively and dynamically adjusts the preset pruning threshold based on the preliminary audio recognition results. This not only prunes the truly effective audio segments in the target speech, but also reduces false triggering of noise pruning, thereby effectively improving the accuracy of the recognition results and enhancing the user experience and recognition effect.

[0063] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0064] First Embodiment

[0065] See Figure 1 This is a flowchart illustrating a speech recognition method provided in this embodiment. The method includes the following steps:

[0066] S101: Obtain the target speech to be recognized.

[0067] In this embodiment, any speech to be recognized is defined as the target speech. It should be noted that this embodiment does not limit the language type of the target speech; for example, the target speech can be Chinese or English. Furthermore, this embodiment does not limit the length of the target speech; for example, the target speech can be a sentence or a paragraph.

[0068] It is understood that the target speech can be obtained through recording or other means as needed. For example, telephone conversations in people's daily lives or recordings of conference systems can be used as target speech. At the same time as the target speech is obtained, the solution provided in this embodiment is used to process the target speech in order to identify the text information corresponding to the target speech.

[0069] S102: Detect the start and end frames of the target speech by real-time speech activity endpoint detection, and determine the valid audio segments in the target speech based on the start and end frames.

[0070] In this embodiment, after obtaining the target speech to be recognized through step S101, in order to accurately identify the text information corresponding to the target speech, this application uses Real Time Voice Activity Detection (RTVad) instead of Vad to detect the start frame and end frame of the target speech, so as to remove the margins detected by Vad, thereby more accurately determining the effective audio segments in the target speech, so as to achieve effective recognition of the target speech through the subsequent step S103.

[0071] Specifically, when performing real-time recognition of target speech, the target speech is typically input into the real-time speech recognition system frame by frame. For example, audio frames can be selected for input using a sliding window (with overlap in the middle), and the duration of the input audio frames is not limited. For instance, the sliding window value can be set to 10ms, and each speech frame can be defined as {x1, x2, x3...x...} n Then, the RTVAD method is used to detect the start and end frames of the target speech to identify the start and end positions of the valid audio segments in the target speech, filtering out invalid audio parts to obtain the truly valid audio segments in the target speech. The start and end frames of the detected valid audio segments in the target speech are defined as x. bg , and x ed , with x bg , and x ed Taking the second and (n-1)th speech frames as examples, when using RTVad to process the target speech {x1, x2, x3...x...} n After detecting the start and end frames, the representation of each speech frame in the target speech can be obtained as {x1, x2, x3}. bg ,x3...x ed x n}, where the valid audio segments are {x} bg x t ...x ed}

[0072] like Figure 2 The diagram illustrates the detection of the start and end frames of target speech using the RTVad method. In the diagram, `vad start` and `vad end` represent the start and end frame positions of the target speech detected by the ordinary `vad` method, respectively, while `rtvad start` and `rtvad end` represent the start and end frame positions of the target speech detected by the RTVad method, respectively. The audio frame position obtained by subtracting the position indicated by the `vad start` arrow from the position indicated by each adjacent `rtvad start` arrow is the margin audio frame, and the audio frame position obtained by subtracting the position indicated by the `rtvad end` arrow from the position indicated by each adjacent `vad end` arrow is also the margin audio frame. Therefore, after detecting the target speech using the RTVad method, the obtained start and end frame positions more closely match the start and end points of the actual valid audio segment.

[0073] S103: Extract the audio features of the effective audio segments in the target speech; and decode the target speech based on the audio features to obtain preliminary recognition results.

[0074] In this embodiment, the effective audio segments {x} in the target speech are determined through step S102. bg x t ...x ed Afterwards, to improve the accuracy of target speech recognition, it is necessary to further utilize existing or future feature extraction methods to extract the effective audio segments {x} from the target speech. bg x t ...x ed The audio features of the target speech are used as the basis for recognition. The target speech is decoded to obtain a preliminary recognition result, which is then used to achieve effective recognition of the target speech in the subsequent step S104.

[0075] Specifically, one possible implementation is to determine the valid audio segments {x} in the target speech. bg x t ...x ed Afterwards, it can be input into a preset neural network to extract the audio features of the effective audio segments in the target speech. This application does not limit the specific structure of the preset neural network; it can be set according to actual conditions, such as using a Long Short-Term Memory (LSTM) network or a Bi-directional Long Short-Term Memory (BiLSTM) network. Furthermore, the effective audio segments {x} in the target speech can be extracted. bg x t ...x ed The result of feature extraction from the predefined neural network is represented as {h}. bg h t ...h ed}=f({x bg x t ...x ed}), where f represents the preset neural network; h (e.g., h bg h t (etc.) represents the specific audio features extracted by the pre-defined neural network. It can be a 15003-dimensional vector representing the pronunciation probability of the logical triphone. The larger the value, the closer the pronunciation of this audio frame is to a certain phoneme. For normal audio, the probability values ​​in the 15003-dimensional feature vector are relatively concentrated and show a peak shape, while the probability values ​​in the 15003-dimensional feature vector of noisy audio are relatively scattered and have little distinguishability.

[0076] It should be noted that the embodiments of this application do not limit the method for extracting audio features of the target speech, nor do they limit the specific extraction process. Appropriate extraction methods and corresponding feature extraction operations can be selected according to the actual situation. The reason for preferentially using a preset neural network to extract audio features of effective audio segments in the target speech is that neural networks have powerful fitting and modeling capabilities and can achieve very good results in various scenarios. Therefore, neural networks can be used to extract audio features of effective audio segments in the target speech.

[0077] In addition, other feature extraction methods can also be used to extract audio features. Specifically, when extracting audio features from effective audio segments in the target speech, the effective audio segments can first be segmented into frames to obtain the corresponding speech frame sequences. Then, the segmented speech frame sequences are pre-emphasized. Then, the audio features of each speech frame are extracted sequentially. The audio features refer to the feature data used to characterize the speech information of the corresponding speech frame. For example, they can be SDC features, Mel-scale Frequency Cepstral Coefficients (MFCC) features, or Perceptual Linear Predictive (PLP) features.

[0078] Furthermore, by inputting the extracted audio features into the decoder for decoding, preliminary text recognition results can be obtained, which can then be used to execute subsequent steps S104.

[0079] S104: Based on the preliminary recognition results, the preset cropping threshold is adaptively and dynamically adjusted; and based on the adjusted cropping threshold, the target speech and the preliminary recognition results are cropped to obtain the final recognition result corresponding to the target speech.

[0080] In this embodiment, after obtaining the audio features of the effective audio segments in the target speech and the preliminary text recognition results of the target speech through step S103, in order to effectively improve the accuracy of the target speech recognition results, the sum of the acoustic states of each frame can be cached during the decoding process of inputting the extracted audio features into the decoder. In the last frame after decoding the effective audio segment, the frame average number of effective states of this effective audio segment is calculated. The specific calculation formula is as follows:

[0081]

[0082] Among them, f avgstate This represents the average number of valid states per frame in this valid audio segment, calculated from the start frame x of the valid audio segment. bg Until the end frame x edThe average of the total number of acoustic states across all frames between positions; i represents the number of audio features h corresponding to this valid audio segment; j represents the number of audio features h (e.g., hi). bg h t The vector dimension (e.g., a vector with 15003 dimensions).

[0083] It should be noted that f avgstate This reflects the reliability of the audio quality of this valid audio segment. If it is normal audio, then f avgstate The value of f is too small; if it is noisy audio, then f avgstate The value is too large. Based on this, the noise audio in the effective audio segment can be decoded and trimmed according to the preset frame average effective state number trimming threshold to obtain more accurate speech recognition results.

[0084] Specifically, to avoid generating high noise false triggers that would cause normal audio to be cut off, this application does not use a single cutting threshold for cutting. Instead, it first adaptively and dynamically adjusts the preset frame average effective state number cutting threshold based on the semantic content contained in the preliminary recognition results. Then, based on the adjusted cutting threshold, the corresponding noise audio and its corresponding preliminary text recognition results in the target speech are cut off to improve the accuracy of the target speech recognition results in noisy scenarios.

[0085] One optional implementation involves setting the preset cropping thresholds to include a first preset cropping threshold, a second preset cropping threshold, and a third preset cropping threshold; wherein the first preset cropping threshold is greater than the second preset cropping threshold, and the second preset cropping threshold is greater than the third preset cropping threshold. It should be noted that this application does not limit the specific values ​​of the first, second, and third preset cropping thresholds; they can be determined based on actual conditions and experience. For example, the first preset cropping threshold can be set to 2000; the second preset cropping threshold can be set to 1600; and the third preset cropping threshold can be set to 1130, etc.

[0086] Specifically, one possible implementation is that the specific implementation process of step S104 may include: first, determining whether the preliminary recognition result contains words from the preset whitelist; if so, adjusting the preset pruning threshold to the first preset pruning threshold (e.g., 2000); and then calculating the frame average number of valid states f of the effective audio segments in the target speech using the above formula. avgstate Then, the effective audio segments (i.e., noisy audio) corresponding to the average number of effective states of the frames that are not less than the first preset pruning threshold (e.g., 2000) and their preliminary recognition results can be pruned to obtain the final recognition result corresponding to the target speech.

[0087] In this implementation, the pre-built whitelist includes, but is not limited to, commonly used word segmentation terms, such as common internet slang, and the specific construction method is not limited. When it is determined that the preliminary recognition result contains words from the pre-built whitelist, the cropping threshold can be increased, that is, the cropping threshold can be adjusted and set to the first pre-built cropping threshold (e.g., 2000), thereby making f avgstate Normal audio with a value less than the first preset cropping threshold (e.g., 2000) will not be cropped, thus enabling the recognition results to identify the text of all normal audio as much as possible.

[0088] Another optional implementation is that the specific implementation process of this step S104 may include: first, determining whether the preliminary recognition result contains words from the preset blacklist; if so, adjusting the preset pruning threshold to a third preset pruning threshold (e.g., 1130); then, calculating the frame average number of valid states f of the effective audio segments in the target speech using the above formula. avgstate Then, the effective audio segments (i.e., noisy audio) corresponding to the average number of effective states of the frames that are not less than the third preset pruning threshold (e.g., 1130) and their preliminary recognition results can be pruned to obtain the final recognition result corresponding to the target speech.

[0089] In this implementation, the pre-built blacklist includes, but is not limited to, commonly used modal particles and uncivilized words, such as "ah" and "humph," and the specific construction method is not limited. When the preliminary recognition result is found to contain words from the pre-built blacklist, the cropping threshold can be reduced, that is, the cropping threshold can be adjusted and set to the third pre-built cropping threshold (e.g., 1130), thereby making f avgstate Normal audio with a value less than the third preset cropping threshold (e.g., 2000) will not be cropped, thus ensuring that the recognition results crop out the noisy audio as much as possible.

[0090] Another optional implementation is that the specific implementation process of step S104 may include: first, determining whether the preliminary recognition result contains words from the preset whitelist and blacklist; if it is determined that the preliminary recognition result does not contain words from either the preset whitelist or the preset blacklist, then the default preset pruning threshold is set to the second preset pruning threshold (e.g., 1600); then, the frame average number of valid states f of the effective audio segments in the target speech is calculated using the above formula. avgstate Then, the effective audio segments (i.e., noisy audio) corresponding to the average number of effective states of the frames that are not less than the second preset pruning threshold (e.g., 1600) and their preliminary recognition results can be pruned to obtain the final recognition result corresponding to the target speech.

[0091] In this way, by adaptively and dynamically adjusting the preset pruning threshold, the target speech and the preliminary recognition results are pruned, which can preserve the normal audio recognition results and remove the noisy audio, thereby improving the accuracy of speech recognition results in noisy scenarios.

[0092] In summary, the speech recognition method provided in this embodiment first acquires the target speech to be recognized, then detects the start frame and end frame of the target speech through real-time speech activity endpoint detection, and determines the effective audio segments in the target speech based on the start frame and end frame; next, it extracts the audio features of the effective audio segments in the target speech; and decodes the target speech based on the audio features to obtain a preliminary recognition result; then, it adaptively and dynamically adjusts a preset pruning threshold based on the preliminary recognition result; and prunes the target speech and the preliminary recognition result based on the adjusted pruning threshold to obtain the final recognition result corresponding to the target speech.

[0093] As can be seen, this application first uses real-time speech activity endpoint detection to more accurately detect the start and end frames of the target speech, and then adaptively and dynamically adjusts the preset pruning threshold based on the preliminary audio recognition results. This not only prunes the truly effective audio segments in the target speech, but also reduces false triggering of noise pruning, thereby effectively improving the accuracy of the recognition results and enhancing the user experience and recognition effect.

[0094] Second Embodiment

[0095] This embodiment will introduce a speech recognition device; please refer to the above method embodiment for related content.

[0096] See Figure 3 This is a schematic diagram of the composition of a voice recognition device provided in this embodiment. The device 300 includes:

[0097] Acquisition unit 301 is used to acquire the target speech to be recognized;

[0098] The detection unit 302 is used to detect the start frame and end frame of the target speech by means of real-time speech activity endpoint detection, and determine the effective audio segment in the target speech based on the start frame and end frame;

[0099] The extraction unit 303 is used to extract the audio features of the effective audio segments in the target speech; and decode the target speech according to the audio features to obtain a preliminary recognition result;

[0100] The cropping unit 304 is used to adaptively and dynamically adjust a preset cropping threshold according to the preliminary recognition result; and to crop the target speech and the preliminary recognition result according to the adjusted cropping threshold to obtain the final recognition result corresponding to the target speech.

[0101] In one implementation of this embodiment, the extraction unit 303 is specifically used for:

[0102] The effective audio segments in the target speech are input into a preset neural network to extract the audio features of the effective audio segments in the target speech.

[0103] In one implementation of this embodiment, the preset cropping threshold includes a first preset cropping threshold, a second preset cropping threshold, and a third preset cropping threshold; the first preset cropping threshold is greater than the second preset cropping threshold; and the second preset cropping threshold is greater than the third preset cropping threshold.

[0104] In one implementation of this embodiment, the trimming unit 304 includes:

[0105] The first judgment subunit is used to determine whether the preliminary identification result contains words from a preset whitelist;

[0106] The first adjustment subunit is used to adjust the preset cropping threshold to the first preset cropping threshold if it is determined that the preliminary identification result contains words in the preset whitelist.

[0107] The first trimming subunit is used to calculate the frame average number of valid states of the valid audio segments in the target speech, and trim the valid audio segments and their preliminary recognition results corresponding to the frame average number of valid states that are not less than the first preset trimming threshold to obtain the final recognition result corresponding to the target speech.

[0108] In one implementation of this embodiment, the cropping unit 304 includes:

[0109] The second judgment subunit is used to determine whether the preliminary identification result contains words from a preset blacklist.

[0110] The second adjustment subunit is used to adjust the preset cropping threshold to the third preset cropping threshold if it is determined that the preliminary identification result contains words from the preset blacklist.

[0111] The second trimming subunit is used to calculate the frame average number of valid states of the valid audio segments in the target speech, and trim the valid audio segments and their preliminary recognition results corresponding to the frame average number of valid states that are not less than the third preset trimming threshold to obtain the final recognition result corresponding to the target speech.

[0112] In one implementation of this embodiment, the cropping unit 304 includes:

[0113] The third adjustment subunit is used to adjust the preset cropping threshold to the second preset cropping threshold when it is determined that the preliminary identification result does not contain words in the preset whitelist or the preset blacklist.

[0114] The third trimming subunit is used to calculate the frame average number of valid states of the valid audio segments in the target speech, and trim the valid audio segments and their preliminary recognition results corresponding to the frame average number of valid states that are not less than the second preset trimming threshold, so as to obtain the final recognition result corresponding to the target speech.

[0115] In one implementation of this embodiment, the first preset cropping threshold is 2000; the second preset cropping threshold is 1600; and the third preset cropping threshold is 1130.

[0116] Furthermore, embodiments of this application also provide a voice recognition device, including: a processor, a memory, and a system bus;

[0117] The processor and the memory are connected via the system bus;

[0118] The memory is used to store one or more programs, the one or more programs including instructions, which, when executed by the processor, cause the processor to perform any of the above-described implementations of the speech recognition method.

[0119] Furthermore, embodiments of this application also provide a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform any of the above-described implementations of the speech recognition method.

[0120] Furthermore, this application embodiment also provides a computer program product, which, when run on a terminal device, causes the terminal device to execute any of the above-described implementation methods of the speech recognition method.

[0121] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0122] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0123] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0124] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A speech recognition method, characterized in that, include: Obtain the target speech to be recognized; The start and end frames of the target speech are detected by real-time speech activity endpoint detection, and the effective audio segments in the target speech are determined based on the start and end frames. Extract the audio features of the effective audio segments from the target speech; The target speech is then decoded based on the audio features to obtain preliminary recognition results; Based on the semantic content contained in the preliminary identification results, the preset frame average effective state number pruning threshold is adaptively and dynamically adjusted. Based on the adjusted cropping threshold, the target speech and the preliminary recognition result are cropped to obtain the final recognition result corresponding to the target speech; The preset frame average effective state number pruning threshold includes a first preset pruning threshold, a second preset pruning threshold, and a third preset pruning threshold; The first preset cropping threshold is greater than the second preset cropping threshold; The second preset cropping threshold is greater than the third preset cropping threshold; The step of cropping the target speech and the preliminary recognition result according to the adjusted cropping threshold to obtain the final recognition result corresponding to the target speech includes: Determine whether the preliminary recognition result contains words from the preset whitelist. If so, adjust the preset frame average effective state number pruning threshold to the first preset pruning threshold. After calculating the frame average effective state number of the effective audio segments in the target speech, prune the effective audio segments and their preliminary recognition results that are not less than the first preset pruning threshold to obtain the final recognition result corresponding to the target speech. Alternatively, determine whether the preliminary recognition result contains words from the preset blacklist. If so, adjust the preset frame average effective state number pruning threshold to the third preset pruning threshold. After calculating the frame average effective state number of the effective audio segments in the target speech, prune the effective audio segments and their preliminary recognition results corresponding to the frame average effective state number that is not less than the third preset pruning threshold to obtain the final recognition result corresponding to the target speech. Alternatively, if it is determined that the preliminary recognition result does not contain words from the preset whitelist or the preset blacklist, the preset frame average effective state number pruning threshold is adjusted to the second preset pruning threshold. After calculating the frame average effective state number of the effective audio segments in the target speech, the effective audio segments corresponding to the frame average effective state number that is not less than the second preset pruning threshold and their preliminary recognition results are pruned to obtain the final recognition result corresponding to the target speech.

2. The method according to claim 1, characterized in that, The extraction of audio features from the effective audio segments of the target speech includes: The effective audio segments in the target speech are input into a preset neural network to extract the audio features of the effective audio segments in the target speech.

3. The method according to any one of claims 1-2, characterized in that, The first preset cropping threshold is 2000; the second preset cropping threshold is 1600; and the third preset cropping threshold is 1130.

4. A voice recognition device, characterized in that, include: The acquisition unit is used to acquire the target speech to be recognized; The detection unit is used to detect the start frame and end frame of the target speech by means of real-time speech activity endpoint detection, and to determine the effective audio segment in the target speech based on the start frame and end frame. An extraction unit is used to extract audio features of effective audio segments from the target speech; The target speech is then decoded based on the audio features to obtain preliminary recognition results; The cropping unit is used to adaptively and dynamically adjust the preset frame average effective state number cropping threshold according to the semantic content contained in the preliminary identification result. Based on the adjusted cropping threshold, the target speech and the preliminary recognition result are cropped to obtain the final recognition result corresponding to the target speech; The preset frame average effective state number pruning threshold includes a first preset pruning threshold, a second preset pruning threshold, and a third preset pruning threshold; the first preset pruning threshold is greater than the second preset pruning threshold; the second preset pruning threshold is greater than the third preset pruning threshold; The cutting unit is specifically used for: Determine whether the preliminary recognition result contains words from the preset whitelist. If so, adjust the preset frame average effective state number pruning threshold to the first preset pruning threshold. After calculating the frame average effective state number of the effective audio segments in the target speech, prune the effective audio segments and their preliminary recognition results that are not less than the first preset pruning threshold to obtain the final recognition result corresponding to the target speech. Alternatively, determine whether the preliminary recognition result contains words from the preset blacklist. If so, adjust the preset frame average effective state number pruning threshold to the third preset pruning threshold. After calculating the frame average effective state number of the effective audio segments in the target speech, prune the effective audio segments and their preliminary recognition results corresponding to the frame average effective state number that is not less than the third preset pruning threshold to obtain the final recognition result corresponding to the target speech. Alternatively, if it is determined that the preliminary recognition result does not contain words from the preset whitelist or the preset blacklist, the preset frame average effective state number pruning threshold is adjusted to the second preset pruning threshold. After calculating the frame average effective state number of the effective audio segments in the target speech, the effective audio segments corresponding to the frame average effective state number that is not less than the second preset pruning threshold and their preliminary recognition results are pruned to obtain the final recognition result corresponding to the target speech.

5. A voice recognition device, characterized in that, include: Processor, memory, system bus; The processor and the memory are connected via the system bus; The memory is used to store one or more programs, the one or more programs including instructions that, when executed by the processor, cause the processor to perform the method according to any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a terminal device, cause the terminal device to perform the method described in any one of claims 1-3.

Images