Speech recognition method, device, electronic device and readable storage medium

Abstract

The present application discloses a speech recognition method, device, electronic device and readable storage medium, which belongs to the field of artificial intelligence technology. The method includes: performing frequency division processing on the target audio to obtain a target frame; obtaining the maximum value of the sampling point of the target frame; when the maximum values ​​of the sampling points of a1 consecutive target frames are all less than a first threshold, the consecutive a1 target frames are determined as a gap frame subsequence, and the consecutive N1 gap frame subsequences are a gap frame sequence, where a1 and N1 are both positive integers; when the maximum value of the sampling points of at least one target frame in a2 consecutive target frames is greater than or equal to the first threshold, the consecutive a2 target frames are determined as a voice frame subsequence, where a2 and N2 are both positive integers, and the consecutive N2 voice frame subsequences are a voice frame sequence; determining the truncation position according to the silent segment, the gap frame sequence and the voice frame sequence; and identifying the target frame associated with the truncation position.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence technology, and specifically relates to a speech recognition method, device, electronic device and readable storage medium. Background Art

[0002] Currently, speech recognition is divided into streaming and non-streaming. Streaming recognition recognizes speech as the user speaks, while non-streaming recognition recognizes speech after the user has finished speaking a sentence or paragraph. Streaming recognition, with its low latency, provides a better user experience and is therefore more widely used.

[0003] The streaming recognition process is as follows: for an audio clip of "a certain country is rising", when the user says "a certain country", the audio segment reaches a preset value (such as 500ms), and the recognized result is "a certain country"; the time is reset, and when the user continues to say "rise", the audio segment reaches 500ms again, and the recognized result is "rise"; and so on, until the recognition of the entire audio clip is completed. In the streaming recognition process, multiple recognitions are completed in sequence using a fixed time length as the recognition unit. Therefore, it is very likely that a certain word said by the user will appear at the boundary of two fixed time lengths, such as the end of the first 500ms and the beginning of the second 500ms, which will cause the audio of this word to be split, and then make it impossible to accurately recognize this word.

[0004] It can be seen that the accuracy of streaming speech recognition in the prior art is low. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to provide a speech recognition method that can solve the problem of low accuracy of streaming speech recognition in the prior art.

[0006] In a first aspect, an embodiment of the present application provides a speech recognition method, which includes: performing frequency division processing on the target audio to obtain a target frame; obtaining the maximum value of the sampling point of the target frame; when the maximum values ​​of the sampling points of a1 consecutive target frames are all less than a first threshold, determining the a1 consecutive target frames as a gap frame subsequence, and the consecutive N1 gap frame subsequences as a gap frame sequence, and a1 and N1 are both positive integers; when the maximum value of the sampling points of at least one target frame in a2 consecutive target frames is greater than or equal to the first threshold, determining the a2 consecutive target frames as a voice frame subsequence, a2 and N2 are both positive integers, and the consecutive N2 voice frame subsequences as a voice frame sequence; determining the truncation position according to the silent segment of the target audio, the gap frame sequence of the target audio and the voice frame sequence of the target audio; and identifying the target frame associated with the truncation position.

[0007] In a second aspect, an embodiment of the present application provides a speech recognition device, which includes: a framing module for performing frequency division processing on the target audio to obtain a target frame; an acquisition module for obtaining the maximum value of the sampling point of the target frame; a first determination module for determining the continuous a1 target frames as a gap frame subsequence when the maximum values ​​of the sampling points of the continuous a1 target frames are all less than a first threshold, and the continuous N1 gap frame subsequences are a gap frame sequence, and a1 and N1 are both positive integers; a second determination module for determining the continuous a2 target frames as a voice frame subsequence when the maximum value of the sampling points of at least one target frame in the continuous a2 target frames is greater than or equal to the first threshold, and a2 and N2 are both positive integers, and the continuous N2 voice frame subsequences are a voice frame sequence; a third determination module for determining the truncation position according to the silent segment of the target audio, the gap frame sequence of the target audio and the voice frame sequence of the target audio; an identification module for identifying the target frame associated with the truncation position.

[0008] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0009] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0010] In a fifth aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the method described in the first aspect.

[0011] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the method described in the first aspect.

[0012] Thus, in an embodiment of the present application, when the target audio is received, the received audio data is framed in real time to obtain continuous target frames. Further, a plurality of continuous target frames are taken as a judgment unit to judge whether the maximum values ​​of the sampling points of the plurality of target frames are all less than the first threshold value. If the judgment condition is met, the plurality of (a1) target frames are taken as a gap frame subsequence; based on the above judgment method, the N1 consecutive gap frame subsequences can be taken as a gap frame sequence to represent the breathing action when the user speaks. Conversely, a plurality of continuous target frames are taken as a judgment unit to judge whether the maximum values ​​of the sampling points of the plurality of target frames are greater than or equal to the first threshold value. If the judgment condition is met, the plurality of (a2) target frames are taken as a voice frame subsequence; based on the above judgment method, the N2 consecutive voice frame subsequences can be taken as a voice frame sequence to represent the voice signal spoken by the user. Thus, when performing real-time audio recognition, the truncation position is determined based on the known gap frame sequence, speech frame sequence, and silence segment, and the target frame before the truncation position is recognized. This allows the truncation position to avoid the interval of the speech frame sequence, thereby ensuring that the pronunciation of a word is not split into two recognitions. It can be seen that based on the embodiments of the present application, the accuracy of speech recognition can be improved during the streaming recognition process. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a flow chart of the speech recognition method according to an embodiment of the present application;

[0014] Figures 2 to 5 is a waveform diagram of the target audio according to an embodiment of the present application;

[0015] Figure 6 is a block diagram of a speech recognition device according to an embodiment of the present application;

[0016] Figure 7 This is one of the hardware structure diagrams of the electronic device according to the embodiment of the present application;

[0017] Figure 8 This is the second hardware structure diagram of the electronic device according to the embodiment of the present application. DETAILED DESCRIPTION

[0018] The following will be combined with the accompanying drawings of the embodiments of the present application to clearly describe the technical solutions of the embodiments of the present application. Obviously, the embodiments described are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0019] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0020] The speech recognition method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0021] Figure 1 A flowchart of a speech recognition method according to an embodiment of the present application is shown. The method is applied to an electronic device and includes:

[0022] Step 110: Perform frequency division processing on the target audio to obtain a target frame.

[0023] Before this step, the method further includes the steps of receiving input, and receiving target audio in response to the input.

[0024] Application scenarios include: a user long presses the voice recognition button on the device to start receiving the target audio.

[0025] Optionally, the target audio is received by a microphone.

[0026] In this step, the received audio data is framed in real time to obtain target frames.

[0027] It should be noted that this application is applicable to streaming speech recognition, and correspondingly, the target audio is used to indicate the currently received audio. As the amount of received audio data gradually increases, the audio data to be processed is updated accordingly, and the number of target frames obtained also gradually increases.

[0028] For example, the audio data currently being processed is 25ms, and one frame is obtained after framing. As more audio data is received, more frames are gradually obtained.

[0029] Step 120: Obtain the maximum value of the sampling points of the target frame.

[0030] In this step, the absolute values ​​of all sampling points of the target frame are taken, and the maximum value of all sampling points is used as the sampling value of the target frame.

[0031] The positive and negative values ​​of the sampling points of the target frame represent the displacement of the microphone membrane caused by vibration near the equilibrium position. The displacement of the equilibrium position is "0", and the displacements on both sides are positive and negative, respectively.

[0032] In addition, in this application, the maximum value among the sampling points is used as the sampling value of the target frame to determine whether the target frame includes a speech signal, because the larger the numerical value of the sampling value, the stronger the ability of the sound emitted.

[0033] Step 130: When the maximum values ​​of the sampling points of a1 consecutive target frames are all less than the first threshold, the a1 consecutive target frames are determined as an interstitial frame subsequence, and the N1 consecutive interstitial frame subsequences are determined as an interstitial frame sequence, where a1 and N1 are both positive integers.

[0034] In this step, a1 consecutive target frames are used as a judgment unit. If the maximum value corresponding to any target frame in the a1 consecutive target frames is less than the first threshold (n), then the sampling values ​​of the target frames in this group are considered to be non-n, and the target frames in this group are defined as a gap frame subsequence. Furthermore, if the sampling values ​​of N1 consecutive gap frame subsequences are non-n, then the sequence is considered to be a gap frame sequence.

[0035] For example, the speech received by the microphone is: "How is the weather today?" and "I'm going to the park", and the complete waveforms corresponding to these two sentences are as follows: Figure 2 As shown. When the user speaks, he will breathe, which is reflected in the waveform. The sampling value of the waveform during breathing is very low, or even zero. Therefore, based on the above phenomenon, this embodiment performs real-time frame processing on the received target audio to find the gap frame sequence in the target audio, wherein the gap frame sequence is used to indicate that there is no voice signal or the voice signal is weak. Thus, see Figure 3 In the waveform diagram, the waveforms before and after ventilation are distinguished, where a box is used to represent the waveform corresponding to the text above, and the small gaps between adjacent boxes are used to represent the gap frame sequence.

[0036] Step 140: When the maximum value of the sampling points of at least one target frame among a2 consecutive target frames is greater than or equal to the first threshold, the a2 consecutive target frames are determined as a speech frame subsequence, a2 and N2 are both positive integers, and the N2 consecutive speech frame subsequences are a speech frame sequence.

[0037] In this step, if the maximum value corresponding to at least one target frame among a2 consecutive target frames is greater than or equal to the first threshold (n), the maximum value of the sampling points of the group of target frames is reset to "n", and the group of target frames is defined as a speech frame subsequence. Furthermore, if the sampling values ​​of N2 consecutive speech frame subsequences are all "n", the sequence is considered to be a speech frame sequence.

[0038] Typically, based on the characteristic of high sound intensity when people speak, continuous frames containing speech signals are grouped together as a speech frame sequence for subsequent recognition, thereby avoiding inaccurate recognition caused by the speech signal of a certain word being split.

[0039] Therefore, the speech frame sequence in this step is used to indicate that there is a speech signal or the speech signal is relatively strong.

[0040] Optionally, a1 and a2 have the same value so that audio of the same length is selected for processing.

[0041] In one audio segment, gap frame sequences and speech frame sequences appear alternately.

[0042] For example, see Figure 4 , there is a gap frame sequence between every two speech frame sequences.

[0043] Step 150: Determine a truncation position according to the silent segment of the target audio, the gap frame sequence of the target audio, and the speech frame sequence of the target audio.

[0044] In this step, based on the gap frame sequence and speech frame sequence in the target audio and the silent segment in the target audio, the truncation position is determined during the speech recognition process.

[0045] Optionally, based on the feature information of each frame, a speech segment or a silence segment is determined in the received target audio.

[0046] Exemplarily, the features of each frame are processed through a Voice Activity Detection (VAD) algorithm to obtain speech segments and silence segments.

[0047] See also Figure 5 For example, the complete audio received is "What's the weather like today? I'm going to the park". "What's the weather like today" and "I'm going to the park" correspond to voice segments. There is a pause between the two sentences, and the interval is long, which corresponds to a silent segment.

[0048] In an audio segment, speech segments and silence segments appear alternately.

[0049] See also Figure 5 , for example, there is a silence segment between two speech segments.

[0050] Optionally, in combination with the speech frame sequence in the target audio, the truncation position is determined in the interval of the interstitial frame sequence and the silent segment of the target audio, that is, the audio is truncated at the user's breathing or pause point, so that the audio on both sides of the truncated audio represent complete words or phrases respectively.

[0051] Step 160: Identify the target frame associated with the truncation location.

[0052] In this step, after the truncation position is determined, the audio before the truncation position is recognized.

[0053] For example, based on the features of streaming recognition, a certain position of the target audio can be used as the starting point and the truncation position as the end point, and all audio signals between the starting point and the end point can be recognized.

[0054] Thus, in an embodiment of the present application, when the target audio is received, the received audio data is framed in real time to obtain continuous target frames. Further, a plurality of continuous target frames are taken as a judgment unit to judge whether the maximum values ​​of the sampling points of the plurality of target frames are all less than the first threshold value. If the judgment condition is met, the plurality of (a1) target frames are taken as a gap frame subsequence; based on the above judgment method, the N1 consecutive gap frame subsequences can be taken as a gap frame sequence to represent the breathing action when the user speaks. Conversely, a plurality of continuous target frames are taken as a judgment unit to judge whether the maximum values ​​of the sampling points of the plurality of target frames are greater than or equal to the first threshold value. If the judgment condition is met, the plurality of (a2) target frames are taken as a voice frame subsequence; based on the above judgment method, the N2 consecutive voice frame subsequences can be taken as a voice frame sequence to represent the voice signal spoken by the user. Thus, when performing real-time audio recognition, the truncation position is determined based on the known gap frame sequence, speech frame sequence, and silence segment, and the target frame before the truncation position is recognized. This allows the truncation position to avoid the interval of the speech frame sequence, thereby ensuring that the pronunciation of a word is not split into two recognitions. It can be seen that based on the embodiments of the present application, the accuracy of speech recognition can be improved during the streaming recognition process.

[0055] In the process of the speech recognition method of another embodiment of the present application, step 150 includes:

[0056] Sub-step A1: when the number of the detected speech frame sequences meets the second threshold, determining a truncation position in the interval where the next gap frame sequence adjacent to the last speech frame sequence is located.

[0057] Optionally, the second threshold is "2" to achieve real-time speech recognition function.

[0058] For reference, the implementation process of this embodiment is as follows: first, a count value (count) for counting speech frame sequences is set, and count is set to 0; second, each consecutive sequence is sequentially detected to see if it is a speech frame sequence. If so, count is incremented by 1 based on the original value; if not, count remains unchanged until count = 2. Furthermore, after completing this detection, count is reset to 0, and detection continues.

[0059] For example, if the first sequence is detected as a speech frame sequence, then count=1; if the second sequence is detected as a gap frame sequence, then count=1; if the third sequence is detected as a speech frame sequence, then count=2.

[0060] In the present embodiment, the truncation position is determined in the interval where the next gap frame sequence adjacent to the last speech frame sequence is located.

[0061] Optionally, since the gap frame sequence includes at least a1 target frames, the truncation position may be determined at the a1th frame after the last speech frame sequence.

[0062] For example, based on the above example, the fourth sequence is a gap frame sequence, and a truncation position is determined in the fourth sequence.

[0063] The target frame associated with the truncation position includes: a target frame detected in a continuous speech frame sequence, and a target frame detected in a gap frame sequence spaced between continuous speech frame sequences.

[0064] In this embodiment, the target frames to be recognized include: all target frames from the first sequence of the current detection to the end of the truncation position. This recognition method can recognize the intervals between words and improve the recognition accuracy.

[0065] Optionally, the detected speech frame sequence and the gap frame sequence between each speech frame sequence are input into the model at the same time to output the corresponding recognition result.

[0066] For example, based on the above example, the first sequence, the second sequence, the third sequence, and the first a1 frame of the fourth sequence are input into the model at the same time to output corresponding recognition results.

[0067] In this embodiment, a recognition method is provided, which uses whether the number of detected speech frame sequences meets a second threshold as a condition to determine whether to truncate the audio, thereby achieving real-time recognition and reducing recognition delay while ensuring recognition accuracy.

[0068] In the process of the speech recognition method of another embodiment of the present application, step 150 includes:

[0069] Sub-step B1: when a first speech frame sequence and a first silence segment are detected in sequence, a truncation position is determined in the interval where the first silence segment is located.

[0070] In this embodiment, during the detection process, if a speech frame sequence is followed by a silent segment, it is considered to be a sentence pause, and the truncation position is determined in the silent segment.

[0071] Optionally, since the pause duration of a silent segment must be greater than the duration of a gap frame sequence, a silent segment includes at least a1 target frames, and the truncation position can be determined at the a1th frame after the first speech frame sequence.

[0072] The target frame associated with the truncation position includes: a target frame in a first speech segment, and the first speech segment includes a first speech frame sequence.

[0073] In this embodiment, the target frames to be recognized include all target frames from the start of the previous speech segment immediately preceding the first silence segment to the truncation position. This recognition method not only identifies the spaces between words but also pauses in sentences, thereby improving recognition accuracy.

[0074] The first speech frame sequence is the last speech frame sequence in the first speech segment.

[0075] In this step, based on the recognition of the entire speech segment, other recognition results in the speech segment that have been displayed on the screen can also be updated.

[0076] Optionally, all target frames to be recognized are input into the model at the same time to perform overall recognition on the first speech segment.

[0077] In this embodiment, when a silent segment is recognized, a speech segment immediately preceding the silent segment may also be recognized as a whole, so as to update the recognition result in combination with context information and further improve the accuracy of speech recognition.

[0078] In the process of the speech recognition method of another embodiment of the present application, step 150 includes:

[0079] Sub-step C1: when a target input is detected at the next moment of the second speech frame sequence, a truncation position is determined in the corresponding interval at the next moment of the second speech frame sequence, and the target input is used to stop receiving the target audio.

[0080] Target input includes touch input by the user on the screen, not limited to clicks, slides, and drags. It can also include airborne input, such as hand gestures and facial movements. It also includes input on physical buttons on the device, not limited to presses. Furthermore, target input includes one or more inputs, where multiple inputs can be continuous or spaced apart.

[0081] In this step, the target input is used to stop receiving the target audio.

[0082] An application scenario is when a user continues to press the voice recognition button and then releases it.

[0083] Optionally, since the target input is used to indicate the end of a speech segment, the interval occupied in the audio must be greater than the duration of a gap frame sequence. Therefore, the truncation position can be determined at the a1 frame after the second speech frame sequence.

[0084] The target frame associated with the truncation position includes: a target frame in the second speech segment, and the second speech segment includes a second speech frame sequence.

[0085] The second speech frame sequence is the last speech frame sequence in the second speech segment.

[0086] In this embodiment, the target frames to be recognized include all target frames from the start position of the previous speech segment immediately preceding the target input to the truncation position. This recognition method can not only identify the intervals between words, but also identify the end of sentences, thereby improving recognition accuracy.

[0087] In this step, based on the recognition of the entire speech segment, other recognition results in the speech segment that have been displayed on the screen can also be updated.

[0088] In this embodiment, when an instruction to stop receiving the target audio is recognized, a speech segment before receiving the instruction can also be recognized as a whole to update the recognition result in combination with context information, thereby further improving the accuracy of speech recognition.

[0089] In the process of the speech recognition method of another embodiment of the present application, the method further includes:

[0090] Step D1: Determine the value of a and the first threshold value through a clustering algorithm.

[0091] Among them, a1=a2=a.

[0092] Optionally, the value of a and the first threshold are determined by a k-means clustering algorithm (k-means clustering algorithm for short).

[0093] For reference, the value of a is "5" and the first threshold (n) is "1000".

[0094] In this embodiment, a and n are mainly obtained by using statistical methods.

[0095] For reference, in the first step, the speech data set is divided into frames, the absolute values ​​of all sampling points in each frame are taken, and the maximum value of all sampling points in each frame is taken as the sampling value of the current frame; all speech segments and silent segments in the speech data set are obtained by the VAD algorithm; in the second step, the sampling values ​​of all speech frames and gap frames belonging to the speech segments in the speech data set are respectively obtained by the k-means clustering algorithm (assuming k is 3) to obtain the clustered sampling values ​​of the speech frames and gap frames. The clustered sampling values ​​of the speech frames are sorted from small to large as x1, x2, and x3, and the clustered sampling values ​​of the gap frames are sorted from small to large as x1, x2, and x3, respectively. are y1, y2, and y3; in the third step, formula 1 is used to take the average of the medians of the clustered sampling values ​​of the speech frames and gap frames to determine n, where n is used to distinguish between speech frames and gap frames to eliminate the influence of the sampling values ​​of the speech frames and gap frames being too large or too small on the overall sampling value; in addition, k-means clustering is performed on the frame distance of the speech frames belonging to a spoken text whose sampling values ​​are less than n to obtain a3, a4, and a5, and finally a=a4, where a is used to limit the length of the speech frame subsequence and the gap frame subsequence, and indirectly limit the length of the speech frame sequence and the gap frame sequence.

[0096] Among them, formula 1: n = (x2 + y2) / 2.

[0097] In this embodiment, the aggregation algorithm is used to obtain the value of "a" and the first threshold value. On the one hand, the length of the speech frame subsequence or the gap frame subsequence is limited by the value of a. For example, the value of a is "5", that is, every 5 frames are regarded as a subsequence. In this way, there is no need to judge each frame in turn to reduce the processing amount, and it is ensured that when the user says a word (or multiple consecutive words) without breathing, the corresponding number of frames must be more than 5 frames. On the other hand, the sampling value of the voice signal is limited by the first threshold value to distinguish well when speaking and when breathing. It can be seen that this embodiment can further ensure the accuracy of recognition when the relevant parameters determined are valid.

[0098] In the embodiments provided above in this application, they can be implemented individually or in combination with each other.

[0099] Optionally, in combination with the above embodiments, including the detection of speech frame sequences, gap frame sequences, speech segments, silence segments, and target inputs, an implementation manner of the present application is as follows:

[0100] First step: Start with count = 0. Detect the speech frame sequence of "今", count = 1. There is a gap frame after the current speech frame sequence, and continue processing.

[0101] Second step: Detect the speech frame sequence of "天天", count = 2. Since there is a gap frame sequence after the current speech frame sequence, take the 5 frames after the speech frame sequence of "天天" as the truncation point (i.e., the truncation position). Therefore, input the two consecutive speech frame sequences of "今" "天天" and the intermediate gap frame sequence into the model for speech recognition, obtain the result "今天天", display the result "今天天" on the screen, and reset count to 0.

[0102] Third step: Detect the speech frame sequence of "气", count = 1. There is a gap frame after the current speech frame sequence, and continue processing.

[0103] Fourth step: Detect the speech frame sequence of "怎", count = 2. Since there is a gap frame sequence after the current speech frame sequence, take the 5 frames after the speech frame sequence of "气怎" as the truncation point. Therefore, input the two consecutive speech frame sequences of "气" "怎" and the intermediate gap frame sequence into the model for speech recognition, obtain the result "气怎", display the result "今天天气怎" on the screen, and reset count to 0.

[0104] Fifth step: Detect the speech frame sequence of "么", count = 1. There is a gap frame after the current speech frame sequence, and continue processing.

[0105] Sixth step: Detect the speech frame sequence of "样", count = 2. Since there is a silence segment after the current speech frame sequence, take the 5 frames after the speech frame sequence of "样" as the truncation point. Therefore, input the speech segment features of "今" "天天" "怎" "么" "样" into the model for speech recognition, obtain the result "今天天气怎么样", display the result "今天天气怎么样" on the screen, and reset count to 0.

[0106] Seventh step: Detect the speech frame sequence of "我", count = 1. There is a gap frame after the current speech frame sequence, and continue processing.

[0107] Eighth step: Detect the speech frame sequence of "去". Since there is a gap frame sequence after the current speech frame sequence, take the 5 frames after the speech frame sequence of "去" as the truncation point. Therefore, input the two consecutive speech frame sequences of "我" "去" and the intermediate gap frame sequence into the model for speech recognition, obtain the result "我去", display the result "今天天气怎么样,我去" on the screen, and reset count to 0.

[0108] Step 9: The voice frame sequence of "gong" is detected, count = 1. Since the current voice frame sequence is followed by a gap frame, continue the processing.

[0109] Step 10: The voice frame sequence of "yuan" is detected. Since the user releases the voice recognition button after the current voice frame sequence, the voice segment features of "wo", "qu", "gong", and "yuan" are input into the model for voice recognition, and the result "I go to the park" is obtained. The on-screen result is "What's the weather like today, I go to the park".

[0110] In summary, in this application, the threshold is obtained through the clustering algorithm to distinguish the voice frame sequence and the gap frame sequence. A voice frame sequence with a fixed length (such as two) is selected to implement streaming voice recognition. At the same time, truncation is performed through the gap frames between audio and text, so that the audio containing a certain text is not divided into two audio recognitions, improving the streaming on-screen accuracy and user experience. In addition, by re-recognizing the audio features in each voice segment of the audio and updating the previous on-screen result, the voice recognition accuracy is improved.

[0111] For the voice recognition method provided in the embodiment of this application, the execution subject can be a voice recognition device. In the embodiment of this application, taking the voice recognition device executing the voice recognition method as an example, the voice recognition device provided in the embodiment of this application is described.

[0112] Figure 6 The block diagram of the voice recognition device according to another embodiment of this application is shown. The device includes:

[0113] Frequency division module 10, configured to perform frequency division processing on the target audio to obtain target frames;

[0114] Obtaining module 20, configured to obtain the maximum value of the sampling points of the target frames;

[0115] First determination module 30, configured to determine a continuous a1 target frames as a gap frame subsequence when the maximum values of the sampling points of continuous a1 target frames are all less than the first threshold, and continuous N1 gap frame subsequences are a gap frame sequence, where a1 and N1 are both positive integers;

[0116] Second determination module 40, configured to determine a continuous a2 target frames as a voice frame subsequence when at least one of the maximum values of the sampling points of continuous a2 target frames is greater than or equal to the first threshold, where a2 and N2 are both positive integers, and continuous N2 voice frame subsequences are a voice frame sequence;

[0117] Third determination module 50, configured to determine the truncation position according to the mute segment of the target audio, the gap frame sequence of the target audio, and the voice frame sequence of the target audio;

[0118] The identification module 60 is configured to identify a target frame associated with a truncation position.

[0119] Thus, in an embodiment of the present application, when the target audio is received, the received audio data is framed in real time to obtain continuous target frames. Further, a plurality of continuous target frames are taken as a judgment unit to judge whether the maximum values ​​of the sampling points of the plurality of target frames are all less than the first threshold value. If the judgment condition is met, the plurality of (a1) target frames are taken as a gap frame subsequence; based on the above judgment method, the N1 consecutive gap frame subsequences can be taken as a gap frame sequence to represent the breathing action when the user speaks. Conversely, a plurality of continuous target frames are taken as a judgment unit to judge whether the maximum values ​​of the sampling points of the plurality of target frames are greater than or equal to the first threshold value. If the judgment condition is met, the plurality of (a2) target frames are taken as a voice frame subsequence; based on the above judgment method, the N2 consecutive voice frame subsequences can be taken as a voice frame sequence to represent the voice signal spoken by the user. Thus, when performing real-time audio recognition, the truncation position is determined based on the known gap frame sequence, speech frame sequence, and silence segment, and the target frame before the truncation position is recognized. This allows the truncation position to avoid the interval of the speech frame sequence, thereby ensuring that the pronunciation of a word is not split into two recognitions. It can be seen that based on the embodiments of the present application, the accuracy of speech recognition can be improved during the streaming recognition process.

[0120] Optionally, the third determining module 50 includes:

[0121] A first determining unit is configured to determine a truncation position in an interval where a next gap frame sequence adjacent to the last voice frame sequence is located when the number of detected voice frame sequences meets a second threshold;

[0122] The target frame associated with the truncation position includes: a target frame detected in a continuous speech frame sequence, and a target frame detected in a gap frame sequence spaced between continuous speech frame sequences.

[0123] Optionally, the third determining module 50 includes:

[0124] A second determining unit is configured to determine a truncation position in an interval where the first silence segment is located when the first speech frame sequence and the first silence segment are detected in sequence;

[0125] The target frame associated with the truncation position includes: a target frame in a first speech segment, and the first speech segment includes a first speech frame sequence.

[0126] Optionally, the third determining module 50 includes:

[0127] a third determining unit, configured to determine a truncation position in a corresponding interval at the next moment of the second speech frame sequence when a target input is detected at the next moment of the second speech frame sequence, the target input being used to stop receiving the target audio;

[0128] The target frame associated with the truncation position includes: a target frame in the second speech segment, and the second speech segment includes a second speech frame sequence.

[0129] The speech recognition device in the embodiments of the present application can be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other device other than a terminal. For example, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a mobile Internet device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc. It can also be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., and the embodiments of the present application do not specifically limit it.

[0130] The speech recognition device of the embodiment of the present application may be a device having an action system. The action system may be an Android action system, an iOS action system, or other possible action systems, which are not specifically limited in the embodiment of the present application.

[0131] The speech recognition device provided in the embodiment of the present application can implement each process implemented in the above method embodiment. To avoid repetition, it will not be described here.

[0132] Alternatively, as Figure 7 As shown, an embodiment of the present application also provides an electronic device 100, including a processor 101, a memory 102, and a program or instruction stored in the memory 102 and executable on the processor 101. When the program or instruction is executed by the processor 101, each step of any of the above-mentioned speech recognition method embodiments is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0133] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0134] Figure 8 A schematic diagram of the hardware structure of an electronic device implementing an embodiment of the present application.

[0135] The electronic device 1000 includes but is not limited to components such as a radio frequency unit 1001 , a network module 1002 , an audio output unit 1003 , an input unit 1004 , a sensor 1005 , a display unit 1006 , a user input unit 1007 , an interface unit 1008 , a memory 1009 , and a processor 1010 .

[0136] Those skilled in the art will understand that the electronic device 1000 may also include a power source (such as a battery) to power each component, and the power source may be logically connected to the processor 1010 through a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system. Figure 8 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be repeated here.

[0137] Among them, the processor 1010 is used to perform frequency division processing on the target audio to obtain a target frame; obtain the maximum value of the sampling point of the target frame; when the maximum values ​​of the sampling points of a1 consecutive target frames are all less than a first threshold, the consecutive a1 target frames are determined as a gap frame subsequence, and the consecutive N1 gap frame subsequences are a gap frame sequence, and a1 and N1 are both positive integers; when the maximum value of the sampling point of at least one target frame among a2 consecutive target frames is greater than or equal to the first threshold, the consecutive a2 target frames are determined as a voice frame subsequence, a2 and N2 are both positive integers, and the consecutive N2 voice frame subsequences are a voice frame sequence; determine the truncation position according to the silent segment of the target audio, the gap frame sequence of the target audio and the voice frame sequence of the target audio; identify the target frame associated with the truncation position.

[0138] Thus, in an embodiment of the present application, when the target audio is received, the received audio data is framed in real time to obtain continuous target frames. Further, a plurality of continuous target frames are taken as a judgment unit to judge whether the maximum values ​​of the sampling points of the plurality of target frames are all less than the first threshold value. If the judgment condition is met, the plurality of (a1) target frames are taken as a gap frame subsequence; based on the above judgment method, the N1 consecutive gap frame subsequences can be taken as a gap frame sequence to represent the breathing action when the user speaks. Conversely, a plurality of continuous target frames are taken as a judgment unit to judge whether the maximum values ​​of the sampling points of the plurality of target frames are greater than or equal to the first threshold value. If the judgment condition is met, the plurality of (a2) target frames are taken as a voice frame subsequence; based on the above judgment method, the N2 consecutive voice frame subsequences can be taken as a voice frame sequence to represent the voice signal spoken by the user. Thus, when performing real-time audio recognition, the truncation position is determined based on the known gap frame sequence, speech frame sequence, and silence segment, and the target frame before the truncation position is recognized. This allows the truncation position to avoid the interval of the speech frame sequence, thereby ensuring that the pronunciation of a word is not split into two recognitions. It can be seen that based on the embodiments of the present application, the accuracy of speech recognition can be improved during the streaming recognition process.

[0139] Optionally, the processor 1010 is further used to determine the truncation position in the interval where the next gap frame sequence adjacent to the last voice frame sequence is located when the number of detected voice frame sequences meets a second threshold; wherein the target frame associated with the truncation position includes: the target frame in the detected continuous voice frame sequence, and the target frame in the gap frame sequence between the detected continuous voice frame sequences.

[0140] Optionally, the processor 1010 is also used to determine the truncation position in the interval where the first silence segment is located when a first speech frame sequence and a first silence segment are detected in sequence; wherein the target frame associated with the truncation position includes: the target frame in the first speech segment, and the first speech segment includes the first speech frame sequence.

[0141] Optionally, the processor 1010 is further used to determine a truncation position in a corresponding interval at the next moment of the second voice frame sequence when a target input is detected at the next moment of the second voice frame sequence, and the target input is used to stop receiving the target audio; wherein the target frame associated with the truncation position includes: the target frame in the second voice segment, and the second voice segment includes the second voice frame sequence.

[0142] In summary, this application uses a clustering algorithm to obtain a threshold value to distinguish between voice frame sequences and gap frame sequences, selects a fixed length (such as two) of voice frame sequences to realize streaming voice recognition, and at the same time truncates the audio by the gap frames between the audio text, so that the audio containing a certain text is not divided into two audio recognitions, thereby improving the streaming screen accuracy and user experience. In addition, by re-recognizing the audio features in each voice segment in the audio and updating the previous screen results, the voice recognition accuracy is improved.

[0143] It should be understood that in an embodiment of the present application, the input unit 1004 may include a graphics processing unit (GPU) 10041 and a microphone 10042, and the graphics processor 10041 processes the image data of a static picture or video image obtained by an image capture device (such as a camera) in a video image capture mode or an image capture mode. The display unit 1006 may include a display panel 10061, and the display panel 10061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 1007 includes a touch panel 10071 and at least one of the other input devices 10072. The touch panel 10071 is also called a touch screen. The touch panel 10071 may include two parts: a touch detection device and a touch controller. Other input devices 10072 may include but are not limited to a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and an action stick, which will not be repeated here. The memory 1009 can be used to store software programs and various data, including but not limited to applications and action systems. The processor 1010 may integrate an application processor and a modem processor, wherein the application processor mainly processes the action system, user pages and applications, etc., and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 1010.

[0144] The memory 1009 can be used to store software programs and various data. The memory 1009 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, applications or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 1009 may include a volatile memory or a non-volatile memory, or the memory 1009 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM), and a direct memory bus random access memory (DRRAM). The memory 1009 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0145] Processor 1010 may include one or more processing units. Optionally, processor 1010 integrates an application processor and a modem processor. The application processor primarily handles operations related to the operating system, user interface, and application programs, while the modem processor primarily processes wireless communication signals, such as a baseband processor. It is understood that the modem processor may not be integrated into processor 1010.

[0146] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned speech recognition method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0147] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0148] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned speech recognition method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0149] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0150] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned speech recognition method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0151] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0152] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0153] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. A speech recognition method, characterized in that: The method comprises: Perform frame processing on the target audio to obtain the target frame; Obtaining the maximum value of the sampling points of the target frame; When the maximum values ​​of the sampling points of a1 consecutive target frames are all less than the first threshold, the a1 consecutive target frames are determined as an interstitial frame subsequence, and the N1 consecutive interstitial frame subsequences are determined as an interstitial frame sequence, where a1 and N1 are both positive integers; If the maximum value of the sampling points of at least one target frame among a2 consecutive target frames is greater than or equal to the first threshold, the a2 consecutive target frames are determined as a speech frame subsequence, and the N2 consecutive speech frame subsequences are determined as a speech frame sequence, where a2 and N2 are both positive integers; Determining a truncation position according to a silence segment of the target audio, a gap frame sequence of the target audio, and a speech frame sequence of the target audio; A target frame associated with the truncation location is identified.

2. The method according to claim 1, characterized in that The determining of the truncation position according to the silent segment of the target audio, the gap frame sequence of the target audio, and the speech frame sequence of the target audio includes: When the number of the detected speech frame sequences meets the second threshold, determining a truncation position in an interval where a next gap frame sequence adjacent to the last speech frame sequence is located; The target frame associated with the truncation position includes: a target frame detected in a continuous speech frame sequence, and a target frame detected in a gap frame sequence spaced between continuous speech frame sequences.

3. The method according to claim 1, characterized in that The determining of the truncation position according to the silent segment of the target audio, the gap frame sequence of the target audio, and the speech frame sequence of the target audio includes: When a first speech frame sequence and a first silence segment are detected in sequence, determining a truncation position in an interval where the first silence segment is located; The target frame associated with the truncation position includes: a target frame in a first speech segment, and the first speech segment includes the first speech frame sequence.

4. The method according to claim 1, wherein The determining of the truncation position according to the silent segment of the target audio, the gap frame sequence of the target audio, and the speech frame sequence of the target audio includes: determining a truncation position in a corresponding interval at the next moment of the second speech frame sequence when a target input is detected at the next moment of the second speech frame sequence, wherein the target input is used to stop receiving the target audio; The target frame associated with the truncation position includes: a target frame in a second speech segment, and the second speech segment includes the second speech frame sequence.

5. A speech recognition device, characterized in that: The device comprises: The frequency division module is used to perform frame processing on the target audio to obtain the target frame; An acquisition module, configured to acquire a maximum value of the sampling points of the target frame; A first determining module is configured to, when the maximum values ​​of the sampling points of a1 consecutive target frames are all less than a first threshold, determine the a1 consecutive target frames as an interstitial frame subsequence, and determine N1 consecutive interstitial frame subsequences as an interstitial frame sequence, where a1 and N1 are both positive integers; a second determining module, configured to, if, among a2 consecutive target frames, the maximum value of the sampling points of at least one target frame is greater than or equal to the first threshold, determine the a2 consecutive target frames as a speech frame subsequence, and the N2 consecutive speech frame subsequences as a speech frame sequence, where a2 and N2 are both positive integers; A third determining module is configured to determine a truncation position according to a silence segment of the target audio, a gap frame sequence of the target audio, and a speech frame sequence of the target audio; The identification module is used to identify the target frame associated with the truncation position.

6. The device according to claim 5, characterized in that The third determining module includes: A first determining unit is configured to determine a truncation position in an interval where a next gap frame sequence adjacent to the last voice frame sequence is located when the number of detected voice frame sequences meets a second threshold; The target frame associated with the truncation position includes: a target frame detected in a continuous speech frame sequence, and a target frame detected in a gap frame sequence spaced between continuous speech frame sequences.

7. The device according to claim 5, characterized in that The third determining module includes: a second determining unit, configured to determine a truncation position in an interval where the first silence segment is located when a first speech frame sequence and a first silence segment are detected in sequence; The target frame associated with the truncation position includes: a target frame in a first speech segment, and the first speech segment includes the first speech frame sequence.

8. The device according to claim 5, characterized in that The third determining module includes: a third determining unit, configured to determine a truncation position in a corresponding interval at the next moment of the second speech frame sequence when a target input is detected at the next moment of the second speech frame sequence, wherein the target input is used to stop receiving the target audio; The target frame associated with the truncation position includes: a target frame in a second speech segment, and the second speech segment includes the second speech frame sequence.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the speech recognition method according to any one of claims 1 to 4 are implemented.

10. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of the speech recognition method according to any one of claims 1 to 4 are implemented.

Images