Audio interception method and device in short speech recognition, equipment, medium and product

Abstract

The invention provides an audio interception method and device in short speech recognition, equipment, a medium and a product, and relates to the technical field of big data. According to the method, a real-time audio data stream is obtained and framed, sound energy of each audio frame is determined and sent to a voice activity detection module to obtain voice activity detection attributes, and then a user voice starting time point is determined according to the sound energy of each audio frame and the voice activity detection attributes in a time sequence; and determining a termination time point according to the starting time point and the voice activity detection attribute, and finally intercepting a target audio frame between the starting time point and the termination time point and sending the target audio frame to a voice recognition module. According to the method provided by the invention, the problem of poor environmental adaptability of a fixed energy threshold scheme is solved, and the defects that the interception boundary is inaccurate and the voice integrity is difficult to guarantee due to the fact that voice activity detection is easily interfered by burst noise are avoided, so that the accuracy and the reliability of short voice audio interception are improved.

Description

Technical Field

[0001] This application relates to the field of big data technology, and in particular to audio extraction methods, apparatus, devices, media and products in short speech recognition. Background Technology

[0002] Currently, with the popularization of voice interaction technology, short speech recognition has become one of the core functions of smart devices. Short speech recognition refers to converting a brief (usually no more than 60 seconds) segment of speech into text, replacing traditional input scenarios, and can be applied to scenarios such as voice search and human-computer interaction. Specifically, short speech recognition relies on accurately extracting segments containing valid user speech from the real-time audio data stream captured by the microphone. If the extraction range is too large, it will increase the processing load of the speech recognition module, leading to increased recognition latency; if the extraction range is too small, it will destroy the integrity of the speech semantics, resulting in recognition errors; if background noise is mistakenly identified as the start or end of the speech, it will directly cause extraction failure.

[0003] In existing technologies, audio segmentation schemes mainly employ two methods to extract short speech audio. One method uses a fixed energy threshold to determine whether an audio frame is speech or non-speech, but this approach suffers from poor environmental adaptability. The other method uses a voice activity detection (VAD) module to extract spectral and pitch features from the audio to distinguish between speech and non-speech. However, VAD is susceptible to sudden noise interference, potentially misinterpreting short bursts of high-energy noise as the start of speech or short pauses in the middle of speech as the end of speech, resulting in incomplete extracted speech segments. Therefore, existing short speech audio segmentation methods suffer from poor environmental adaptability, inaccurate segmentation boundaries, and difficulty in guaranteeing speech integrity. Summary of the Invention

[0004] This application provides an audio truncation method, apparatus, device, medium, and product for short speech recognition, to solve the problems of poor environmental adaptability, inaccurate truncation boundaries, and difficulty in ensuring speech integrity in existing short speech audio truncation methods.

[0005] In a first aspect, this application provides an audio truncation method for short speech recognition, including:

[0006] Acquire the real-time audio data stream from the microphone and divide the audio data stream into frames to obtain multiple audio frames;

[0007] The sound energy of each audio frame is determined, the sound energy of each audio frame is sent to the speech activity detection module for detection, and the speech activity detection attributes of each audio frame returned by the speech activity detection module are received.

[0008] Based on the sound energy and speech activity detection attributes of each audio frame, the audio frames are traversed in chronological order to determine the start time of the user's speech.

[0009] Based on the start time point and the speech activity detection attributes of each audio frame, the end time point of the user's speech is determined.

[0010] At least one target audio frame is captured between the start time point and the end time point, and the target audio frame is sent to the speech recognition module.

[0011] In one possible design, determining the start time of the user's speech by traversing the audio frames sequentially based on their sound energy and speech activity detection attributes includes:

[0012] The audio frames are traversed in chronological order. If any audio frame is found to meet the preset audio start condition, the corresponding audio frame is determined as a candidate start frame.

[0013] The preset audio start condition is that the sound energy of any audio frame is greater than a preset sound energy threshold, and the speech activity detection attribute of the audio frame is speech;

[0014] The audio frames following the candidate start frame are traversed. If N consecutive audio frames satisfy the preset audio start condition, the time corresponding to the candidate start frame is determined as the start time point of the user's speech; where N is a positive integer greater than 1.

[0015] In one possible design, determining the termination time of the user's speech based on the start time point and the speech activity detection attributes of each of the audio frames includes:

[0016] Starting from the audio frame corresponding to the starting time point, each audio frame is traversed in chronological order to monitor the speech activity detection attributes of each audio frame.

[0017] If the speech activity detection attribute of M consecutive audio frames is not speech, the traversal of each audio frame is stopped, and the time corresponding to the last traversed audio frame is determined as the termination time point of the user's speech; where M is a positive integer greater than 1.

[0018] In one possible design, determining the sound energy of each of the audio frames includes:

[0019] Extract the values ​​of the audio sampling points within each audio frame;

[0020] The values ​​of each audio sampling point are calculated using a preset energy calculation method to obtain the sound energy corresponding to each audio frame; wherein, the preset energy calculation method is the sum of squares method or the sum of absolute values ​​method.

[0021] In one possible design, the speech activity detection module detects speech activity detection attributes for each of the audio frames, including:

[0022] Extract auxiliary features from each of the audio frames; wherein the auxiliary features include short-time zero-crossing rate and fundamental frequency;

[0023] If the sound energy of the audio frame is greater than a preset sound energy threshold, the short-time zero-crossing rate is within a first preset range, and the fundamental frequency is within a second preset range, then the speech activity detection attribute of the audio frame is determined to be speech.

[0024] If the sound energy of the audio frame is less than or equal to a preset sound energy threshold, or the short-time zero-crossing rate is not within the first preset range, or the fundamental frequency is not within the second preset range, then the speech activity detection attribute of the audio frame is determined to be non-speech.

[0025] One possible design also includes:

[0026] The target audio frames between the start time point and the corresponding end time point are subjected to noise reduction preprocessing; wherein, the noise reduction preprocessing includes removing environmental noise in the target audio frames using spectral subtraction or Wiener filtering algorithms;

[0027] Each of the noise-reduced target audio frames is converted into a target format that matches the speech recognition module.

[0028] Secondly, this application provides an audio truncation device for short speech recognition, comprising:

[0029] The acquisition module is used to acquire the real-time audio data stream from the microphone and divide the audio data stream into frames to obtain multiple audio frames.

[0030] The voice attribute determination module is used to determine the sound energy of each audio frame, send the sound energy of each audio frame to the voice activity detection module for detection, and receive the voice activity detection attributes of each audio frame returned by the voice activity detection module.

[0031] The start time determination module is used to determine the start time point of the user's speech by traversing each audio frame in chronological order based on the sound energy and speech activity detection attributes of each audio frame.

[0032] The termination time determination module is used to determine the termination time of the user's voice based on the start time point and the voice activity detection attributes of each audio frame.

[0033] An audio capture module is used to capture at least one target audio frame between the start time point and the end time point, and send the target audio frame to the speech recognition module.

[0034] Thirdly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0035] The memory stores computer-executed instructions;

[0036] The processor executes computer execution instructions stored in the memory to implement the method as described in any of the first aspects.

[0037] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any of the first aspects.

[0038] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the method described in any of the first aspects.

[0039] The audio truncation method, apparatus, device, medium, and product for short speech recognition provided in this application acquire real-time audio data streams and perform frame segmentation. First, the sound energy of each audio frame is determined and sent to a speech activity detection module to obtain speech activity detection attributes. Then, based on the sound energy and speech activity detection attributes of each audio frame, the start time point of the user's speech is determined in chronological order. Next, the end time point is determined based on the start time point and speech activity detection attributes. Finally, the target audio frame between the start and end time points is truncated and sent to the speech recognition module. This method solves the problem of poor environmental adaptability of fixed energy threshold schemes and avoids the defects of speech activity detection being easily affected by sudden noise interference, leading to inaccurate truncation boundaries and difficulty in guaranteeing speech integrity, thereby improving the accuracy and reliability of short speech audio truncation. Attached Figure Description

[0040] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0041] Figure 1 An application scenario diagram corresponding to the audio truncation method in short speech recognition provided in an embodiment of this application;

[0042] Figure 2 A flowchart illustrating an audio truncation method in short speech recognition provided in an embodiment of this application;

[0043] Figure 3 A flowchart illustrating an audio truncation method in short speech recognition provided in another embodiment of this application;

[0044] Figure 4 A flowchart illustrating an audio truncation method in short speech recognition provided in another embodiment of this application;

[0045] Figure 5 This is a schematic diagram of the structure of an audio truncation device in short speech recognition provided in an embodiment of this application;

[0046] Figure 6 This is a structural example diagram of an electronic device provided in an embodiment of this application.

[0047] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0048] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0049] The collection, storage, use, processing, transmission, provision, and disclosure of financial data or user data involved in the technical solution of this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0050] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0051] To clearly understand the technical solution of this application, the solutions of the prior art will be described in detail first.

[0052] With the popularization of voice interaction technology, short speech recognition has become one of the core functions of smart devices. Short speech recognition refers to converting a brief (usually no more than 60 seconds) segment of speech into text, replacing traditional input scenarios, and is suitable for voice search, human-computer interaction, and other scenarios. The prerequisite for short speech recognition is to accurately extract a segment containing valid user speech from the real-time audio data stream captured by the microphone. If the extraction range is too large (containing a lot of background noise), it will increase the processing load of the speech recognition module, leading to increased recognition latency; if the extraction range is too small (missing some speech content), it will destroy the integrity of the speech semantics, leading to recognition errors; if background noise (such as coughing or environmental noise) is mistakenly identified as the start or end of the speech, it will directly cause extraction failure.

[0053] In existing technologies, audio extraction schemes mainly employ two methods to extract short audio segments. The first method involves setting a fixed energy threshold, classifying audio frames with energy above the threshold as speech frames and those below as non-speech frames. However, in noisy environments, the energy of background noise may approach or even exceed that of valid speech, leading to misclassification. In quiet environments, the energy of a user speaking softly may be below the fixed energy threshold, resulting in missed detections. The second method uses a VAD (Video Audio Analyzer) module to extract spectral and pitch features from the audio to distinguish between speech and non-speech. However, VAD is susceptible to sudden noise interference, potentially misinterpreting short bursts of high-energy noise as the start of speech or short pauses in the middle of speech as the end of speech, resulting in incomplete extracted audio segments.

[0054] In summary, existing short speech audio extraction schemes suffer from poor environmental adaptability, inaccurate extraction boundaries, and difficulty in ensuring speech integrity.

[0055] Therefore, when facing the technical problems of the existing technology, considering that real-time audio data streams need to be broken down before segmented analysis, and that frame segmentation is a routine basic operation for audio processing, the audio data stream is first segmented into frames. Then, the sound energy and VAD attribute of each audio frame must be obtained to provide a basis for subsequent judgments. When determining the start time point of speech, relying solely on sound energy or VAD attributes can easily lead to problems. Therefore, it is necessary to traverse each audio frame in chronological order, while simultaneously referring to sound energy and VAD attributes, to reduce the possibility of mistaking noise for the start of speech or missing soft speech. When determining the end time point, given a clear start time point, combining it with VAD attributes can avoid misjudging pauses in the middle of speech as termination, ensuring the integrity of the speech. Finally, multiple audio frames between the start and end time points are extracted and sent to the speech recognition module.

[0056] Figure 1 This is an application scenario diagram corresponding to the audio truncation method in short speech recognition provided in an embodiment of this application, such as... Figure 1As shown, the application scenario provided in this embodiment includes: user equipment 11, audio interception device 12, voice activity detection module 13, and voice recognition module 14. User equipment 11, voice activity detection module 13, and voice recognition module 14 are all communicatively connected to audio interception device 12.

[0057] Specifically, when a user engages in voice communication through user device 11, audio interception device 12 acquires the real-time audio data stream from the microphone of user device 11, segments the audio data stream into frames to obtain multiple audio frames, determines the sound energy of each audio frame, sends the sound energy of each audio frame to the voice activity detection module 13 for detection, and receives the voice activity detection attributes of each audio frame returned by the voice activity detection module 13; based on the sound energy and voice activity detection attributes of each audio frame, it traverses each audio frame in chronological order to determine the start time point of the user's voice; based on the start time point and the voice activity detection attributes of each audio frame, it determines the end time point of the user's voice; finally, it intercepts at least one target audio frame between the start time point and the end time point and sends the target audio frame to the voice recognition module 14.

[0058] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0059] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0060] Figure 2 This is a flowchart illustrating an audio truncation method in short speech recognition provided in an embodiment of this application, as shown below. Figure 2As shown, the execution subject of this embodiment is an audio extraction device in short speech recognition. This audio extraction device can be implemented through a computer program, or through a medium storing the relevant computer program, such as a USB flash drive and / or optical disc; alternatively, it can be implemented through a physical device integrating or installing the relevant computer program, such as a chip or electronic device. The electronic device can be a computer or a server, etc. The audio extraction method in short speech recognition provided in this embodiment includes the following steps:

[0061] S201. Obtain the real-time audio data stream from the microphone and divide the audio data stream into frames to obtain multiple audio frames.

[0062] Optionally, real-time audio signals can be acquired through the microphone (or microphone array) of an electronic device. The analog audio signal can be converted into a digital audio data stream based on the microphone driver. A common speech processing format can be used, such as a PCM (Pulse Code Modulation) format with a sampling rate of 16kHz, a bit depth of 16bit, and mono. This speech processing format can reduce the data transmission and processing load while ensuring the preservation of speech details, and is suitable for short speech real-time processing scenarios.

[0063] Optionally, the digital audio data stream can be divided into frames at fixed time intervals to obtain multiple consecutive audio frames. It should be noted that the framing parameters need to balance temporal resolution and intra-frame feature stability, with a preferred frame length of 20-30ms and a frame shift of 10-15ms. The purpose of using a frame shift design is to avoid overlapping areas between frames, ensuring the continuity of subsequent feature extraction and preventing the loss of speech features due to frame boundary breaks.

[0064] S202. Determine the sound energy of each audio frame, send the sound energy of each audio frame to the speech activity detection module for detection, and receive the speech activity detection attributes of each audio frame returned by the speech activity detection module.

[0065] Alternatively, for each audio frame, the sound energy E can be calculated using the following formula:

[0066]

[0067] Where K is the number of sampling points in a single audio frame, x i Let be the amplitude of the i-th sampling point.

[0068] It should be noted that in short speech scenarios, choosing absolute value and computational energy can effectively distinguish speech from noise (the energy of a speech frame is usually 3-5 times that of a background noise frame), and the computational complexity is lower, which is suitable for real-time processing requirements.

[0069] Optionally, the raw data or preprocessed features of each audio frame are sent to the speech activity detection module. The speech activity detection module judges each audio frame using a preset speech / non-speech classification model (such as a classifier based on Gaussian Mixture Model GMM or deep learning model CNN-LSTM) and returns the speech activity detection attributes of the audio frame.

[0070] The speech activity detection attributes can include frame type label and confidence level. The frame type label is "speech frame" or "non-speech frame", and the confidence level is a value between 0 and 1. The higher the confidence level, the greater the probability that the audio frame is a speech frame.

[0071] It should be noted that by combining sound energy and speech activity detection attributes, and comprehensively considering the energy features and speech activity features of audio frames, this method can more comprehensively and accurately determine whether an audio frame contains valid speech compared to relying solely on a fixed energy threshold or extracting only some features through the VAD module.

[0072] S203. Based on the sound energy and speech activity detection attributes of each audio frame, traverse each audio frame in chronological order to determine the start time of the user's speech.

[0073] Optionally, all audio frames are traversed in the order of their generation time (from early to late), and the speech start frame is selected based on the sound energy and speech activity detection attributes of each audio frame to avoid misjudgment due to abnormal features of a single frame (such as sudden noise frames).

[0074] Optionally, the average sound energy E of the first 10-20 frames (audio frames before speech is detected, which are considered background noise frames by default) is first obtained. avg The speech energy threshold can be set to E. avg 1.5 to 2 times that of the previous setting. With this setting, the speech energy threshold can adaptively adjust according to the noise level of the current environment; in noisy environments, E... avg As the threshold increases, the speech energy threshold also increases synchronously to avoid misinterpreting background noise as speech; in quiet environments, E... avg The threshold for voice energy is lowered accordingly to avoid missing soft-spoken speech.

[0075] Optionally, when 3-5 consecutive audio frames simultaneously meet the starting condition, the first audio frame in the consecutive frames can be determined as the speech start frame, and its corresponding time point is the start time point of the user's speech. Optionally, the starting condition can be that the sound energy of the audio frame is greater than the speech energy threshold, or that the frame type in the speech activity detection attribute of the audio frame is marked as "speech frame" and the confidence level is >0.8.

[0076] It should be noted that selecting continuous multi-frame verification is to filter out single bursts of noise. This type of noise typically lasts only 1-2 frames, which cannot meet the requirement of 4 consecutive frames, thus avoiding misjudging the start time point. Furthermore, continuous multi-frame verification can also effectively avoid misjudging the start point due to high background noise energy in noisy environments, and avoid missing the start point due to low energy from the user speaking softly in quiet environments.

[0077] S204. Based on the start time point and the speech activity detection attributes of each audio frame, determine the end time point of the user's speech.

[0078] Optionally, starting from the audio frames after the determined speech start time, continue to traverse in chronological order. The purpose is to distinguish between short pauses in the middle of the speech (such as a user taking a breath while speaking, which usually lasts 0.1-0.3 seconds) and the actual end of the speech (pause time > 0.5 seconds and no subsequent speech), so as to ensure the integrity of the speech segment.

[0079] Optionally, when 5-8 consecutive audio frames simultaneously meet the termination condition, the first audio frame in the group of consecutive frames is determined as the speech termination frame, and its corresponding time point is the termination time point of the user's speech. Optionally, the termination condition may be that the sound energy of the audio frame is less than the speech energy threshold, or that the frame type in the speech activity detection attribute of the audio frame is marked as a non-speech frame and the confidence level is <0.3 (low confidence level ensures the reliability of non-speech judgment).

[0080] Optionally, the speech energy threshold for the termination condition is lower than the speech energy threshold for the start condition. This is because the energy at the end of a speech segment typically weakens gradually, and using the same speech energy threshold as the start condition could easily lead to missed detections. Furthermore, performing verification across multiple consecutive frames can filter out short pauses, preventing misinterpretation of user breathing as speech termination and ensuring complete speech capture.

[0081] It should be noted that by combining the start time point and speech activity detection attributes, it is possible to more accurately determine when the speech ends, avoiding misjudging short pauses in the middle of the speech as termination, or incorrectly judging the termination time point due to sudden noise interference, thereby ensuring the integrity of the captured speech segment.

[0082] S205. Capture at least one target audio frame between the start time point and the end time point, and send the target audio frame to the speech recognition module.

[0083] Among them, target frame extraction is to extract all audio frames between the start time point and the end time point.

[0084] Optionally, the target audio frame can be encapsulated in a format supported by the speech recognition module (such as PCM or WAV) and sent to the speech recognition module through a data stream interface.

[0085] It should be noted that by extracting the target audio frame, the segment containing the effective user's voice can be accurately extracted, reducing background noise interference. Compared with sending the full real-time audio stream, sending only the target audio frame can reduce the amount of data (e.g., only 1.5 seconds of effective segment needs to be extracted from 3 seconds of voice), which can reduce the processing latency of the speech recognition module and adapt to the needs of real-time voice interaction.

[0086] The audio truncation method for short speech recognition provided in this application acquires real-time audio data streams and performs frame-by-frame processing. First, it determines the sound energy of each audio frame and sends it to a speech activity detection module to obtain speech activity detection attributes. Then, based on the sound energy and speech activity detection attributes of each audio frame, it determines the user's speech start time in chronological order. Next, it determines the end time point based on the start time point and speech activity detection attributes. Finally, it truncates the target audio frame between the start and end time points and sends it to the speech recognition module. This method solves the problem of poor environmental adaptability of fixed energy threshold schemes and avoids the defects of speech activity detection being easily affected by sudden noise interference, leading to inaccurate truncation boundaries and difficulty in guaranteeing speech integrity, thereby improving the accuracy and reliability of short speech audio truncation.

[0087] It should be noted that the start and end times of the speech are determined by verifying multiple consecutive frames. By taking advantage of the characteristic that speech is a continuous semantic segment, single and instantaneous noise interference is filtered out, and short pauses and true terminations are also distinguished to ensure that the extracted speech segments are free of redundant noise and semantic loss.

[0088] It should also be noted that the audio framing, sound energy calculation, and VAD interaction all adopt mature industry technologies, requiring no additional hardware costs. They can be directly integrated into the voice processing modules of existing smart devices, exhibiting strong compatibility and facilitating industrialization.

[0089] Figure 3 A flowchart illustrating an audio truncation method in short speech recognition provided in another embodiment of this application is shown below. Figure 3 As shown in the illustration, as an optional implementation, based on any of the above embodiments, the audio frames are traversed sequentially according to their sound energy and speech activity detection attributes to determine the start time point of the user's speech, including the following steps:

[0090] S301. Traverse each audio frame in chronological order. If any audio frame is found to meet the preset audio start condition, determine the corresponding audio frame as a candidate start frame. The preset audio start condition is that the sound energy of any audio frame is greater than the preset sound energy threshold, and the speech activity detection attribute of the audio frame is speech.

[0091] Specifically, all audio frames are traversed in the order of their generation time (from early to late), and each audio frame is checked to see if it meets the preset audio start conditions.

[0092] The preset audio start condition includes the sound energy of any audio frame being greater than a preset sound energy threshold. Optionally, the preset sound energy threshold can be a dynamic energy threshold, i.e., 1.5 to 2 times the average energy of the background noise in the previous 10 to 20 frames.

[0093] The preset audio start conditions also include that the speech activity detection attribute of the audio frame is speech, that is, the frame type is marked as "speech".

[0094] Optionally, if any audio frame simultaneously meets the above-mentioned preset audio start conditions, then the audio frame is marked as a candidate start frame and its corresponding time point is recorded.

[0095] S302. Traverse each audio frame after the candidate start frame. If N audio frames are detected consecutively and meet the preset audio start conditions, determine the time corresponding to the candidate start frame as the start time point of the user's speech; where N is a positive integer greater than 1.

[0096] Optionally, after determining the candidate starting frame, the audio frames following the candidate frame are traversed in chronological order to determine whether the continuous verification condition is met.

[0097] Specifically, starting from the next frame after the candidate start frame, N audio frames are continuously detected. If all N consecutive audio frames meet the preset audio start conditions, the candidate start frame is confirmed as the start frame of valid speech, and the corresponding time is the start time point of the user's speech.

[0098] Where N is a positive integer greater than 1, preferably N=2-5, and the specific value is adjusted according to the noise fluctuation level of the scene. For example, N=2-3 is used in the noise stable scene, and N=4-5 is used in the noise fluctuating scene.

[0099] Optionally, if any audio frame does not meet the preset audio start condition (e.g., VAD is marked as non-speech) during continuous detection, the marker of the candidate start frame is canceled, and subsequent audio frames are traversed to re-select new candidate start frames.

[0100] The audio truncation method for short speech recognition provided in this application first traverses audio frames in chronological order, using the condition that the sound energy is greater than a preset threshold and the speech activity detection attribute is a preset audio start condition to determine candidate start frames. Then, after traversing the candidate start frames, the starting time point is determined only when multiple audio frames meeting the condition are detected consecutively. This effectively avoids misjudgment of the starting point due to accidental energy fluctuations or brief interference, improves the accuracy of determining the user's speech start time point, and thus enhances the precision of short speech audio truncation.

[0101] Figure 4 A flowchart illustrating an audio truncation method in short speech recognition provided in another embodiment of this application is shown below. Figure 4 As shown in the illustration, as an optional implementation, based on any of the above embodiments, the termination time of the user's speech is determined based on the start time point and the speech activity detection attributes of each audio frame, including the following steps:

[0102] S401. Starting from the audio frame corresponding to the initial time point, traverse each audio frame in chronological order and monitor the speech activity detection attributes of each audio frame.

[0103] Optionally, starting from the user's voice start frame determined in S203, the traversal is performed in the order of audio frame generation time, without backtracking to non-voice frames before the start point. Furthermore, during the traversal, only the voice activity detection attribute is focused on, and the sound energy of the audio frames is not considered, in order to avoid interference from high-energy non-voice frames caused by wind noise, equipment vibration, etc., on the termination boundary.

[0104] S402. If the speech activity detection attribute of M consecutive audio frames is not speech, stop traversing each audio frame and determine the time corresponding to the last traversed audio frame as the termination time point of the user's speech; where M is a positive integer greater than 1.

[0105] Optionally, for each audio frame traversed, a valid non-speech frame can be determined if the following conditions are met: the frame type of the speech activity detection attribute is marked as non-speech, and the non-speech confidence is greater than or equal to a preset confidence level, which can be 0.7.

[0106] Where M is a positive integer greater than 1, preferably M=3-6, and dynamically adapted according to the normal pause duration in short speech scenarios. For example, in short command scenarios, M can be 3-4 to balance response speed and pause filtering. In short command scenarios such as "turn on the light" or "check the weather", the normal breathing pause is ≤0.3 seconds, corresponding to ≤3 frames with a frame shift of 10ms. In long command segment scenarios, M can be 5-6 to avoid misjudging pauses as termination. In long command segment scenarios such as "remind me of the meeting at 9 am tomorrow", the normal thinking pause is ≤0.5 seconds, corresponding to ≤5 frames.

[0107] Optionally, when M valid non-speech frames are detected consecutively, the traversal is stopped, and the time point corresponding to the last audio frame among the M non-speech frames is determined as the termination time point of the user's speech.

[0108] Optionally, if the audio data stream reaches its end (e.g., no subsequent audio input after the user stops speaking) and no consecutive M valid non-speech frames are detected, the time point corresponding to the last audio frame at the end of the audio data stream can be used as the termination time point of the user's speech, ensuring that the semantics at the end of the speech are not lost.

[0109] The audio truncation method for short speech recognition provided in this application starts by traversing and monitoring the speech activity detection attributes of audio frames in chronological order, beginning with the audio frame corresponding to the start time point. The termination time point is determined only when multiple audio frames are continuously detected as having non-speech attributes. This effectively avoids misinterpreting short pauses in speech as the end of speech, prevents erroneous speech truncation due to interference from individual non-speech frames, ensures the integrity of the truncated speech segment, and improves the accuracy of audio truncation in short speech recognition.

[0110] As an optional implementation, based on any of the above embodiments, the sound energy of each audio frame is determined, including:

[0111] First, extract the values ​​of the audio sampling points within each audio frame.

[0112] Optionally, for each audio frame obtained after S201 framing, the values ​​of all audio sampling points within each audio frame are extracted. Optionally, the numerical format of the sampling points can be PCM audio format (16kHz sampling rate, 16-bit depth), and the value range of a single sampling point is -32768 to 32767, representing the amplitude of the audio signal at that moment; a single audio frame contains T sampling points (e.g., a frame length of 25ms corresponds to T=400, a frame length of 30ms corresponds to T=480), and the values ​​of all T sampling points within the audio frame must be extracted completely to avoid energy calculation errors due to missed sampling points.

[0113] Secondly, a preset energy calculation method is used to calculate the values ​​of each audio sampling point to obtain the sound energy corresponding to each audio frame; the preset energy calculation method is either the sum of squares method or the sum of absolute values ​​method.

[0114] For the extracted T sampling point values, the sound energy of the audio frame is calculated using a preset energy calculation method. The preset energy calculation method is either the sum of squares method or the sum of absolute values ​​method. The two methods can be dynamically selected according to the device's computing power and the noise characteristics of the scene.

[0115] Among them, the sum of squares method is based on the physical definition of acoustic energy, which is proportional to the square of the signal amplitude. By summing the squares of the amplitudes of all sampling points, the total energy of the audio frame can be obtained.

[0116]

[0117] in, Let be the value of the u-th sampling point, where U is the number of sampling points in a single audio frame. This represents the sound energy of the audio frame.

[0118] It should be noted that the sum of squares method is more sensitive to high-energy sampling points (such as burst noise) and can more accurately distinguish between strong and weak speech frames.

[0119] Optionally, in order to reduce the computing power consumption of the device, the sum of the absolute values ​​of the amplitudes of all sampling points is used to approximate the calculation of the sum of squares of physical energy. In short speech scenarios, the correlation between the sum of absolute values ​​and the sum of squares is above 0.92, which can meet the threshold judgment requirements.

[0120] The audio truncation method for short speech recognition provided in this application first extracts the audio sampling point values ​​within an audio frame, and then uses the sum of squares method or the sum of absolute values ​​method to obtain the sound energy. This allows for precise quantification of the energy of each audio frame, providing a reliable basis for accurately determining the start and end of speech, effectively avoiding misjudgments caused by inaccurate energy calculation, and improving the accuracy of speech boundary determination during short speech audio truncation.

[0121] As an optional implementation, based on any of the above embodiments, the speech activity detection module detects the speech activity detection attributes of each audio frame, including the following steps:

[0122] First, auxiliary features are extracted from each audio frame; these auxiliary features include short-time zero-crossing rate and fundamental frequency.

[0123] The short-time zero-crossing rate (ZCR) is the number of times an audio signal crosses the zero level (from positive to negative or from negative to positive) per unit time. It reflects the frequency distribution characteristics of the signal. Unvoiced sounds (such as "s" and "sh") have a higher ZCR, voiced sounds (such as "a" and "o") have a lower ZCR, and non-human noise (such as wind noise and white noise) usually has a higher or irregular ZCR.

[0124] Optionally, the audio frame is pre-emphasized using a first-order high-pass filter with coefficient α=0.97 to compensate for high-frequency signal attenuation and avoid low-frequency noise interference with zero-crossing detection. Then, all sampling points within the audio frame are iterated over; if the signs of any adjacent sampling points are opposite, a zero-crossing is counted once. The short-time zero-crossing rate is obtained by dividing the number of zero-crossings by the duration of the audio frame.

[0125] The fundamental frequency (F0) is the basic frequency of human vocal cord vibration, reflecting the pitch characteristics of speech. The F0 of male speech is usually 80~200Hz, female speech is 160~350Hz, and children's speech is 200~400Hz. Non-human noise (such as car horns, keyboard typing) has no stable F0 or exceeds this range.

[0126] Optionally, the audio frame is windowed to reduce spectral leakage at the frame edge, and then passed through a 200~3400Hz bandpass filter; an efficient autocorrelation algorithm is used to calculate the autocorrelation function of the signal within the audio frame, find the delay step corresponding to the first peak of the autocorrelation function, and determine the fundamental frequency based on the audio sampling rate and the delay step corresponding to the autocorrelation peak.

[0127] Specifically, if the sound energy of an audio frame is greater than a preset sound energy threshold, the short-time zero-crossing rate is within a first preset range, and the fundamental frequency is within a second preset range, the speech activity detection attribute of the audio frame is determined to be speech.

[0128] The first preset range is a range pre-set in conjunction with the short speech scenario. Optionally, the first preset range is set to 300~1500 times / second. Below 300 times / second may be low-frequency mechanical noise, and above 1500 times / second may be high-frequency wind noise or white noise.

[0129] The second preset range is a pre-set range of speech values ​​covering all groups of people, including men, women, the elderly, and children. Optionally, the second preset range is set to 80~400Hz, where below 80Hz is low-frequency mechanical noise and above 400Hz is high-frequency non-human noise.

[0130] It should be noted that an audio frame is determined to have speech activity detection attribute only if it meets all three conditions simultaneously, thus ensuring the accuracy of the judgment result.

[0131] Specifically, if the sound energy of an audio frame is less than or equal to a preset sound energy threshold, or the short-time zero-crossing rate is not within a first preset range, or the fundamental frequency is not within a second preset range, then the speech activity detection attribute of the audio frame is determined to be non-speech.

[0132] It should be noted that if an audio frame does not meet any of the three conditions, the audio frame is determined to be non-speech in order to filter out low-energy noise, high ZCR wind noise, and non-human high-frequency noise, thus ensuring the accuracy of the judgment results.

[0133] The audio truncation method for short speech recognition provided in this application extracts auxiliary features such as short-time zero-crossing rate and fundamental frequency, and combines this with a comparison of sound energy with preset thresholds and ranges to determine the speech activity detection attributes of audio frames. Thus, it comprehensively utilizes multiple audio features, avoiding the limitations of relying solely on a single feature, effectively distinguishing between speech and non-speech, reducing misjudgments caused by sudden noise or environmental changes, improving the accuracy of speech activity detection, and consequently enhancing the precision of short speech audio truncation.

[0134] As an optional implementation, based on any of the above embodiments, the method further includes the following steps:

[0135] Noise reduction preprocessing is performed on each target audio frame between the start time point and its corresponding end time point; this preprocessing includes removing environmental noise from the target audio frames using spectral subtraction or Wiener filtering algorithms. The denoised target audio frames are then converted into a target format compatible with the speech recognition module.

[0136] Specifically, for all target audio frames captured between the start and end time points in S205, noise reduction preprocessing is first performed to remove residual environmental noise and retain the clean speech signal. Optionally, in this embodiment, two preset noise reduction algorithms are provided for dynamic selection, including spectral subtraction and Wiener filtering.

[0137] Spectral subtraction, based on the separability of speech signals and stationary noise in the frequency domain, estimates the background noise spectrum and subtracts the noise spectrum from the noisy speech spectrum to reconstruct a clean speech signal. Spectral subtraction is suitable for stationary noise scenarios (such as home air conditioner noise or office fan noise). The implementation steps of spectral subtraction include noise spectrum estimation, noisy speech spectrum calculation, spectral subtraction reconstruction, and post-processing. The advantages of spectral subtraction are that it only requires basic Fast Fourier Transform and Inverse Fast Fourier Transform, making it adaptable to low-computing-power devices (such as smartwatches, with single-frame processing time ≤1ms), and it has a high reduction rate for stationary noise.

[0138] Wiener filtering, based on the minimum mean square error criterion, designs an adaptive filter by estimating the power spectral density ratio of speech to noise. This suppresses noise while preserving speech details to the greatest extent possible. Wiener filtering is suitable for non-stationary noise scenarios (such as time-varying noises like footsteps on outdoor streets and vehicle horns). The implementation steps of Wiener filtering include initial signal-to-noise ratio estimation, Wiener filter design, and speech reconstruction. The advantages of Wiener filtering lie in its low speech distortion and strong adaptability to non-stationary noise, such as outdoor varying noise.

[0139] Optionally, spectral subtraction or Wiener filtering can be adaptively selected for noise reduction based on the type of scene noise corresponding to the target audio frame.

[0140] Optionally, the target audio frame after noise reduction needs to be converted to a target format compatible with the speech recognition module (ASR) to ensure that the data can be directly parsed by ASR.

[0141] Optionally, the target audio frame can be converted to PCM, WAV, or MP3 format.

[0142] For PCM format, the sampling rate (16kHz / 8kHz), bit depth (16bit / 8bit), and number of channels (mono) must be consistent with ASR. A WAV format header (containing metadata such as sampling rate, bit depth, number of channels, and data length) can be added before the PCM data.

[0143] Optionally, if ASR only supports 8kHz, the denoised frame sampled at 16kHz can be converted to 8kHz using linear interpolation; if ASR supports 8bit, the 16bit sampled value is shifted right by 8 bits; if ASR only supports mono, the left channel data is taken from the denoised frame of the stereo channel.

[0144] Optionally, the short-time energy and fundamental frequency of the converted audio are extracted and compared with the corresponding features of the original audio frame after noise reduction. The error must be ≤10% to ensure that the speech features are not destroyed during the format conversion.

[0145] The audio truncation method for short speech recognition provided in this application first performs noise reduction preprocessing on the truncated target audio frame using spectral subtraction or Wiener filtering algorithms, effectively removing environmental noise, improving audio quality, and reducing noise interference with subsequent speech recognition. Then, the noise-reduced audio frame is converted into a target format that matches the speech recognition module, ensuring that the audio data can be accurately recognized. Therefore, by performing noise reduction processing and format conversion on the target audio frame to adapt to the speech recognition module, the usability and recognition accuracy of audio data in short speech recognition are improved.

[0146] Figure 5 This is a schematic diagram of the structure of an audio truncation device in short speech recognition provided in an embodiment of this application, as shown below. Figure 5 As shown, the audio truncation device for short speech recognition provided in this embodiment is located in an electronic device. The audio truncation device 50 for short speech recognition provided in this embodiment includes: an acquisition module 51, a speech attribute determination module 52, a start time determination module 53, an end time determination module 54, and an audio truncation module 55.

[0147] Specifically, the acquisition module 51 is used to acquire the real-time audio data stream from the microphone and divide the audio data stream into frames to obtain multiple audio frames; the voice attribute determination module 52 is used to determine the sound energy of each audio frame, send the sound energy of each audio frame to the voice activity detection module for detection, and receive the voice activity detection attributes of each audio frame returned by the voice activity detection module; the start time determination module 53 is used to traverse each audio frame in chronological order based on the sound energy and voice activity detection attributes of each audio frame to determine the start time point of the user's voice; the end time determination module 54 is used to determine the end time point of the user's voice based on the start time point and the voice activity detection attributes of each audio frame; and the audio truncation module 55 is used to truncate at least one target audio frame between the start time point and the end time point and send the target audio frame to the speech recognition module.

[0148] Optionally, the start time determination module 53, when determining the start time point of the user's speech by traversing each audio frame in chronological order based on the sound energy and speech activity detection attributes of each audio frame, specifically performs the following: traversing each audio frame in chronological order, and determining the corresponding audio frame as a candidate start frame when any audio frame is detected to meet a preset audio start condition; wherein, the preset audio start condition is that the sound energy of any audio frame is greater than a preset sound energy threshold, and the speech activity detection attribute of the audio frame is speech; traversing each audio frame after the candidate start frame, and determining the time corresponding to the candidate start frame as the start time point of the user's speech when N consecutive audio frames are detected to meet the preset audio start condition; wherein, N is a positive integer greater than 1.

[0149] Optionally, the termination time determination module 54, when determining the termination time of the user's speech based on the start time point and the speech activity detection attributes of each audio frame, specifically performs the following: starting from the audio frame corresponding to the start time point, traversing each audio frame in chronological order and monitoring the speech activity detection attributes of each audio frame; when the speech activity detection attributes of M consecutive audio frames are all non-speech, stopping the traversal of each audio frame, and determining the time corresponding to the last traversed audio frame as the termination time of the user's speech; where M is a positive integer greater than 1.

[0150] Optionally, the voice attribute determination module 52, when determining the sound energy of each audio frame, is specifically used to: extract the values ​​of the audio sampling points in each audio frame; calculate the values ​​of each audio sampling point using a preset energy calculation method to obtain the sound energy corresponding to each audio frame; wherein, the preset energy calculation method is the sum of squares method or the sum of absolute values ​​method.

[0151] Optionally, the speech activity detection module detects the speech activity detection attributes of each audio frame, including: extracting auxiliary features of each audio frame; wherein the auxiliary features include short-time zero-crossing rate and fundamental frequency; if the sound energy of the audio frame is greater than a preset sound energy threshold, the short-time zero-crossing rate is within a first preset range, and the fundamental frequency is within a second preset range, the speech activity detection attribute of the audio frame is determined to be speech; if the sound energy of the audio frame is less than or equal to the preset sound energy threshold, or the short-time zero-crossing rate is not within the first preset range, or the fundamental frequency is not within the second preset range, the speech activity detection attribute of the audio frame is determined to be non-speech.

[0152] Optionally, the audio extraction device in short speech recognition also includes a noise reduction module and a format conversion module.

[0153] Optionally, the noise reduction module is used to: perform noise reduction preprocessing on each target audio frame between the start time point and its corresponding end time point; wherein, the noise reduction preprocessing includes removing environmental noise in the target audio frame using spectral subtraction or Wiener filtering algorithms; the format conversion module is used to: convert each denoised target audio frame into a target format that matches the speech recognition module.

[0154] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, as shown below. Figure 6 As shown, the electronic device 60 provided in this embodiment includes a processor 61 and a memory 62 communicatively connected to the processor 61.

[0155] The memory 62 stores computer execution instructions; the processor 61 executes the computer execution instructions stored in the memory 62 to implement the method provided in any of the above embodiments.

[0156] The program may include program code, which includes computer-executable instructions. Memory 62 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device.

[0157] In this embodiment, the memory 62 and the processor 61 are connected via a bus. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6The bus is represented by a single straight line, but this does not mean that there is only one bus or one type of bus.

[0158] This application also provides a computer-readable storage medium, including computer-executable instructions stored in the computer-readable storage medium, which, when executed by a processor, are used to implement the method provided in any of the above embodiments.

[0159] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the method provided in any of the above embodiments.

[0160] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement the solution of this embodiment according to actual needs.

[0161] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The unit composed of the above modules can be implemented in hardware or in the form of hardware plus software functional units.

[0162] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application.

[0163] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0164] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk or optical disc, etc.

[0165] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0166] The aforementioned storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0167] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. The processor and storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components in an electronic control unit or main control device.

[0168] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0169] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for audio truncation in short speech recognition, characterized in that, include: Acquire the real-time audio data stream from the microphone and divide the audio data stream into frames to obtain multiple audio frames; The sound energy of each audio frame is determined, the sound energy of each audio frame is sent to the speech activity detection module for detection, and the speech activity detection attributes of each audio frame returned by the speech activity detection module are received. Based on the sound energy and speech activity detection attributes of each audio frame, the audio frames are traversed in chronological order to determine the start time of the user's speech. Based on the start time point and the speech activity detection attributes of each audio frame, the end time point of the user's speech is determined. At least one target audio frame is captured between the start time point and the end time point, and the target audio frame is sent to the speech recognition module.

2. The method according to claim 1, characterized in that, The step of determining the start time point of the user's speech by traversing each audio frame in chronological order based on the sound energy and speech activity detection attributes of each audio frame includes: The audio frames are traversed in chronological order. If any audio frame is found to meet the preset audio start condition, the corresponding audio frame is determined as a candidate start frame. The preset audio start condition is that the sound energy of any audio frame is greater than a preset sound energy threshold, and the speech activity detection attribute of the audio frame is speech; The audio frames following the candidate start frame are traversed. If N consecutive audio frames satisfy the preset audio start condition, the time corresponding to the candidate start frame is determined as the start time point of the user's speech; where N is a positive integer greater than 1.

3. The method according to claim 1, characterized in that, Determining the termination time of the user's speech based on the start time point and the speech activity detection attributes of each audio frame includes: Starting from the audio frame corresponding to the starting time point, each audio frame is traversed in chronological order to monitor the speech activity detection attributes of each audio frame. If the speech activity detection attribute of M consecutive audio frames is not speech, the traversal of each audio frame is stopped, and the time corresponding to the last traversed audio frame is determined as the termination time point of the user's speech; where M is a positive integer greater than 1.

4. The method according to claim 1, characterized in that, Determining the sound energy of each audio frame includes: Extract the values ​​of the audio sampling points within each audio frame; The values ​​of each audio sampling point are calculated using a preset energy calculation method to obtain the sound energy corresponding to each audio frame; wherein, the preset energy calculation method is the sum of squares method or the sum of absolute values ​​method.

5. The method according to claim 1, characterized in that, The speech activity detection module detects the speech activity detection attributes of each audio frame, including: Extract auxiliary features from each of the audio frames; wherein the auxiliary features include short-time zero-crossing rate and fundamental frequency; If the sound energy of the audio frame is greater than a preset sound energy threshold, the short-time zero-crossing rate is within a first preset range, and the fundamental frequency is within a second preset range, then the speech activity detection attribute of the audio frame is determined to be speech. If the sound energy of the audio frame is less than or equal to a preset sound energy threshold, or the short-time zero-crossing rate is not within the first preset range, or the fundamental frequency is not within the second preset range, then the speech activity detection attribute of the audio frame is determined to be non-speech.

6. The method according to any one of claims 1-5, characterized in that, Also includes: The target audio frames between the start time point and the corresponding end time point are subjected to noise reduction preprocessing; wherein, the noise reduction preprocessing includes removing environmental noise in the target audio frames using spectral subtraction or Wiener filtering algorithms; Each of the noise-reduced target audio frames is converted into a target format that matches the speech recognition module.

7. An audio extraction device for short speech recognition, characterized in that, include: The acquisition module is used to acquire the real-time audio data stream from the microphone and divide the audio data stream into frames to obtain multiple audio frames. The voice attribute determination module is used to determine the sound energy of each audio frame, send the sound energy of each audio frame to the voice activity detection module for detection, and receive the voice activity detection attributes of each audio frame returned by the voice activity detection module. The start time determination module is used to determine the start time point of the user's speech by traversing each audio frame in chronological order based on the sound energy and speech activity detection attributes of each audio frame. The termination time determination module is used to determine the termination time of the user's voice based on the start time point and the voice activity detection attributes of each audio frame. An audio capture module is used to capture at least one target audio frame between the start time point and the end time point, and send the target audio frame to the speech recognition module.

8. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1-6.

Images