An audio acquisition method, apparatus, device, and medium

Abstract

The application relates to an audio acquisition method, device and equipment and a medium, and relates to the technical field of voice interaction. The method comprises the following steps: obtaining a target effective frame and an identification frame of target audio; the target effective frame is an effective frame adjacent to the identification frame; the identification frame comprises a buffer frame and a supplementary frame; a link frame of the identification frame is extracted according to the amplitude and zero-crossing rate of the target effective frame; and the link frame and all the effective frames are integrated to obtain output audio. The method can complete the missing parts at the beginning and end of effective voice, ensure the completeness and coherence of the output audio, and improve the recognition accuracy of the back-end voice application end.

Description

Technical Field

[0001] This invention relates to the field of voice interaction technology, and in particular to an audio acquisition method, apparatus, device, and medium. Background Technology

[0002] Currently, devices with voice interaction capabilities mostly achieve voice detection through active triggering or calculation of differences in the time and frequency domain statistical characteristics of noise. Although the calculation is simple, the performance is poor in low signal-to-noise ratio, non-stationary noise scenarios, and voice classification. While large AI models on the market have advantages in the VAD field, they have high requirements for computing power, are difficult to port to ordinary mobile terminals, and lack real-time performance, failing to meet the audio preprocessing needs of terminal devices with limited computing power for real-time voice detection. Summary of the Invention

[0003] This invention provides an audio acquisition method to solve the problem that existing speech detection schemes often have missing beginnings and ends of effective speech, which in turn affects the accuracy of speech recognition.

[0004] In a first aspect, the present invention provides an audio acquisition method, the method comprising:

[0005] Obtain the target valid frame and the identifier frame of the target audio; the target valid frame is a valid frame adjacent to the identifier frame, and the identifier frame includes a buffer frame and a supplementary frame;

[0006] The connecting frames of the identified frames are extracted based on the amplitude and zero-crossing rate of the target valid frames;

[0007] The output audio is obtained by integrating the connecting frames and all the valid frames.

[0008] Secondly, the present invention provides an audio acquisition device, comprising:

[0009] The acquisition unit is used to acquire the target valid frame and the identifier frame of the target audio; the target valid frame is a valid frame adjacent to the identifier frame, and the identifier frame includes a buffer frame and a supplementary frame;

[0010] An extraction unit is used to extract the connecting frames of the identifier frames based on the amplitude and zero-crossing rate of the target valid frames;

[0011] An integration unit is used to integrate the connecting frames and all the valid frames to obtain the output audio.

[0012] Thirdly, an electronic device is provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0013] Memory, used to store computer programs;

[0014] When a processor executes a program stored in a memory, it implements the steps of the audio acquisition method described in any embodiment of the first aspect.

[0015] Fourthly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the audio acquisition method as described in any embodiment of the first aspect.

[0016] The technical solutions provided in the embodiments of the present invention have the following advantages compared with the prior art:

[0017] This audio acquisition method is easy to port, requires low CPU computing power, and can be quickly imported into different embedded platforms. It can accurately identify multiple audio types, has a high probability of noise filtering, and effectively reduces CPU usage and backend processing model computing power. By completing the missing parts at the beginning and end of the effective speech, it ensures that the output audio is complete and coherent, significantly improving the recognition accuracy of the backend speech application and achieving the effect of "hearing clearly and accurately". Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating an audio acquisition method provided in an embodiment of the present invention;

[0021] Figure 2 This is a schematic diagram of a sub-process of an audio acquisition method provided in an embodiment of the present invention;

[0022] Figure 3 A schematic diagram of a sub-process of another audio acquisition method provided in an embodiment of the present invention;

[0023] Figure 4 A schematic diagram of a sub-process of another audio acquisition method provided in an embodiment of the present invention;

[0024] Figure 5 A schematic diagram of a sub-process of another audio acquisition method provided in an embodiment of the present invention;

[0025] Figure 6 This is a schematic diagram of the structure of an audio acquisition device provided in an embodiment of the present invention;

[0026] Figure 7 A schematic diagram of the structure of a computer device provided in an embodiment of the present invention;

[0027] Figure 8 A schematic diagram of a target audio effect provided in an embodiment of the present invention;

[0028] Figure 9 This is a schematic diagram of a marked target audio effect provided in an embodiment of the present invention. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] Example

[0031] See Figures 1-9 This invention proposes an audio acquisition method, which specifically includes the following steps S101-S103.

[0032] S101, obtain the target valid frame and the identifier frame of the target audio; the target valid frame is a valid frame adjacent to the identifier frame, and the identifier frame includes a buffer frame and a supplementary frame.

[0033] S102, extract the connecting frame of the identifier frame based on the amplitude and zero-crossing rate of the target valid frame.

[0034] S103, integrate the connecting frames and all the valid frames to obtain the output audio.

[0035] In a specific embodiment, the target valid frame refers to the audio frame adjacent to the identifier frame that is determined by the classification model to contain valid human voice; the identifier frame refers to the audio frame used to supplement the beginning and end information of valid speech, including buffer frames and supplementary frames, which are initially determined to be invalid frames but may contain continuous human voice segments; amplitude refers to the vibration amplitude of the audio signal, reflecting the strength of the sound; zero-crossing rate refers to the number of times the audio signal waveform crosses the zero level per unit time, reflecting the frequency characteristics of the sound; the connecting frame refers to the audio segment extracted from the identifier frame that is continuously matched with the amplitude and zero-crossing rate characteristics of the target valid frame; the output audio refers to the complete and clean human voice audio obtained after integrating the connecting frame with all valid frames.

[0036] In a voice wake-up scenario, the user says "play music" to the speaker. After the speaker's microphone picks up the raw audio, it first identifies the valid frames containing the words "play," "play," "sound," and "music." Simultaneously, it identifies adjacent buffer frames (frames corresponding to the unstressed syllables before "play") and supplementary frames (frames corresponding to the final sound after "music"). It calculates the average amplitude and zero-crossing rate of the target valid frames corresponding to "play," and extracts the feature-matching connecting frames (i.e., the unstressed syllable segment before "play") from the buffer frames. It also calculates the average amplitude and zero-crossing rate of the target valid frames corresponding to "music," and extracts the feature-matching connecting frames (i.e., the final sound segment after "music") from the supplementary frames. Finally, it integrates these connecting frames with all valid frames to form a complete "play music" audio message, which is then transmitted to the backend. The backend quickly and accurately recognizes the command and executes the playback operation.

[0037] This method overcomes the limitations of traditional audio acquisition, which only retains clearly defined valid frames. By extracting connecting frames from the identified frames, it fills in easily missing segments at the beginning and end of valid speech, ensuring audio integrity. Connecting frames are filtered based on amplitude and zero-crossing rate to ensure a natural and coherent connection between the connecting parts and valid frames, avoiding audio breaks. The integrated audio has less noise and more complete information, providing high-quality data for backend recognition and significantly improving recognition accuracy. This audio acquisition method is simple to port, requires low CPU computing power, and can be quickly imported into different embedded platforms. It can accurately identify multiple audio types, has a high probability of noise filtering, and effectively reduces CPU usage and the computing power of the backend processing model.

[0038] In one embodiment, step S101 above includes steps S201-S202:

[0039] S201, the original audio is divided into multiple audio frames according to the preset duration.

[0040] S202, the audio frames are divided into one of the following according to the pre-trained classification model: buffer frames, valid frames, and supplementary frames.

[0041] In a specific embodiment, the preset duration refers to the fixed time length for segmenting the original audio, which is used to convert continuous audio into discrete audio frames for processing; the original audio refers to the unprocessed audio signal directly acquired by acquisition devices such as microphones; the audio frame refers to the discrete audio segment obtained after segmenting the original audio according to the preset duration; the pre-trained classification model is a model trained on a large amount of audio data and has the ability to identify audio type and determine frame type.

[0042] In a voice input scenario, the user inputs voice information through a microphone. First, the collected original audio is segmented into multiple audio frames according to a preset duration of 280 ms. Subsequently, these audio frames are input into a pre-trained classification model, which quickly identifies the audio types of each frame, determines that some frames are valid frames containing human voices, marks the invalid frames before the valid frames as cache frames, and marks the invalid frames after the valid frames as supplementary frames, preparing for subsequent coherent frame extraction and audio integration.

[0043] Segmenting the audio by a fixed preset duration makes subsequent processing more regular and operable, adapting to the frame processing logic of the model; accurately classifying frame types with the aid of a pre-trained classification model provides a reliable basis for valid frame screening and identification frame determination, with higher efficiency and stronger accuracy compared to traditional manual determination of frame types, laying a foundation for the real-time performance and effectiveness of the overall solution.

[0044] In one embodiment, when the identification frame is a cache frame, step S103 above includes the steps of: merging the coherent frame into the target valid frame based on the reverse amplitude.

[0045] In a specific embodiment, the reverse amplitude refers to the amplitude feature calculated in the reverse direction from the adjacent end of the valid frame of the cache frame to the other end, and is used to screen the segments coherent with the target valid frame.

[0046] In a vehicle voice control scenario, when the user says "open the window", in the collected original audio, the audio frame corresponding to the word "open" is marked as a cache frame due to the low initial volume, and the frame corresponding to the word "open" is marked as the target valid frame. At this time, based on the reverse amplitude calculation, the amplitude is analyzed segment by segment in the reverse direction from the adjacent end of the word "open" in the cache frame, and the audio segment (coherent frame) of the word "open" that is coherent with the amplitude feature of the frame of the word "open" is extracted, and this coherent frame is merged in front of the target valid frame corresponding to the word "open" to ensure the integrity of the beginning of the audio of "open the window".

[0047] Clarify the merging method of the coherent frame corresponding to the cache frame, accurately locate the coherent human voice segment through the reverse amplitude calculation, and avoid the problem of missing the beginning of the voice caused by directly discarding the cache frame; the merging operation is simple and efficient, without adding too much additional computing power burden, taking into account both audio integrity and processing efficiency.

[0048] In one embodiment, step S202 above includes steps S301 - S304:

[0049] S301, identify the audio types of each audio frame through the classification model, screen out the target type corresponding to the human voice and score it, with the scoring range being 0 - 100;

[0050] S302, if the scoring result is higher than the preset threshold, mark this audio frame as a valid frame;

[0051] S303, if the current audio frame is an invalid frame and the next frame is a valid frame, then mark the current invalid frame as a buffered frame;

[0052] S304. If the current audio frame is an invalid frame and the previous frame is a valid frame, then mark the current invalid frame as a supplementary frame.

[0053] In a specific embodiment, an invalid frame refers to an audio frame that, after being identified and scored by the classification model, has a score lower than a preset threshold and is therefore determined to be an audio frame that does not contain valid human voices. The preset threshold is a scoring threshold used to distinguish between valid and invalid frames and can be adjusted according to the usage scenario.

[0054] In a voice command acquisition scenario, a pre-trained classification model identifies each segmented audio frame and can distinguish between various types such as environmental noise, human voice, and animal sounds. It only selects the target type corresponding to human voice and scores it from 0 to 100. A preset threshold of 65 is set. If an audio frame scores 72, which is higher than the threshold, it is marked as a valid frame. If an audio frame scores 48, which is lower than the threshold, it is an invalid frame. If the next frame is a valid frame, it is marked as a cached frame. If another audio frame scores 53, which is lower than the threshold, it is an invalid frame. If the previous frame is a valid frame, it is marked as a supplementary frame.

[0055] The scoring mechanism quantifies the effectiveness of audio frames, making the determination of effective frames more accurate and avoiding errors from subjective judgment. The criteria for determining cached frames and supplementary frames are clearly defined, providing a clear basis for the extraction of subsequent connecting frames, ensuring that the frame type classification logic is rigorous, and improving the standardization and reliability of the overall solution.

[0056] In one embodiment, the preset duration ranges from 250ms to 350ms.

[0057] In a specific embodiment, the preset duration ranges from 250ms to 350ms, taking into account both the false recognition rate and the delay to determine the audio frame segmentation time range, thus balancing recognition accuracy and real-time performance. Generally, a shorter preset duration results in a higher false recognition rate, while a longer preset duration results in a higher delay.

[0058] In the audio acquisition scenario of portable voice recorders, for quiet indoor environments, the preset duration is set to 320ms to reduce the false recognition rate, resulting in more complete audio frame information after segmentation, which facilitates accurate model recognition. In noisy outdoor environments, the preset duration is adjusted to 270ms to reduce latency, ensuring real-time audio processing and quickly filtering out effective human voice frames.

[0059] The preset duration can be flexibly adjusted within the range of 250ms-350ms, which can adapt to the requirements of different usage scenarios for false recognition rate and latency. Compared with fixed duration segmentation, it is more adaptable and can maintain good audio acquisition effect in various environments.

[0060] In one embodiment, the training process of the classification model includes steps S401-S404:

[0061] S401, Obtain the training dataset of audio type, and perform frame segmentation processing on the audio in the dataset.

[0062] S402 extracts the spectral features, temporal dynamic features, and amplitude features of each audio frame to construct multi-label training samples.

[0063] S403 is an offline model built on the deep separable convolutional structure of MobileNet. Training samples are input into the model for training, enabling the model to classify audio types and output probability feedback in the 0-1 interval.

[0064] S404 maps the 0-1 probability output by the model to a score of 0-100, thus completing the training of the classification model.

[0065] In a specific embodiment, the training dataset contains multiple audio types and serves as the audio data set for training the classification model; spectral features refer to the characteristics of the audio signal in the frequency domain, reflecting the frequency distribution of the sound; temporal dynamic features refer to the changing characteristics of the audio signal in the time domain, reflecting the temporal evolution of the sound; MobileNet's deep separable convolutional structure is a lightweight neural network structure that can reduce computational load and parameter size while ensuring model performance; multi-label training samples contain training data of multiple features of audio frames and corresponding category labels, used to improve the model's classification accuracy.

[0066] First, a training dataset containing over 520 audio types, including environmental noise, human voices, animal sounds, and mechanical noises, was collected. The audio in the dataset was segmented into frames with a duration of 300ms. The spectral features, temporal dynamic features, and amplitude features of each audio frame were extracted, and each frame was labeled with its corresponding audio type to construct multi-label training samples. An offline model was built based on the depthwise separable convolutional structure of MobileNet. The training samples were input into the model for iterative training, enabling the model to accurately classify audio types and output a 0-1 range of feedback probabilities. Finally, the 0-1 probabilities output by the model were mapped to 0-100 points to complete the training of the classification model.

[0067] The model employs a training dataset containing over 520 audio types, enabling it to possess broad audio recognition capabilities. Multi-dimensional audio features are extracted to construct training samples, improving the model's classification accuracy. The model is built based on MobileNet's deep separable convolutional structure, achieving lightweight design, reducing computational consumption, and adapting to common embedded devices. Furthermore, probability mapping scoring provides an intuitive and reliable basis for frame type determination.

[0068] In one embodiment, step S102 above includes steps S501-S503:

[0069] S501, calculate the average amplitude and zero-crossing rate of the target effective frame, and determine the feature threshold.

[0070] S502 calculates the average amplitude and zero-crossing rate of each segment in reverse order from the adjacent end of the valid frame of the identifier frame to the other end.

[0071] S503, select segments whose average amplitude and zero-crossing rate meet the aforementioned feature thresholds as connecting frames.

[0072] In a specific embodiment, the feature threshold is a judgment criterion determined based on the average amplitude and zero-crossing rate of the target valid frame, used to filter segments in the identifier frame that are consistent with the features of the valid frame; the valid frame adjacent end refers to the end of the identifier frame that is adjacent to the target valid frame.

[0073] In a voice call audio acquisition scenario, the audio frame corresponding to the user's "hello" during a call is marked as the target valid frame. Its adjacent buffered frames may contain a humming segment before the "hello". First, the average amplitude of the target valid frame for the "hello" is calculated to be 0.6 and the zero-crossing rate is 120. The feature threshold is determined to be an average amplitude of 0.4-0.8 and a zero-crossing rate of 100-140. Starting from the adjacent valid frame of the buffered frames and working backwards to the other end, the average amplitude and zero-crossing rate of each segment are extracted. The segment with an average amplitude of 0.5 and a zero-crossing rate of 115 is selected. This segment meets the feature threshold and is used as a connecting frame to be merged with the target valid frame to ensure that the beginning of the call audio is complete and clear.

[0074] The threshold is determined based on the characteristics of the target valid frame, making the screening of connecting frames more targeted and ensuring that the audio characteristics of connecting frames and valid frames are consistent and natural. The reverse segment-by-segment calculation method can accurately locate valid segments, avoiding omissions or misselections. At the same time, the calculation process is simple and does not increase the computational burden too much, thus balancing the screening accuracy and processing efficiency.

[0075] See Figure 6 This invention also provides an audio acquisition device 400, which includes an acquisition unit 401, an extraction unit 402, and an integration unit 403.

[0076] The acquisition unit 401 is used to acquire the target valid frame and the identifier frame of the target audio; the target valid frame is a valid frame adjacent to the identifier frame, and the identifier frame includes a buffer frame and a supplementary frame.

[0077] In one embodiment, before acquiring the target valid frame and the identifier frame of the target audio, the method further includes:

[0078] The original audio is divided into multiple audio frames according to a preset duration;

[0079] The audio frames are classified into one of the following types based on the pre-trained classification model: buffered frames, valid frames, and supplementary frames.

[0080] In one embodiment, classifying the audio frames into one of buffer frames, valid frames, and supplementary frames according to a pre-trained classification model includes:

[0081] The classification model is used to identify the audio type of each audio frame, filter out the target type corresponding to human voice and score it, with a score range of 0-100.

[0082] If the score is higher than the preset threshold, the audio frame is marked as a valid frame.

[0083] If the current audio frame is invalid and the next frame is valid, then the current invalid frame is marked as a buffered frame;

[0084] If the current audio frame is invalid and the previous frame is valid, then the current invalid frame is marked as a supplementary frame.

[0085] In one embodiment, the preset duration ranges from 250ms to 350ms.

[0086] In one embodiment, the training process of the classification model includes:

[0087] Obtain a training dataset of audio types, and perform frame segmentation on the audio in the dataset;

[0088] Extract the spectral features, temporal dynamic features, and amplitude features of each audio frame to construct multi-label training samples;

[0089] An offline model is built based on the deep separable convolutional structure of MobileNet. Training samples are input into the model for training, enabling the model to classify audio types and output the probability of feedback in the 0-1 interval.

[0090] The 0-1 probability output by the model is mapped to a score of 0-100, thus completing the training of the classification model.

[0091] Extraction unit 402 is used to extract the connecting frame of the identifier frame based on the amplitude and zero-crossing rate of the target valid frame.

[0092] In one embodiment, extracting the connecting frames of the identifier frame based on the amplitude and zero-crossing rate of the target valid frame includes:

[0093] Calculate the average amplitude and zero-crossing rate of the target valid frames to determine the feature threshold;

[0094] Calculate segment by segment from the valid frame adjacent end of the identified frame to the other end, and extract the average amplitude and zero-crossing rate of each segment;

[0095] Segments whose average amplitude and zero-crossing rate meet the aforementioned characteristic thresholds are selected as connecting frames.

[0096] Integration unit 403 is used to integrate the connecting frames and all the valid frames to obtain output audio.

[0097] In one embodiment, when the identifier frame is a buffered frame, the step of integrating the connecting frame and all the valid frames to obtain the output audio includes:

[0098] The connecting frames are merged into the target valid frame based on the reverse amplitude.

[0099] like Figure 5 As shown, Figure 5 This is a schematic block diagram of a computer device provided in an embodiment of this application. The computer device 500 can be a terminal or a server. The terminal can be an electronic device with communication functions, such as a smartphone, tablet, laptop, desktop computer, personal digital assistant, or wearable device. The server can be a standalone server or a server cluster composed of multiple servers.

[0100] The computer device 500 includes a processor 502, a memory, and a network interface 505 connected via a system bus 501. The memory may include a non-volatile storage medium 503 and internal memory 504.

[0101] The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. When the computer program 5032 is executed, it causes the processor 502 to perform an audio acquisition method.

[0102] The processor 502 provides computing and control capabilities to support the operation of the entire computer device 500.

[0103] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute an audio acquisition method.

[0104] The network interface 505 is used for network communication with other devices. Those skilled in the art will understand that the above structure is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device 500 to which the present application is applied. A specific computer device 500 may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements.

[0105] It should be understood that in the embodiments of this application, the processor 502 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0106] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program may be stored in a storage medium, which is a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0107] Therefore, the present invention also provides a storage medium. This storage medium can be a computer-readable storage medium. The storage medium stores a computer program.

[0108] The storage medium is a physical, non-transient storage medium, such as a USB flash drive, external hard drive, read-only memory (ROM), magnetic disk, or optical disk, or any other physical storage medium capable of storing program code. The computer-readable storage medium can be non-volatile or volatile.

[0109] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0110] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0111] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the device of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0112] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0113] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0114] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Since these modifications and variations fall within the scope of the claims and their equivalents, this invention also intends to include these modifications and variations.

[0115] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An audio acquisition method, characterized in that, The method includes: Obtain the target valid frame and the identifier frame of the target audio; the target valid frame is a valid frame adjacent to the identifier frame, and the identifier frame includes a buffer frame and a supplementary frame; wherein, the buffer frame is an invalid frame before the valid frame, and the supplementary frame is an invalid frame after the valid frame; The connecting frames of the identified frames are extracted based on the amplitude and zero-crossing rate of the target valid frames; The output audio is obtained by integrating the connecting frames and all the valid frames. The step of extracting the connecting frames of the identifier frame based on the amplitude and zero-crossing rate of the target valid frame includes: Calculate the average amplitude and zero-crossing rate of the target valid frames to determine the feature threshold; Calculate segment by segment from the valid frame adjacent end of the identified frame to the other end, and extract the average amplitude and zero-crossing rate of each segment; Segments whose average amplitude and zero-crossing rate meet the aforementioned characteristic thresholds are selected as connecting frames.

2. The method according to claim 1, characterized in that, Before acquiring the target valid frame and the identifier frame of the target audio, the process also includes: The original audio is divided into multiple audio frames according to a preset duration; The audio frames are classified into one of the following types based on the pre-trained classification model: buffered frames, valid frames, and supplementary frames.

3. The method according to claim 1, characterized in that, When the identifier frame is a buffered frame, the process of integrating the connecting frame and all valid frames to obtain the output audio includes: The connecting frames are merged into the target valid frame based on the reverse amplitude, where the reverse amplitude is the amplitude feature calculated in reverse from the valid frame adjacent end of the buffered frame to the other end.

4. The method according to claim 2, characterized in that, The step of classifying the audio frames into one of three types—buffered frames, valid frames, and supplementary frames—based on a pre-trained classification model includes: The classification model is used to identify the audio type of each audio frame, filter out the target type corresponding to human voice and score it, with a score range of 0-100. If the score is higher than the preset threshold, the audio frame is marked as a valid frame. If the current audio frame is invalid and the next frame is valid, then the current invalid frame is marked as a buffered frame; If the current audio frame is invalid and the previous frame is valid, then the current invalid frame is marked as a supplementary frame.

5. The method according to claim 2, characterized in that, The preset duration ranges from 250ms to 350ms.

6. The method according to claim 2, characterized in that, The training process of the classification model includes: Obtain a training dataset of audio types, and perform frame segmentation on the audio in the dataset; Extract the spectral features, temporal dynamic features, and amplitude features of each audio frame to construct multi-label training samples; An offline model is built based on the deep separable convolutional structure of MobileNet. Training samples are input into the model for training, enabling the model to classify audio types and output the probability of feedback in the 0-1 interval. The 0-1 probability output by the model is mapped to a score of 0-100, thus completing the training of the classification model.

7. An audio acquisition device, characterized in that, include: An acquisition unit is used to acquire a target valid frame and an identifier frame of the target audio; the target valid frame is a valid frame adjacent to the identifier frame, and the identifier frame includes a buffer frame and a supplementary frame; wherein, the buffer frame is an invalid frame before the valid frame, and the supplementary frame is an invalid frame after the valid frame; An extraction unit is used to extract the connecting frames of the identifier frames based on the amplitude and zero-crossing rate of the target valid frames; An integration unit is used to integrate the connecting frames and all the valid frames to obtain the output audio; The step of extracting the connecting frames of the identifier frame based on the amplitude and zero-crossing rate of the target valid frame includes: Calculate the average amplitude and zero-crossing rate of the target valid frames to determine the feature threshold; Calculate segment by segment from the valid frame adjacent end of the identified frame to the other end, and extract the average amplitude and zero-crossing rate of each segment; Segments whose average amplitude and zero-crossing rate meet the aforementioned characteristic thresholds are selected as connecting frames.

8. A computer device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method according to any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-6.

Images