Audio data segmentation method and apparatus

Abstract

This invention discloses an audio data segmentation method and apparatus. The method involves segmenting the acquired audio data and searching for the largest silent segment within a predetermined search window at each segment. The starting and ending positions of the found largest silent segment are used to determine the audio segmentation point, and the segmentation is then performed. This reduces the computational load and shortens the segmentation time, while also resulting in more uniform audio segment lengths, facilitating subsequent processing.

Description

Technical Field

[0001] This invention relates to the field of audio technology, and more specifically to an audio data segmentation method and apparatus. Background Technology

[0002] With the development of society, people are using speech recognition more and more widely in their daily lives and work.

[0003] In speech recognition applications, the time required from providing audio to the recognizer to obtaining a complete recognition result is called end-to-end latency. Typically, it is positively correlated with the length of the input speech. In long speech recognition applications, the speech data is longer, and the end-to-end latency is more significant. To reduce the end-to-end latency of long speech recognition systems, given that the speech data is provided all at once, the system can adopt a parallel recognition approach. This involves first segmenting the input audio into several segments, then recognizing each segment simultaneously on different processors, and finally merging the recognition results of each segment to obtain a complete recognition result. For parallel speech recognition, audio data segmentation is necessary. Silence segments in the audio are suitable locations for speech segmentation. VAD (Voice Activity Detection) technology can be used to find silence segments in the audio data. The traditional method of VAD for finding silence segments is to scan the entire audio from left to right. During the scan, each audio frame is sequentially determined to be a silence frame. When the number of consecutive (not necessarily continuous) silence frames exceeds a certain threshold, a silence segment is considered detected.

[0004] However, traditional speech segmentation methods require processing all audio frames, resulting in a large computational load. Since processing time is directly proportional to the length of the input audio, it is also time-consuming when the audio is long. Furthermore, the algorithm for selecting which silence segments to segment after scanning all silence segments is relatively complex. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide an audio data segmentation method and apparatus to reduce the amount of computation during audio segmentation, shorten the audio segmentation time, and at the same time make the length of the segmented audio segments more uniform for subsequent processing.

[0006] In a first aspect, embodiments of the present invention provide an audio data segmentation method, the method comprising:

[0007] Acquire audio data, including speech;

[0008] The audio data is segmented according to a first length, and the segmentation points are determined.

[0009] A search window of predetermined length is determined at all segmentation points, the length of which is a second length;

[0010] Search for the largest silent segment in the audio data within each search window;

[0011] The segmentation point is determined and segmentation is performed based on the start and end positions of the largest silent segment found in the search.

[0012] Wherein, the first length is greater than the second length.

[0013] Optionally, the method further includes:

[0014] The first length is determined based on the length of the audio data and the expected number of segments.

[0015] Optionally, at least one silent segment exists within the search window.

[0016] Optionally, the search for the maximum silent segment includes:

[0017] The maximum silent segment is searched using a speech activity detection algorithm.

[0018] Optionally, the method further includes:

[0019] Voice activity detection is performed on multiple search windows in parallel.

[0020] Optionally, the step involves determining the segmentation point and performing segmentation based on the start and end positions of the searched maximum silent segment. Specifically:

[0021] The first segmentation point is the third length extending backward from the starting position of the largest silent segment found in the search, and the second segmentation point is the third length extending forward from the ending position of the largest silent segment found in the search.

[0022] Optionally, the third length is no greater than half the second length.

[0023] Secondly, embodiments of the present invention provide an audio data segmentation apparatus, the apparatus comprising:

[0024] The acquisition module is configured to acquire audio data, including speech.

[0025] The segmentation module is configured to segment the audio data into segments of a first length and determine the segmentation points;

[0026] A window module is configured to determine a search window of a predetermined length at all segmentation points, the search window length being a second length;

[0027] The search module is configured to search for the largest silent segment in the audio data within each search window;

[0028] The segmentation module is configured to determine the segmentation point based on the start and end positions of the largest silent segment found and then perform segmentation.

[0029] Wherein, the first length is greater than the second length.

[0030] Thirdly, embodiments of the present invention provide an electronic device, including a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method as described in the first aspect of the present invention.

[0031] Fourthly, embodiments of the present invention provide a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the method as described in the first aspect of the present invention.

[0032] This invention segmentes the acquired audio data and searches for the largest silent segment within a predetermined search window at each segment. The starting and ending positions of the found largest silent segment are used to determine the audio segmentation point, and the segmentation is then performed. This reduces the computational load and shortens the segmentation time, while also resulting in more uniform audio segment lengths, facilitating subsequent processing. Attached Figure Description

[0033] The above and other objects, features and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings, in which:

[0034] Figure 1 This is a flowchart of the audio data segmentation method according to an embodiment of the present invention;

[0035] Figure 2 This is a schematic diagram showing the audio segmentation locations and the search window locations according to an embodiment of the present invention;

[0036] Figure 3 This is a schematic diagram showing the location of the maximum silent segment and the audio segmentation location in an embodiment of the present invention;

[0037] Figure 4 This is a schematic diagram of an audio data segmentation device according to an embodiment of the present invention;

[0038] Figure 5 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0039] The present invention is described below based on embodiments, but the invention is not limited to these embodiments. In the detailed description of the invention below, certain specific details are described in detail. Those skilled in the art will fully understand the invention even without these details. To avoid obscuring the essence of the invention, well-known methods, processes, flows, elements, and circuits are not described in detail.

[0040] Furthermore, those skilled in the art should understand that the accompanying drawings provided herein are for illustrative purposes only and are not necessarily drawn to scale.

[0041] Unless the context explicitly requires it, words such as "including" or "contains" in the instruction manual should be interpreted as including rather than exclusive or exhaustive; that is, meaning "including but not limited to".

[0042] In the description of this invention, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0043] Figure 1 This is a flowchart of an audio data segmentation method according to an embodiment of the present invention. Figure 1 As shown, the audio data segmentation method of this invention includes the following steps:

[0044] Step S110: Obtain audio data including speech.

[0045] Step S120: Segment the audio data according to a first length and determine the segmentation points. Optionally, the first length is determined based on the length of the audio data and the expected number of segments.

[0046] Step S130: Determine a search window of a predetermined length at all segment points, the search window length being a second length. Optionally, a search window of a predetermined length is determined centered on each segment point. Optionally, the predetermined length is less than the segment length. Optionally, the predetermined length should be set such that the search window contains at least one silent segment.

[0047] Step S140: Search for the largest silent segment in the audio data within each search window. Optionally, the largest silent segment can be searched using a speech activity detection algorithm. Preferably, speech activity detection is performed on multiple search windows in parallel. This method ensures that the time required to complete the entire audio segmentation is actually only related to the length of a single search window, and not to the overall length of the audio, thus making the audio segmentation time stable and controllable. Since the time required to complete the entire audio segmentation is only related to the length of a single search window, and the length of the search window is much smaller than the length of the entire audio, the time required to complete the overall segmentation is shorter.

[0048] Step S150: Determine the segmentation point and perform segmentation based on the start and end positions of the searched maximum silence segment. Optionally, the first segmentation point is located three lengths backward from the start position of the searched maximum silence segment, and the second segmentation point is located three lengths forward from the end position of the searched maximum silence segment. This allows for a safety margin to be reserved at the segmentation points to better ensure the integrity of the original audio content.

[0049] This embodiment segments the acquired audio data and searches for the largest silent segment within a pre-defined search window at each segment. The audio segmentation point is determined based on the start and end positions of the found largest silent segment, and the segmentation is then performed. This reduces the computational load and shortens the segmentation time, while also resulting in more uniform audio segment lengths, facilitating subsequent processing.

[0050] Figure 2 This is a schematic diagram illustrating the audio segmentation locations and the search window locations according to an embodiment of the present invention. Figure 2 As shown, after acquiring the audio data, it is divided into several expected segments 21 according to the audio data length and the expected number of segments. Specifically, the audio data length is divided by the expected number of segments to obtain the length of the expected segment 21, and then the audio data is divided into several segments according to this length. For example, if an audio file is 3 minutes long, and currently 6 threads can be allocated to perform speech recognition in parallel, it is expected to be divided into 6 segments, that is, 6 segments, each segment being 30 seconds long. A search window of a predetermined length is determined at all segmentation points 22, and 231, 232, and 233 are search windows at different segmentation points. Optionally, the audio data is not necessarily divided into segments as shown. Figure 2The number of segments shown, specifically the number of segments and the segment length, can be adjusted according to actual needs. Optionally, at least one silent segment exists within the search windows 231, 232, and 233 of the predetermined length. Since the length of each sentence spoken by a normal person is usually no more than 10 seconds, the predetermined length can be set to 10 seconds in this embodiment. This ensures that at least one silent segment is contained within each search window, thereby ensuring that the audio is segmented within the search window. Since VAD operations are performed only within the search window centered on each segmentation point, rather than processing the entire audio, the computational load is significantly reduced. Preferably, VAD detection is performed on all search windows in parallel. That is, the search windows in the entire audio are processed simultaneously, making the time for segmenting the entire audio essentially equal to the time for segmenting the audio within a single search window. In this embodiment of the invention, VAD detection is performed on search windows 231, 232, and 233 in parallel, so that the time to complete the entire audio segmentation is essentially equal to the time to complete VAD detection for a single search window 231. Therefore, the time required to complete the entire audio segmentation is actually only related to the length of a single search window, and not to the overall length of the audio. Thus, the segmentation time is stable and controllable. Since the time required to complete the entire audio segmentation is only related to the length of a single search window, and the length of the search window is much smaller than the length of the entire audio, the time required to complete the overall segmentation is even shorter.

[0051] The search window determines the segmentation location based on the largest silent segment found. Figure 3 In search window 231 (same) Figure 2 The example shown in Figure 231) illustrates in detail the determination of audio segmentation position by searching for the largest silent segment and determining the segmentation position. Figure 3 This is a schematic diagram showing the location of the maximum silence segment and the audio segmentation location according to an embodiment of the present invention. For example... Figure 3 As shown, after finding the largest silent segment 31 within the search window 231, a safety length is reserved at both the start and end positions of the largest silent segment before segmentation. Specifically, a safety length 331 is added to the right of the largest silent segment 31 as the first segmentation point, and this segmentation point serves as the end position of the preceding audio segment. A safety length 332 is added to the left of the largest silent segment 31 as the second segmentation point, and this segmentation point serves as the start position of the subsequent audio segment. The safety lengths 331 and 332 are equal, and the segmentation point 32 is not located within either the safety lengths 331 or 332. In an optional embodiment, the sum of the safety lengths 331 and 332 is not greater than the length of the largest silent segment 31. In an optional embodiment, the safety lengths 331 and 332 are not equal. In an optional embodiment, the segmentation point 32 may also be located within either the safety lengths 331 or 332.

[0052] Since the length of each sentence spoken by a normal person typically does not exceed 10 seconds, this embodiment sets the search window length to 10 seconds (other search window lengths can be used in practice based on speech characteristics). Assuming the longest silence segment detected in search window 231 is 4 seconds, and the set safety length is 1 second, an audio segmentation is performed at the end of the first second of the 4-second silence segment, retaining the first second of the 4-second silence segment in the first audio segment. Another audio segmentation is performed at the end of the third second of the 4-second silence segment, so that the fourth second of the 4-second silence segment is retained in the second audio segment. The second and third seconds of the 4-second silence segment are truncated and not processed further. In an optional embodiment, the safety lengths before and after the longest silence segment can also be unequal. For example, an audio segmentation can be performed at the end of the 2nd second of the aforementioned 4-second silent segment, retaining the 1st and 2nd seconds of the silent segment in the first audio segment. Then, another audio segmentation can be performed at the end of the 3rd second of the same 4-second silent segment, retaining the 4th second of the silent segment in the second audio segment, while the 3rd second is truncated and not processed further. The safe length can be set as needed, and this embodiment does not impose any limitations on it. This method of finding the start and end positions of audio segments within a search window centered on the segmentation point ensures that the difference in segment length will not exceed the length of the search window, resulting in more uniform segment sizes, which is beneficial for subsequent processing stages.

[0053] Figure 4 This is a schematic diagram of an audio data segmentation device according to an embodiment of the present invention. Figure 4 As shown, the device includes an acquisition module 41, a segmentation module 42, a window module 43, a search module 44, and a division module 45.

[0054] The acquisition module 41 is configured to acquire audio data including speech. The segmentation module 42 is configured to segment the audio data into segments of a first length and determine segmentation points. Optionally, the first length is determined based on the length of the audio data and the expected number of segments. The window module 43 is configured to determine search windows of a predetermined length at all segmentation points, the search window length being a second length. Optionally, a search window of a predetermined length is determined centered on each segmentation point. Optionally, the predetermined length is less than the segment length. Optionally, the predetermined length is set such that the search window contains at least one silent segment. The search module 44 is configured to search for the largest silent segment in the audio data within each search window. Optionally, the largest silent segment is searched using a speech activity detection algorithm. Preferably, speech activity detection is performed on multiple search windows in parallel. This method ensures that the time required to complete the entire audio segmentation is actually only related to the length of a single search window, and not to the overall length of the audio, thus making the audio segmentation time stable and controllable. Since the time required to complete the entire audio segmentation depends only on the length of a single search window, and the length of the search window is much shorter than the length of the entire audio, the time required to complete the overall segmentation is shorter. The segmentation module 45 is configured to determine the segmentation point based on the start and end positions of the found maximum silence segment and then perform the segmentation. Optionally, the first segmentation point is located three lengths backward from the start position of the found maximum silence segment, and the second segmentation point is located three lengths forward from the end position of the found maximum silence segment. This allows for a safety margin to be reserved at the segmentation points to ensure the integrity of the original audio content.

[0055] This embodiment segments the acquired audio data and searches for the largest silent segment within a pre-defined search window at each segment. The audio segmentation point is determined based on the start and end positions of the found largest silent segment, and the segmentation is then performed. This reduces the computational load and shortens the segmentation time, while also resulting in more uniform audio segment lengths, facilitating subsequent processing.

[0056] This invention also provides an electronic device. Figure 5 This is a schematic diagram of an electronic device according to an embodiment of the present invention. (For example...) Figure 5 As shown, Figure 5 The illustrated electronic device is a general-purpose address lookup device, comprising a general-purpose computer hardware architecture, including at least a processor 51 and a memory 52. ​​The processor 51 and memory 52 are connected via a bus 53. The memory 52 is adapted to store instructions or programs executable by the processor 51. The processor 51 can be a standalone microprocessor or a collection of one or more microprocessors. Thus, the processor 51 executes instructions stored in the memory 52, thereby performing tasks such as... Figure 1The method flow of the illustrated embodiment implements data processing and control of other devices. Bus 53 connects the aforementioned components together, and also connects these components to display controller 54, display device, and input / output (I / O) device 55. Input / output (I / O) device 55 may be a mouse, keyboard, modem, network interface, touch input device, motion-sensing input device, printer, and other devices known in the art. Typically, input / output device 55 is connected to the system via input / output (I / O) controller 56.

[0057] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus (devices), or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0058] This application is described with reference to flowchart illustrations of methods, apparatus (devices), and computer program products according to embodiments of this application. It should be understood that each step in the flowchart can be implemented by computer program instructions.

[0059] These computer program instructions may be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction means, the implementation process of which is described in the instruction means. Figure 1 The function specified in one or more processes.

[0060] These computer program instructions may also be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, produce instructions for implementing processes. Figure 1 A device for a function specified in one or more processes.

[0061] Another embodiment of the present invention relates to a non-volatile storage medium for storing a computer-readable program for use by a computer to execute some or all of the above-described method embodiments.

[0062] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program specifying the relevant hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0063] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. For those skilled in the art, the present invention can be modified and varied in various ways. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of protection of the present invention.

Claims

1. An audio data segmentation method, characterized by, The method comprises: acquiring audio data comprising speech; segmenting the audio data by a first length to determine segmentation points, the first length being determined according to a length of the audio data divided by an expected number of segments; determining search windows of a predetermined length at all segmentation points, the search window length being a second length, the second length being set so that each search window contains at least one silent segment; performing speech activity detection on audio data in multiple search windows in a parallel manner to search for a maximum silent segment; determining a split point and performing splitting according to a start position and an end position of the searched maximum silent segment; wherein the first length is greater than the second length; the determining a split point and performing splitting according to a start position and an end position of the searched maximum silent segment specifically comprises: extending the start position of the searched maximum silent segment backward by a third length as a first split point and performing splitting, and extending the end position of the searched maximum silent segment forward by a third length as a second split point and performing splitting, the third length being preset, wherein a sum of the third length extended backward and the third length extended forward is not greater than a length of the maximum silent segment, to ensure that the first split point and the second split point are both located within a corresponding search window.

2. An audio data segmentation apparatus characterized by comprising: The apparatus comprises: an acquisition module configured to acquire audio data comprising speech; a segmentation module configured to segment the audio data by a first length to determine segmentation points, the first length being determined according to a length of the audio data divided by an expected number of segments; a window module configured to determine search windows of a predetermined length at all segmentation points, the search window length being a second length, the second length being set so that each search window contains at least one silent segment; a search module configured to perform speech activity detection on audio data in multiple search windows in a parallel manner to search for a maximum silent segment; a splitting module configured to determine a split point and perform splitting according to a start position and an end position of the searched maximum silent segment; wherein the first length is greater than the second length; the splitting module is further configured to: extend the start position of the searched maximum silent segment backward by a third length as a first split point and perform splitting, and extend the end position of the searched maximum silent segment forward by a third length as a second split point and perform splitting, the third length being preset, wherein a sum of the third length extended backward and the third length extended forward is not greater than a length of the maximum silent segment, to ensure that the first split point and the second split point are both located within a corresponding search window.

3. An electronic device comprising a memory and a processor, characterized in that, The memory is configured to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method as claimed in claim 1.

4. A computer readable storage medium having stored thereon computer program instructions, characterized in that, The computer program instructions, when executed by the processor, implement the method as claimed in claim 1.

Images