Methods and systems for speech separation

Abstract

This disclosure relates to a speech separation method and system using a sliding window. The method includes: acquiring at least one speech from at least one user by at least one microphone and storing the at least one speech as a speech signal in a sound recording module; extracting the speech signal from the sound recording module through a sliding window and processing the extracted speech signal; and transmitting the processed speech signal to a DUET module for speech separation.

Description

Technical Field

[0001] The present invention relates to a system for speech separation and a method performed in the system, and more particularly to a system and method for improving speech separation performance by means of a sliding window. Background Technology

[0002] In recent years, more and more vehicles have voice recognition capabilities. However, when more than one person speaks in the vehicle at the same time, the vehicle's main unit will be unable to quickly identify the driver's voice from multiple voices, making it impossible to execute the corresponding operations accurately and promptly according to the driver's instructions, and easily causing erroneous operations.

[0003] Currently, there are two main methods for performing speech separation. The first is to create a microphone array for speech enhancement, and the second is to use algorithms for speech separation. Various algorithms used for speech separation can include FDICA (Frequency Domain Independent Component Analysis), DUET (Degradation Separation Estimation Technique), or their extended algorithms.

[0004] The DUET blind source separation method can separate any number of speech sources using only two mixtures. The method is effective when the sources are W-disjoint orthogonal, i.e., when the windowed Fourier transforms of the signals in the mixture support disjointness. For anechoic mixtures of attenuation and delay sources, the method allows for estimation of mixing parameters by clustering relative attenuation-delay pairs extracted from the ratio of the mixed time-frequency representation. The estimated mixing parameters are then used to partition a mixed time-frequency representation to recover the original source.

[0005] Figure 1 This diagram illustrates a conventional speech separation system comprising two microphones, a sound recording module, and a DUET module. For example, both microphones are initially turned on simultaneously, initiating recording. As two people begin to speak, the sound recording module receives and stores the speech signals from both microphones. Figure 1 In the example shown, the first sound (sound 1) belongs to the first person (person 1) and the second sound (sound 2) belongs to the second person (person 2). The DUET module receives the signal from the sound recording module, and then analyzes and separates the signal to recover the original sound source.

[0006] In practice, for example, if the speech segment is 4 seconds long, the DUET module will directly process the 4-second speech segment. Due to the complexity of the DUET algorithm, processing speech data will take a relatively long time. Typically, speech signals are sparse and contain a large amount of information concentrated in a very short time period. Most of the time, there is no speech signal in the received signal. However, due to the complexity of the DUET algorithm, the DUET module still waits for a certain period of time (such as the entire speech segment, 4 seconds) and spends a relatively long time processing the received signal.

[0007] Therefore, there is a need to develop an improved speech separation system and method that can quickly perform speech separation in order to rapidly recover the original sound source. Summary of the Invention

[0008] In one or more illustrative embodiments, a method for speech separation is provided. The method uses at least one microphone to acquire at least one speech from at least one user and stores the at least one speech as a speech signal in a sound recording module. The method further extracts the speech signal from the sound recording module via a sliding window and processes the extracted speech signal, and transmits the processed speech signal to a DUET module for speech separation.

[0009] Preferably, in one embodiment, the method uses a sliding window by: traversing the extracted speech signal to determine the maximum amplitude of the speech signal; determining the starting position of the sliding window, the starting position being the position where the amplitude of the speech signal first exceeds a predetermined proportion of the maximum amplitude at the beginning of the speech signal; determining the ending position of the sliding window, the ending position being the position where the amplitude of the speech signal first exceeds the predetermined proportion of the maximum amplitude at the end of the speech signal and returns to the beginning of the speech signal; and selecting segments of the speech signal located between the starting position and the ending position of the sliding window as the processed speech signal for speech separation.

[0010] Preferably, in another embodiment, the method uses a sliding window by: traversing the extracted speech signal to determine the average amplitude of the speech signal; determining the starting position of the sliding window, the starting position being the position where the amplitude of the speech signal first exceeds the average amplitude at the beginning of the speech signal; determining the ending position of the sliding window, the ending position being the position where the amplitude of the speech signal first exceeds the average amplitude at the beginning of the speech signal from the end of the speech signal; and selecting a segment of the speech signal located between the starting position and the ending position of the sliding window as the processed speech signal for speech separation.

[0011] In one or more illustrative embodiments, a system for speech separation is provided. The system for speech separation includes: at least one microphone for acquiring at least one speech from at least one user; a sound recording module for storing the at least one speech as a speech signal; a sliding window for extracting the speech signal from the sound recording module and processing the extracted speech signal; and a DUET module for receiving the processed speech signal for speech separation.

[0012] Preferably, in one embodiment, the sliding window is configured to: traverse the extracted speech signal to determine the maximum amplitude of the speech signal; determine the starting position of the sliding window, the starting position of the sliding window being the position where the amplitude of the speech signal first exceeds a predetermined proportion of the maximum amplitude from the beginning of the speech signal; determine the ending position of the sliding window, the ending position of the sliding window being the position where the amplitude of the speech signal first exceeds the predetermined proportion of the maximum amplitude from the end of the speech signal back to the beginning of the speech signal; and select a segment of the speech signal located between the starting position and the ending position of the sliding window as the processed speech signal for speech separation.

[0013] Preferably, in another embodiment, the sliding window is configured to: traverse the extracted speech signal to determine the average amplitude of the speech signal; determine the starting position of the sliding window, the starting position of the sliding window being the position where the amplitude of the speech signal first exceeds the average amplitude at the beginning of the speech signal; determine the ending position of the sliding window, the ending position of the sliding window being the position where the amplitude of the speech signal first exceeds the average amplitude at the end of the speech signal and returning to the beginning of the speech signal; and select a segment of the speech signal located between the starting position and the ending position of the sliding window as the processed speech signal for speech separation.

[0014] A computer-readable medium having computer-executable instructions for performing the aforementioned method is provided.

[0015] Advantageously, the disclosed speech separation system and method can improve the real-time performance of DUET by using a sliding window.

[0016] Those skilled in the art will understand or become aware of the described systems, methods, features, and advantages upon review of the following drawings and detailed description. It is intended that all such additional systems, methods, features, and advantages be included within this specification and within the scope of the invention. Attached Figure Description

[0017] The features, essence, and advantages of this application can be better understood by referring to the following accompanying drawings and description. The components in the drawings are not necessarily drawn to scale, but rather the focus is on illustrating the principles of the invention. Furthermore, in the drawings, the same reference numerals denote corresponding parts in all different views.

[0018] Figure 1 This is a schematic diagram of a conventional speech separation system.

[0019] Figure 2 A schematic diagram of a speech separation system according to an embodiment of the present invention is shown.

[0020] Figure 3 A sliding window is schematically shown in a speech separation system according to one embodiment of the invention.

[0021] Figure 4 A sliding window is schematically shown in a speech separation system according to another embodiment of the invention.

[0022] Figure 5 A flowchart of a speech separation method according to one embodiment of the present invention is shown. Detailed Implementation

[0023] It should be understood that the descriptions of the following implementation examples are given for illustrative purposes only and should not be considered limiting. The division of the examples into functional blocks, modules, or units in the accompanying drawings should not be construed as indicating that these functional blocks, modules, or units must be implemented as physically separate units. The functional blocks, modules, or units shown or described may be implemented as individual units, circuits, chips, functions, modules, or circuit elements. One or more functional blocks or units may also be implemented in a common circuit, chip, circuit element, or unit.

[0024] Figure 2 A schematic diagram of a speech separation system according to an embodiment of the present invention is shown. The speech separation system can be used in a vehicle and may include at least one microphone, a sound recording module, a sliding window module, and a DUET module. For ease of explanation, Figure 2 Only two microphones (microphone 1 and microphone 2) and two people (person 1 and person 2) are shown, but those skilled in the art will understand that the system may include more microphones. The two microphones can acquire at least one voice from at least one user. Figure 2 Two people are shown as an example. For example, the two people could be a driver and a passenger.

[0025] When the system is working, for example, Figure 2As shown, each of the two microphones acquires speech from two people. For example, the first microphone (microphone 1) can collect a first voice (sound 1) from a first person and a second voice (sound 2) from a second person, and then transmit them to a sound recording module for recording as a speech signal that mixes information from both sound sources. Similarly, the second microphone (microphone 1) can collect a first voice (sound 1) from a first person and a second voice (sound 2) from a second person, and then transmit them to the sound recording module for recording as a speech signal that includes information from both sound sources.

[0026] The sliding window module extracts speech signals from the sound recording module and processes the extracted speech signals. The processed speech signals are then transmitted to the DUET module for speech separation. Finally, different speech sources can be separated. For example, the processed speech signals can ultimately be separated into a first speech (speech 1) from a first person and a second speech (speech 2) from a second person.

[0027] The sliding window will refer to Figure 3 and Figure 4 To illustrate. Figure 3 A sliding window is schematically shown in a speech separation system according to one embodiment of the invention.

[0028] For example, the extracted speech signal can last for four seconds, such as Figure 3 As shown. First, the extracted speech signal is traversed to determine the maximum amplitude of the speech signal. Then, the start and end positions of the sliding window are determined. Starting from the beginning of the speech signal, points (such as...) are found. Figure 3 Point X1 is shown. At point X1, the amplitude of the speech signal first exceeds a predetermined proportion of the maximum amplitude. Preferably, the predetermined proportion is greater than or equal to 1 / 4 and less than or equal to 1 / 2. Then, this point X1 is determined as the starting position of the sliding window. Next, points (such as...) are found from the end of the speech signal to the beginning of the speech signal. Figure 3 Point X2 is shown. At point X2, from the end of the speech signal, the amplitude of the speech signal first exceeds a predetermined proportion of the maximum amplitude. This point X2 is then determined as the end position of the sliding window. The window length of the sliding window can be determined based on the start and end positions of the sliding window; that is, the window length is equal to X2 - X1 (as shown). Figure 3 (As shown in x). Next, the segment of the speech signal located between the start and end positions of the sliding window (i.e., the segment within the sliding window) is selected as the processed speech signal and sent to DUET for speech separation.

[0029] Figure 4A sliding window is schematically shown in a speech separation system according to another embodiment of the invention.

[0030] For example, Figure 4 The extracted speech signal can also last for four seconds. First, the average amplitude of the speech signal is determined by traversing the extracted speech signal. Then, the start and end positions of the sliding window are determined. Starting from the beginning of the speech signal, points (such as...) are found. Figure 4 Point X3 is shown. At point X3, the amplitude of the speech signal first exceeds the average amplitude of the speech signal. Then, this point X3 is determined as the starting position of the sliding window. Next, points (such as...) are found from the end of the speech signal to the beginning of the speech signal. Figure 4 Point X4 is shown. At point X4, the amplitude of the speech signal first exceeds the average amplitude from the end of the speech signal. This point X4 is then determined as the end position of the sliding window. The window length of the sliding window can be determined based on the start and end positions of the sliding window; that is, the window length is equal to X4 - X3 (as shown). Figure 4 (As shown in x). Next, the segment of the speech signal located between the start and end positions of the sliding window (i.e., the segment within the sliding window) is selected as the processed speech signal and sent to DUET for speech separation.

[0031] Figure 5 A flowchart of a speech separation method according to one embodiment of the present invention is shown.

[0032] like Figure 5 As shown, in step 501, at least one voice message from at least one user is acquired by at least one microphone and then stored as a voice signal in the sound recording module. In step 502, the voice signal transmitted from the sound recording module is further processed using a sliding window and then sent to the DUET module for voice separation. In step 503, the processed voice signal is transmitted to the DUET module.

[0033] The process of using a sliding window at step 502 may include determining the window length of the sliding window and selecting a segment of the speech signal that falls within the window length of the sliding window as the speech signal to be processed for further speech separation.

[0034] According to one embodiment of the invention, determining the window length of the sliding window may include traversing the extracted speech signal to determine the maximum amplitude of the speech signal. Then, the start position and end position of the sliding window are determined to obtain the window length. The start position of the sliding window is the position where the amplitude of the speech signal first exceeds a predetermined proportion of the maximum amplitude at the beginning of the speech signal. The end position of the sliding window is the position where the amplitude of the speech signal first exceeds a predetermined proportion of the maximum amplitude at the end of the speech signal. Preferably, the predetermined proportion may be greater than or equal to 1 / 4 and less than or equal to 1 / 2.

[0035] According to another embodiment of the invention, determining the window length of the sliding window may include traversing the extracted speech signal to determine the average amplitude of the speech signal. Then, the start position and end position of the sliding window are determined to obtain the window length. For example, the start position of the sliding window is the position where the amplitude of the speech signal first exceeds the average amplitude at the beginning of the speech signal. The end position of the sliding window is the position where the amplitude of the speech signal first exceeds the average amplitude at the end of the speech signal and the beginning of the speech signal.

[0036] The speech separation method and system of this invention introduces a sliding window to preprocess the data, after which the data collected by the microphone is sent to the DUET module for processing. By extracting the relatively concentrated parts of speech information from signal segments and removing unnecessary parts of the segment signals, the amount of data that the DUET algorithm needs to process is reduced, thereby reducing the running time of the DUET algorithm and improving the overall efficiency of the speech separation system.

[0037] The term "module" can be defined as comprising multiple executable modules. A module may include software, hardware, firmware, or some combination thereof that can be executed by a processor. A software module may include instructions stored in memory or another storage device that can be executed by a processor or other processors. A hardware module may include various devices, components, circuits, gates, circuit boards, etc., that can be executed, booted, and / or controlled by a processor for performance.

[0038] One or more programmatic features of the program product define the functionality of the implementation (including the methods described herein) and may be contained on a variety of computer-readable storage media. Illustrative computer-readable storage media include, but are not limited to: (i) non-writable storage media on which information is permanently stored (e.g., read-only memory devices within a computer, such as optical disc read-only memory (CD-ROM) disks readable by a CD-ROM drive, flash memory, read-only memory (ROM) chips, or any type of solid-state non-volatile semiconductor memory); and (ii) writable storage media on which information is modified (e.g., floppy disks or any type of solid-state random access semiconductor memory within a floppy disk drive or hard disk drive).

[0039] The present invention has been described above with reference to specific embodiments. However, those skilled in the art will understand that various modifications and changes can be made to the specific embodiments without departing from the broader spirit and scope of the invention as set forth in the appended claims.

Claims

1. A method for speech separation, comprising: At least one voice is acquired from at least one user by at least one microphone and the at least one voice is stored as a voice signal in a sound recording module; The audio signal is extracted from the audio recording module through a sliding window and the extracted audio signal is processed. as well as The processed voice signal is transmitted to the DUET module for voice separation; The extracted speech signal processed through the sliding window includes: The extracted speech signals are iterated to determine the maximum amplitude of the speech signals; The starting position of the sliding window is determined from the position where the amplitude of the speech signal first exceeds a predetermined proportion of the maximum amplitude at the beginning of the speech signal. Determine the end position of the sliding window, the end position of the sliding window being the position from the end of the speech signal to the beginning of the speech signal, where the amplitude of the speech signal first exceeds a predetermined proportion of the maximum amplitude; and The segment of the speech signal located between the start position and the end position of the sliding window is selected as the speech signal to be processed for speech separation.

2. The method according to claim 1, wherein the predetermined ratio is greater than or equal to 1 / 4 and less than or equal to 1 / 2.

3. A method for speech separation, comprising: At least one voice is acquired from at least one user by at least one microphone and the at least one voice is stored as a voice signal in a sound recording module; The audio signal is extracted from the audio recording module through a sliding window and the extracted audio signal is processed. as well as The processed voice signal is transmitted to the DUET module for voice separation. The extracted speech signal processed through the sliding window includes: The extracted speech signals are iterated to determine the average amplitude of the speech signals; The starting position of the sliding window is determined from the position where the amplitude of the speech signal first exceeds the average amplitude at the beginning of the speech signal. Determine the end position of the sliding window, which is the position from the end of the speech signal to the beginning of the speech signal, where the amplitude of the speech signal first exceeds the average amplitude; The segment of the speech signal located between the start position and the end position of the sliding window is selected as the speech signal to be processed for speech separation.

4. A system for speech separation, comprising: At least one microphone, the at least one microphone acquiring at least one voice from at least one user; A sound recording module, wherein the sound recording module is used to store the at least one speech as a speech signal; A sliding window, used to extract the speech signal from the sound recording module and process the extracted speech signal; as well as The DUET module is used to receive the processed speech signal for speech separation; The sliding window is further configured as follows: The extracted speech signals are iterated through to determine the maximum amplitude of the speech signals; The starting position of the sliding window is determined from the position where the amplitude of the speech signal first exceeds a predetermined proportion of the maximum amplitude at the beginning of the speech signal. The end position of the sliding window is determined, wherein the end position of the sliding window is the position from the end of the speech signal to the beginning of the speech signal, where the amplitude of the speech signal first exceeds a predetermined proportion of the maximum amplitude; and The segment of the speech signal located between the start position and the end position of the sliding window is selected as the speech signal to be processed for speech separation.

5. The system according to claim 4, wherein the predetermined ratio is greater than or equal to 1 / 4 and less than or equal to 1 / 2.

6. A system for speech separation, comprising: At least one microphone, the at least one microphone acquiring at least one voice from at least one user; A sound recording module, wherein the sound recording module is used to store the at least one speech as a speech signal; A sliding window, used to extract the speech signal from the sound recording module and process the extracted speech signal; as well as The DUET module is used to receive the processed speech signal for speech separation; The sliding window is further configured as follows: The speech signal is extracted by traversal to determine the average amplitude of the speech signal; The starting position of the sliding window is determined from the position where the amplitude of the speech signal first exceeds the average amplitude at the beginning of the speech signal. Determine the end position of the sliding window, which is the position from the end of the speech signal to the beginning of the speech signal, where the amplitude of the speech signal first exceeds the average amplitude; The segment of the speech signal located between the start position and the end position of the sliding window is selected as the speech signal to be processed for speech separation.

7. A computer-readable medium having computer-executable instructions for performing the method according to any one of claims 1-3.

Images