A method, device, electronic device and storage medium for separating two-person voices

Abstract

The present application relates to a method, device, electronic device and storage medium for separating two-person speech. The method comprises: acquiring a mixed speech signal received by a microphone array, performing short-time Fourier transform on the mixed speech signal to obtain a mixed speech signal in a time-frequency domain form; obtaining a speech signal of a first channel and a speech signal of a second channel from the mixed speech signal in the time-frequency domain form through a blind source separation algorithm; detecting the state of the speech signal of the first channel and the state of the speech signal of the second channel; determining a first speech signal of the first channel and a first speech signal of the second channel according to the state; determining the orientation of the first channel and the second channel, and determining a second speech signal of the first channel and a second speech signal of the second channel according to the orientation of the first channel and the second channel; performing short-time Fourier inverse transform on the second speech signal of the first channel and the second speech signal of the second channel to obtain speech time domain signals of two target sound sources, and accurately separating speech.

Description

Technical Field

[0001] The present application relates to the technical field of speech separation, and in particular to a method, device, electronic device and storage medium for separating two-person speech. Background Art

[0002] After de-reverberation, the speech signal collected by the microphone array is blindly separated to obtain the speech signal of each target. There is a lot of speech residue after blind source separation, and the separation result is poor when there is speech overlap; when the application scenario is switched, there may be channel separation errors. Summary of the invention

[0003] Based on the above problems, the present application provides a two-person speech separation method, an electronic device and a storage medium.

[0004] In a first aspect, an embodiment of the present application provides a two-person speech separation method, comprising:

[0005] Acquire a mixed voice signal received by the microphone array, where the mixed voice signal includes: a first target sound source and a second target sound source;

[0006] Performing short-time Fourier transform on the mixed speech signal to obtain the mixed speech signal in time-frequency domain;

[0007] After de-reverberation of the mixed speech signal in the time-frequency domain, a speech signal of the first channel and a speech signal of the second channel are obtained by a blind source separation algorithm;

[0008] Detecting a state of a voice signal of a first channel and a state of a voice signal of a second channel;

[0009] Determine the first voice signal of the first channel and the first voice signal of the second channel according to the state of the voice signal of the first channel and the state of the voice signal of the second channel, preset the threshold of the voice signal of the first channel, preset the threshold of the voice signal of the second channel, set energy comparison 1, set energy comparison 2, and the energy of each frame;

[0010] Determine the pitch angle and azimuth of the first channel and the pitch angle and azimuth of the second channel, determine the pitch angle deviation of the first channel and the azimuth deviation of the first channel, the pitch angle deviation of the second channel and the azimuth deviation of the second channel;

[0011] Determine a second voice signal of the first channel and a second voice signal of the second channel according to the pitch angle deviation of the first channel and the azimuth angle deviation of the first channel and the pitch angle deviation of the second channel and the pitch angle deviation of the second channel;

[0012] The second speech signal of the first channel and the second speech signal of the second channel are subjected to inverse short-time Fourier transform to obtain a speech time domain signal of the first target sound source and a speech time domain signal of the second target sound source.

[0013] Furthermore, in the above two-person speech separation method, the state of the speech signal of the first channel and the state of the speech signal of the second channel are detected, and the state of the speech signal of the first channel and the state of the speech signal of the second channel are respectively detected by a speech activity detection algorithm;

[0014] Further, in the above two-person speech separation method, determining the pitch angle and azimuth of the first channel and the pitch angle and azimuth of the second channel, the pitch angle deviation of the first channel and the azimuth deviation of the first channel, the pitch angle deviation of the second channel and the azimuth deviation of the second channel, includes:

[0015] Acquire the pitch angle and azimuth of the first target sound source and the pitch angle and azimuth of the second target sound source;

[0016] Calculating a pitch angle deviation of the first channel and an azimuth angle deviation of the first channel according to the pitch angle and azimuth angle of the first channel and the pitch angle and azimuth angle of the first target sound source;

[0017] The pitch angle deviation of the second channel and the azimuth angle deviation of the second channel are calculated according to the pitch angle and azimuth angle of the second channel and the pitch angle and azimuth angle of the second target sound source.

[0018] Furthermore, in the above two-person speech separation method, according to the state of the speech signal of the first channel and the state of the speech signal of the second channel, the threshold of the speech signal of the first channel is preset, the threshold of the speech signal of the second channel is preset, the energy contrast 1 is set, the energy contrast 2 is set, and the energy of each frame determines the first speech signal of the first channel and the first speech signal of the second channel, including:

[0019] The state of the voice signal of the first channel is greater than the threshold of the voice signal of the first channel preset and the state of the voice signal of the second channel is greater than the threshold of the voice signal of the second channel preset, using energy contrast 1, when the energy of the first frame in the first channel is greater than or equal to the sum of the energy of the second channel and the set energy contrast 1; then the first frame is the first voice signal of the first channel, when the energy of the first frame in the second channel is greater than or equal to the sum of the energy of the first channel and the set energy contrast 1; then the first frame is the first voice signal of the second channel;

[0020] After the state of the voice signal of the first channel is less than or equal to the threshold of the voice signal of the first channel preset, the state of the voice signal of the second channel is less than or equal to the threshold of the voice signal of the second channel preset, energy contrast 2 is used. When the energy of the first frame in the first channel is greater than or equal to the sum of the energy of the second channel and the set energy contrast 2, the first frame is the first voice signal of the first channel. When the energy of the first frame in the second channel is greater than or equal to the sum of the energy of the first channel and the set energy contrast 2, the first frame is the first voice signal of the second channel.

[0021] Furthermore, in the above two-person speech separation method, determining the pitch angle and azimuth of the first channel and the pitch angle and azimuth of the second channel includes:

[0022] The mixed speech signal received by the microphone array is estimated through two-dimensional direction of arrival to obtain the pitch angle and azimuth angle of each frame;

[0023] When the state of the speech signal of the first channel is greater than the threshold value of the speech signal of the first channel preset, the two-dimensional wave arrival direction estimation of the current frame is recorded; when the state of the speech signal of the first channel is less than the threshold value of the speech signal of the first channel preset, the pitch angle mode and the azimuth angle mode of the first channel estimation are calculated to obtain the pitch angle and azimuth angle of the first channel;

[0024] When the state of the speech signal of the second channel is greater than the threshold of the speech signal of the second channel preset, the two-dimensional wave arrival azimuth estimation of the current frame is recorded; when the state of the speech signal of the second channel is greater than the threshold of the speech signal of the second channel preset, the elevation angle mode and azimuth mode of the second channel estimation are calculated.

[0025] Further, in the above two-person speech separation method, determining the second speech signal of the first channel and the second speech signal of the second channel according to the pitch angle deviation of the first channel and the azimuth angle deviation of the first channel and the pitch angle deviation of the second channel and the pitch angle deviation of the second channel includes:

[0026] If the pitch angle deviation of the first channel is less than or equal to the pitch angle deviation of the second channel, or if the azimuth angle deviation of the first channel is less than or equal to the azimuth angle deviation of the second channel, the corresponding output frame is the second speech signal of the first channel;

[0027] If the pitch angle deviation of the first channel is greater than the pitch angle deviation of the second channel, or if the azimuth angle deviation of the first channel is greater than the azimuth angle deviation of the second channel, the corresponding output frame is the second speech signal of the second channel.

[0028] Furthermore, in the above two-person speech separation method, before performing short-time Fourier inverse transform on the second speech signal of the first channel and the second speech signal of the second channel, the method further includes:

[0029] The single-channel noise reduction algorithm eliminates background noise and obtains a clean first-channel second speech signal and a clean second-channel second speech signal.

[0030] In a second aspect, the embodiment of the present application further provides a two-person speech separation device, comprising:

[0031] An acquisition module is used to acquire a mixed voice signal received by a microphone array, wherein the mixed voice signal includes: a first target sound source and a second target sound source;

[0032] The first obtaining module is used to perform short-time Fourier transform on the mixed speech signal to obtain the mixed speech signal in the time-frequency domain;

[0033] Separation module: used to obtain the speech signal of the first channel and the speech signal of the second channel by using a blind source separation algorithm after de-reverberation of the mixed speech signal in the time-frequency domain;

[0034] Detection module: used to detect the state of the voice signal of the first channel and the state of the voice signal of the second channel;

[0035] A first determination module: used to determine the first voice signal of the first channel and the first voice signal of the second channel according to the state of the voice signal of the first channel and the state of the voice signal of the second channel, the preset threshold of the voice signal of the first channel, the preset threshold of the voice signal of the second channel, the setting of energy comparison 1, the setting of energy comparison 2, and the energy of each frame;

[0036] A second determination module: used to determine the pitch angle and azimuth of the first channel and the pitch angle and azimuth of the second channel, determine the pitch angle deviation of the first channel and the azimuth deviation of the first channel, the pitch angle deviation of the second channel and the azimuth deviation of the second channel;

[0037] A third determination module: used for determining a second voice signal of the first channel and a second voice signal of the second channel according to the pitch angle deviation of the first channel and the azimuth angle deviation of the first channel and the pitch angle deviation of the second channel and the pitch angle deviation of the second channel;

[0038] The second obtaining module is used to perform short-time Fourier inverse transform on the second speech signal of the first channel and the second speech signal of the second channel to obtain the speech time domain signal of the first target sound source and the speech time domain signal of the second target sound source.

[0039] In a third aspect, an embodiment of the present application further provides an electronic device, characterized in that it includes: a processor and a memory;

[0040] The processor is used to execute the above two-person speech separation method by calling the program or instruction stored in the memory.

[0041] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores a program or instruction, and the program or instruction enables the computer to perform the above-mentioned two-person speech separation method.

[0042] The advantages of the embodiments of the present application are: the present application relates to a method, device, electronic device and storage medium for separating two-person voices, the method comprising: obtaining a mixed voice signal received by a microphone array, performing a short-time Fourier transform on the mixed voice signal to obtain a mixed voice signal in a time-frequency domain form; obtaining a voice signal of a first channel and a voice signal of a second channel by a blind source separation algorithm from the mixed voice signal in the time-frequency domain form; detecting the state of the voice signal of the first channel and the state of the voice signal of the second channel; determining the first voice signal of the first channel and the first voice signal of the second channel according to the state; determining the orientation of the first channel and the second channel, and determining the second voice signal of the first channel and the second voice signal of the second channel according to the orientation of the first channel and the second channel; performing an inverse short-time Fourier transform on the second voice signal of the first channel and the second voice signal of the second channel to obtain voice time domain signals of two target sound sources, the present application combines dereverberation and blind source separation technology to achieve a preliminary separation effect, and on this basis, utilizes a voice activity detection algorithm and energy judgment to solve the problem of voice overlap, and utilizes orientation judgment to solve the problem of output errors when switching application scenarios, and finally, single-channel noise reduction is used to make the target voice cleaner and accurately separate the two-channel voice. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the conventional technology, the drawings required for use in the embodiments or the conventional technology descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0044] Figure 1 A method for separating two-person speech provided in the present application is shown in FIG. Figure 1 ;

[0045] Figure 2 A method for separating two-person speech provided in the present application is shown in FIG. Figure 2 ;

[0046] Figure 3 A schematic diagram of a two-person speech separation device provided in an embodiment of the present application;

[0047] Figure 4 It is a schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0050] Figure 1 A method for separating two-person speech provided in the present application is shown in FIG. Figure 1 .

[0051] In the first aspect, the present application embodiment provides a two-person speech separation method, combining Figure 1 , including eight steps from S101 to S108:

[0052] S101: Acquire a mixed speech signal received by a microphone array, where the mixed speech signal includes: a first target sound source and a second target sound source.

[0053] Specifically, the embodiment of the present application is based on a circular microphone array designed with a radius of r. The pitch angle θ of the first target sound source A can be in the direction of 30 degrees, the azimuth angle φ can be in the direction of 60 degrees, and the second target sound source B can be in any other direction.

[0054] S102: Perform short-time Fourier transform on the mixed speech signal to obtain a mixed speech signal in time-frequency domain.

[0055] Specifically, in the embodiment of the present application, the mixed speech signal received by the microphone is short-time Fourier transformed to obtain a mixed speech signal x(t,k) in the time-frequency domain form of data, where t represents the frame number and k represents the frequency.

[0056] S103: After dereverberation of the mixed speech signal in the time-frequency domain, a speech signal of the first channel and a speech signal of the second channel are obtained by using a blind source separation algorithm.

[0057] Specifically, in the embodiment of the present application, the speech signal of the first channel and the speech signal of the second channel are two different signals, and a preliminary separation effect is achieved by combining dereverberation and blind source separation algorithms.

[0058] S104: Detect the state of the voice signal of the first channel and the state of the voice signal of the second channel.

[0059] Specifically, in the embodiment of the present application, the state of the voice signal of the first channel and the state of the voice signal of the second channel are respectively detected by a voice activity detection algorithm.

[0060] S105: Determine the first speech signal of the first channel and the first speech signal of the second channel according to the state of the speech signal of the first channel and the state of the speech signal of the second channel, the preset threshold of the speech signal of the first channel, the preset threshold of the speech signal of the second channel, the setting of energy comparison 1, the setting of energy comparison 2, and the energy of each frame.

[0061] S106: Determine the pitch angle and azimuth angle of the first channel and the pitch angle and azimuth angle of the second channel, determine the pitch angle deviation of the first channel and the azimuth angle deviation of the first channel, the pitch angle deviation of the second channel and the azimuth angle deviation of the second channel.

[0062] S107: Determine a second speech signal of the first channel and a second speech signal of the second channel according to the pitch angle deviation of the first channel and the azimuth angle deviation of the first channel and the pitch angle deviation of the second channel and the pitch angle deviation of the second channel.

[0063] Specifically, in the embodiment of the present application, the speech signals of the first channel and the second channel are compared by energy and orientation using the state of each channel through the above S105 to S107 to obtain the speech signals of the first target sound source and the second target sound source that are finally output.

[0064] S108: Performing inverse short-time Fourier transform on the second speech signal of the first channel and the second speech signal of the second channel to obtain a speech time domain signal of the first target sound source and a speech time domain signal of the second target sound source.

[0065] Specifically, in the embodiment of the present application, the time-frequency domain form of the speech signal is subjected to an inverse short-time Fourier transform to obtain a speech time-domain signal of the first target sound source and a speech time-domain signal of the second target sound source.

[0066] Figure 2 A method for separating two-person speech provided in the present application is shown in FIG. Figure 2 .

[0067] Further, in the above two-person speech separation method, determining the pitch angle and azimuth of the first channel and the pitch angle and azimuth of the second channel, the pitch angle deviation of the first channel and the azimuth deviation of the first channel, the pitch angle deviation of the second channel and the azimuth deviation of the second channel, includes:

[0068] S201: Acquire the pitch angle and azimuth of the first target sound source and the pitch angle and azimuth of the second target sound source;

[0069] S202: Calculate the pitch angle deviation and azimuth deviation of the first channel according to the pitch angle and azimuth of the first channel and the pitch angle and azimuth of the first target sound source; calculate the pitch angle deviation and azimuth deviation of the second channel according to the pitch angle and azimuth of the second channel and the pitch angle and azimuth of the second target sound source.

[0070] Specifically, in the embodiment of the present application, the pitch angle θ of the first target sound source A can be at 30 degrees, the azimuth angle φ can be at 60 degrees, and the second target sound source B can be at any other azimuth. Calculate the pitch angle deviation of the first channel and the azimuth deviation of the first channel Similarly, the pitch angle deviation of the second channel can be obtained Azimuth deviation from the first channel

[0071] Furthermore, in the above two-person speech separation method, according to the state of the speech signal of the first channel and the state of the speech signal of the second channel, the threshold of the speech signal of the first channel is preset, the threshold of the speech signal of the second channel is preset, the energy contrast 1 is set, the energy contrast 2 is set, and the energy of each frame determines the first speech signal of the first channel and the first speech signal of the second channel, including:

[0072] The state of the voice signal of the first channel is greater than the threshold of the voice signal of the first channel preset and the state of the voice signal of the second channel is greater than the threshold of the voice signal of the second channel preset, using energy contrast 1, when the energy of the first frame in the first channel is greater than or equal to the sum of the energy of the second channel and the set energy contrast 1; then the first frame is the first voice signal of the first channel, when the energy of the first frame in the second channel is greater than or equal to the sum of the energy of the first channel and the set energy contrast 1; then the first frame is the first voice signal of the second channel;

[0073] After the state of the voice signal of the first channel is less than or equal to the threshold of the voice signal of the first channel preset, the state of the voice signal of the second channel is less than or equal to the threshold of the voice signal of the second channel preset, energy contrast 2 is used. When the energy of the first frame in the first channel is greater than or equal to the sum of the energy of the second channel and the set energy contrast 2, the first frame is the first voice signal of the first channel. When the energy of the first frame in the second channel is greater than or equal to the sum of the energy of the first channel and the set energy contrast 2, the first frame is the first voice signal of the second channel.

[0074] Specifically, in the embodiment of the present application, if the state of the first channel and the status of the second channel The threshold of the speech signal of the first channel and the threshold of the speech signal of the second channel are respectively and like and Energy contrast 1 is used, that is, when the energy of a frame in the first channel is greater than or equal to the sum of the energy of the second channel and the set energy contrast 1 E1≥E2+σ1, the first channel is retained and the second channel is eliminated; when the energy of a frame in the second channel is greater than or equal to the sum of the energy of the first channel and the set energy contrast 1 E2≥E1+σ1, channel 2 is retained and channel 1 is eliminated; otherwise, energy contrast 2 is used, that is, when the energy of a frame in the first channel is greater than or equal to the sum of the energy of the second channel and the set energy contrast 2 E1≥E2+σ2, the corresponding frame of channel 1 is selected; when the energy of a frame in the second channel is greater than or equal to the sum of the energy of the first channel and the set energy contrast 2 E2≥E1+σ2, the corresponding frame of channel 2 is selected.

[0075] Furthermore, in the above two-person speech separation method, determining the pitch angle and azimuth of the first channel and the pitch angle and azimuth of the second channel includes:

[0076] The mixed speech signal received by the microphone array is estimated through two-dimensional direction of arrival to obtain the pitch angle and azimuth angle of each frame;

[0077] When the state of the speech signal of the first channel is greater than the threshold of the speech signal of the first channel preset, the two-dimensional wave arrival azimuth estimation of the current frame is recorded; when the state of the speech signal of the first channel is less than the threshold of the speech signal of the first channel preset, the pitch angle mode and the azimuth angle mode of the first channel estimation are calculated to obtain the pitch angle and azimuth angle of the first channel.

[0078] When the state of the speech signal of the second channel is greater than the threshold of the speech signal of the second channel preset, the two-dimensional wave arrival azimuth estimation of the current frame is recorded; when the state of the speech signal of the second channel is greater than the threshold of the speech signal of the second channel preset, the elevation angle mode and azimuth mode of the second channel estimation are calculated.

[0079] Specifically, in the embodiment of the present application, when When , the two-dimensional wave arrival direction estimation of the current frame is recorded. Calculate the mode of the saved first channel position estimate The same reason Record the orientation estimate of the current frame. Calculate the mode of the saved second channel position estimate

[0080] Further, in the above two-person speech separation method, determining the second speech signal of the first channel and the second speech signal of the second channel according to the pitch angle deviation of the first channel and the azimuth angle deviation of the first channel and the pitch angle deviation of the second channel and the pitch angle deviation of the second channel includes:

[0081] If the pitch angle deviation of the first channel is less than or equal to the pitch angle deviation of the second channel, or if the azimuth angle deviation of the first channel is less than or equal to the azimuth angle deviation of the second channel, the corresponding output frame is the second speech signal of the first channel;

[0082] If the pitch angle deviation of the first channel is greater than the pitch angle deviation of the second channel, or if the azimuth angle deviation of the first channel is greater than the azimuth angle deviation of the second channel, the corresponding output frame is the second speech signal of the second channel.

[0083] Specifically, in this application, if the pitch angle deviation of the first channel is less than or equal to the pitch angle deviation of the second channel in the example Or the azimuth deviation of the first channel is less than or equal to the azimuth deviation of the second channel Then these frames are the second speech signals of the first channel, otherwise they are the second speech signals of the second channel.

[0084] Furthermore, in the above two-person speech separation method, before performing short-time Fourier inverse transform on the second speech signal of the first channel and the second speech signal of the second channel, the method further includes:

[0085] The single-channel noise reduction algorithm eliminates background noise and obtains a clean first-channel second speech signal and a clean second-channel second speech signal.

[0086] Specifically, in the embodiment of the present application, single-channel noise reduction is used to make the target speech cleaner and accurately separate the two-channel speech.

[0087] Figure 3 A schematic diagram of a two-person speech separation device provided in an embodiment of the present application.

[0088] In a second aspect, the present application also provides a two-person voice separation device, combined with Figure 3 ,include:

[0089] The acquisition module 301 is used to acquire a mixed speech signal received by a microphone array, where the mixed speech signal includes a first target sound source and a second target sound source.

[0090] Specifically, the embodiment of the present application is based on a circular microphone array with a radius r. The pitch angle θ of the first target sound source A obtained by the acquisition module 301 can be in the direction of 30 degrees, the azimuth angle φ can be in the direction of 60 degrees, and the second target sound source B can be in any other direction.

[0091] The first obtaining module 302 is used to perform short-time Fourier transform on the mixed speech signal to obtain the mixed speech signal in the time-frequency domain.

[0092] Specifically, in the embodiment of the present application, the first obtaining module 302 performs short-time Fourier transform on the mixed speech signal received by the microphone to obtain a mixed speech signal x(t,k) in the time-frequency domain, where t represents the frame number and k represents the frequency.

[0093] Separation module 303: used to obtain a speech signal of a first channel and a speech signal of a second channel by using a blind source separation algorithm after de-reverberation of the mixed speech signal in the time-frequency domain.

[0094] Specifically, in the embodiment of the present application, the speech signal of the first channel and the speech signal of the second channel are two different signals, and the separation module 303 combines the dereverberation and blind source separation algorithms to achieve a preliminary separation effect.

[0095] Detection module 304: used to detect the state of the speech signal of the first channel and the state of the speech signal of the second channel.

[0096] Specifically, in the embodiment of the present application, the detection module 304 detects the state of the voice signal of the first channel and the state of the voice signal of the second channel respectively through a voice activity detection algorithm.

[0097] The first determination module 305 is used to determine the first speech signal of the first channel and the first speech signal of the second channel according to the state of the speech signal of the first channel and the state of the speech signal of the second channel, the threshold of the speech signal of the first channel, the threshold of the speech signal of the second channel, the energy comparison 1, the energy comparison 2, and the energy of each frame;

[0098] The second determination module 306 is used to determine the pitch angle and azimuth of the first channel and the pitch angle and azimuth of the second channel, determine the pitch angle deviation of the first channel and the azimuth deviation of the first channel, the pitch angle deviation of the second channel and the azimuth deviation of the second channel;

[0099] The third determining module 307 is used to determine the second speech signal of the first channel and the second speech signal of the second channel according to the pitch angle deviation of the first channel and the azimuth angle deviation of the first channel and the pitch angle deviation of the second channel.

[0100] Specifically, in the embodiment of the present application, the first determination module 305, the second determination module 306 and the third determination module 307 utilize the state of each channel to compare the speech signals of the first channel and the second channel through energy comparison and orientation determination to obtain the speech signals of the first target sound source and the second target sound source that are finally output.

[0101] The second obtaining module 308 is used to perform inverse short-time Fourier transform on the second speech signal of the first channel and the second speech signal of the second channel to obtain a speech time domain signal of the first target sound source and a speech time domain signal of the second target sound source.

[0102] Specifically, in the embodiment of the present application, the second obtaining module 308 performs a short-time inverse Fourier transform on the time-frequency domain form of the speech signal to obtain the speech time domain signal of the first target sound source and the speech time domain signal of the second target sound source.

[0103] In a third aspect, an embodiment of the present application further provides an electronic device, characterized in that it includes: a processor and a memory;

[0104] The processor is used to execute the above two-person speech separation method by calling the program or instruction stored in the memory.

[0105] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores a program or instruction, and the program or instruction enables the computer to perform the above-mentioned two-person speech separation method.

[0106] Figure 4 It is a schematic block diagram of an electronic device provided by an embodiment of the present disclosure.

[0107] like Figure 4 As shown, the electronic device includes: at least one processor 401, at least one memory 402 and at least one communication interface 403. The various components in the electronic device are coupled together through a bus system 404. The communication interface 403 is used to transmit information between external devices. It can be understood that the bus system 404 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 404 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 4 Various buses are labeled as bus system 404 .

[0108] It can be understood that the memory 402 in this embodiment can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories.

[0109] In some implementations, the memory 402 stores the following elements, executable units or data structures, or a subset thereof, or an extended set thereof: an operating system and application programs.

[0110] The operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application includes various application programs, such as a media player (Media Player), a browser (Browser), etc., which are used to implement various application services. A program that implements any method of a two-person voice separation method provided in an embodiment of the present application can be included in the application.

[0111] In an embodiment of the present application, the processor 401 calls a program or instruction stored in the memory 402, specifically, a program or instruction stored in an application, and the processor 401 is used to execute the steps of each embodiment of a two-person speech separation method provided in an embodiment of the present application.

[0112] Acquire a mixed voice signal received by the microphone array, where the mixed voice signal includes: a first target sound source and a second target sound source;

[0113] Performing short-time Fourier transform on the mixed speech signal to obtain the mixed speech signal in time-frequency domain;

[0114] After de-reverberation of the mixed speech signal in the time-frequency domain, a speech signal of the first channel and a speech signal of the second channel are obtained by a blind source separation algorithm;

[0115] Detecting a state of a voice signal of a first channel and a state of a voice signal of a second channel;

[0116] Determine the first voice signal of the first channel and the first voice signal of the second channel according to the state of the voice signal of the first channel and the state of the voice signal of the second channel, preset the threshold of the voice signal of the first channel, preset the threshold of the voice signal of the second channel, set energy comparison 1, set energy comparison 2, and the energy of each frame;

[0117] Determine the pitch angle and azimuth of the first channel and the pitch angle and azimuth of the second channel, determine the pitch angle deviation of the first channel and the azimuth deviation of the first channel, the pitch angle deviation of the second channel and the azimuth deviation of the second channel;

[0118] Determine a second voice signal of the first channel and a second voice signal of the second channel according to the pitch angle deviation of the first channel and the azimuth angle deviation of the first channel and the pitch angle deviation of the second channel and the pitch angle deviation of the second channel;

[0119] The second speech signal of the first channel and the second speech signal of the second channel are subjected to inverse short-time Fourier transform to obtain a speech time domain signal of the first target sound source and a speech time domain signal of the second target sound source.

[0120] Any method in a two-person speech separation method provided in an embodiment of the present application can be applied to the processor 401, or implemented by the processor 401. The processor 401 can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 401 or an instruction in the form of software. The above processor 401 can be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0121] The steps of any method in a two-person speech separation method provided in an embodiment of the present application can be directly embodied as being executed by a hardware decoding processor, or being executed by a combination of hardware and software units in a decoding processor. The software unit can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 402, and the processor 401 reads the information in the memory 402, and completes the steps of a two-person speech separation method in combination with its hardware.

[0122] Those skilled in the art will appreciate that although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features from different embodiments is meant to be within the scope of the present application and to form different embodiments.

[0123] Those skilled in the art will appreciate that the description of each embodiment has its own emphasis, and for parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0124] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for separating two-person speech, characterized in that: include: Acquire a mixed voice signal received by a microphone array, wherein the mixed voice signal includes: a first target sound source and a second target sound source; Performing short-time Fourier transform on the mixed speech signal to obtain a mixed speech signal in time-frequency domain; After de-reverberation of the mixed speech signal in the time-frequency domain, a speech signal of the first channel and a speech signal of the second channel are obtained by a blind source separation algorithm; Detecting a state of a voice signal of a first channel and a state of a voice signal of a second channel; The state of the voice signal of the first channel is greater than the threshold of the voice signal of the first channel preset and the state of the voice signal of the second channel is greater than the threshold of the voice signal of the second channel preset, using energy contrast 1, when the energy of the first frame in the first channel is greater than or equal to the sum of the energy of the second channel and the set energy contrast 1; then the first frame is the first voice signal of the first channel, when the energy of the first frame in the second channel is greater than or equal to the sum of the energy of the first channel and the set energy contrast 1; then the first frame is the first voice signal of the second channel; After the state of the voice signal of the first channel is less than or equal to the threshold of the voice signal of the first channel, the state of the voice signal of the second channel is less than or equal to the threshold of the voice signal of the second channel, using energy comparison 2, when the energy of the first frame in the first channel is greater than or equal to the sum of the energy of the second channel and the set energy comparison 2; then the first frame is the first voice signal of the first channel, when the energy of the first frame in the second channel is greater than or equal to the sum of the energy of the first channel and the set energy comparison 2; then the first frame is the first voice signal of the second channel; Acquire the pitch angle and azimuth of the first target sound source and the pitch angle and azimuth of the second target sound source; Calculating a pitch angle deviation of the first channel and an azimuth angle deviation of the first channel according to the pitch angle and azimuth angle of the first channel and the pitch angle and azimuth angle of the first target sound source; Calculating a pitch angle deviation of the second channel and an azimuth angle deviation of the second channel according to the pitch angle and azimuth angle of the second channel and the pitch angle and azimuth angle of the second target sound source; Determine a second voice signal of the first channel and a second voice signal of the second channel according to the pitch angle deviation of the first channel and the azimuth angle deviation of the first channel and the pitch angle deviation of the second channel and the pitch angle deviation of the second channel; The second speech signal of the first channel and the second speech signal of the second channel are subjected to inverse short-time Fourier transform to obtain a speech time domain signal of the first target sound source and a speech time domain signal of the second target sound source.

2. A two-person speech separation method according to claim 1, characterized in that: The state of the voice signal of the first channel and the state of the voice signal of the second channel are detected by using a voice activity detection algorithm; 3. A two-person speech separation method according to claim 1, characterized in that: The determining of the pitch angle and azimuth of the first channel and the pitch angle and azimuth of the second channel, the pitch angle deviation of the first channel and the azimuth deviation of the first channel, the pitch angle deviation of the second channel and the azimuth deviation of the second channel comprises: Acquire the pitch angle and azimuth of the first target sound source and the pitch angle and azimuth of the second target sound source; Calculating a pitch angle deviation of the first channel and an azimuth angle deviation of the first channel according to the pitch angle and azimuth angle of the first channel and the pitch angle and azimuth angle of the first target sound source; The pitch angle deviation of the second channel and the azimuth angle deviation of the second channel are calculated according to the pitch angle and azimuth angle of the second channel and the pitch angle and azimuth angle of the second target sound source.

4. A two-person speech separation method according to claim 1, characterized in that: The method of determining the second voice signal of the first channel and the second voice signal of the second channel according to the pitch angle deviation of the first channel and the azimuth angle deviation of the first channel and the pitch angle deviation of the second channel and the pitch angle deviation of the second channel comprises: If the pitch angle deviation of the first channel is less than or equal to the pitch angle deviation of the second channel, or if the azimuth angle deviation of the first channel is less than or equal to the azimuth angle deviation of the second channel, the corresponding output frame is the second speech signal of the first channel; If the pitch angle deviation of the first channel is greater than the pitch angle deviation of the second channel, or if the azimuth angle deviation of the first channel is greater than the azimuth angle deviation of the second channel, the corresponding output frame is the second speech signal of the second channel.

5. A two-person speech separation method according to claim 1, characterized in that: Before performing short-time Fourier inverse transform on the second speech signal of the first channel and the second speech signal of the second channel, the method further includes: The single-channel noise reduction algorithm eliminates background noise and obtains a clean first-channel second speech signal and a clean second-channel second speech signal.

6. A two-person speech separation device, characterized in that: include: An acquisition module is used to acquire a mixed voice signal received by a microphone array, wherein the mixed voice signal includes: a first target sound source and a second target sound source; The first obtaining module is used for performing short-time Fourier transform on the mixed speech signal to obtain a mixed speech signal in time-frequency domain; Separation module: used for obtaining a speech signal of a first channel and a speech signal of a second channel by using a blind source separation algorithm after de-reverberation of the mixed speech signal in the time-frequency domain; Detection module: used to detect the state of the voice signal of the first channel and the state of the voice signal of the second channel; A first determination module: used to determine the first voice signal of the first channel and the first voice signal of the second channel according to the state of the voice signal of the first channel and the state of the voice signal of the second channel, the preset threshold of the voice signal of the first channel, the preset threshold of the voice signal of the second channel, the setting of energy comparison 1, the setting of energy comparison 2, and the energy of each frame; A second determination module: used to determine the pitch angle and azimuth of the first channel and the pitch angle and azimuth of the second channel, determine the pitch angle deviation of the first channel and the azimuth deviation of the first channel, the pitch angle deviation of the second channel and the azimuth deviation of the second channel; A third determination module: used for determining a second voice signal of the first channel and a second voice signal of the second channel according to the pitch angle deviation of the first channel and the azimuth angle deviation of the first channel and the pitch angle deviation of the second channel and the pitch angle deviation of the second channel; The second obtaining module is used to perform short-time Fourier inverse transform on the second speech signal of the first channel and the second speech signal of the second channel to obtain the speech time domain signal of the first target sound source and the speech time domain signal of the second target sound source.

7. An electronic device, characterized in that: include: Processor and memory; The processor is used to execute the two-person speech separation method as described in any one of claims 1 to 5 by calling the program or instruction stored in the memory.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program or instruction, and the program or instruction enables a computer to execute a two-person speech separation method as described in any one of claims 1 to 5.

Images