Voice signal processing method, device and sound pickup device

By using the mixed voice signals collected by voice acquisition devices in different directions, and using environmental interference signals to enhance signals, the problem of high computing resources consumption of voice interactive devices in noisy environments is solved and the voice recognition performance is improved.

CN112289335BActive Publication Date: 2024-11-12ALIBABA GROUP HOLDING LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201910671135.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2019-07-24
Publication Date
2024-11-12
Estimated Expiration
2039-07-24

AI Technical Summary

Technical Problem

In noisy environments, voice interactive devices are difficult to effectively reduce the computing resources required for signal enhancement processing, resulting in poor voice recognition performance.

Method used

By acquiring the mixed voice signals collected by the voice acquisition device and the audio acquisition device in different directions, the target voice signal in the first mixed voice signal is enhanced by using the environmental interference signal in the second mixed voice signal, thereby reducing computing resource consumption and Improve voice enhancement effect.

Benefits of technology

Effectively reduce the computing resource consumption of signal enhancement processing, while improving the voice enhancement effect of target voice signals and improving voice recognition performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112289335B_ABST
    Figure CN112289335B_ABST
Patent Text Reader

Abstract

The embodiment of the present application provides a method, device and sound pickup device for processing a speech signal, which relates to the field of speech processing. The method includes: obtaining a first mixed speech signal and a second mixed speech signal respectively collected by a speech collection device and an audio collection device arranged in a speech interaction device, wherein the speech collection direction of the speech collection device is different from that of the audio collection device; utilizing the environmental interference signal in the second mixed speech signal to perform signal enhancement processing on the target speech signal in the first mixed speech signal to obtain the target speech signal after signal enhancement. Through the embodiment of the present application, utilizing the environmental interference signal in the second mixed speech signal to perform signal enhancement processing on the target speech signal in the first mixed speech signal, not only can the computing resources required for signal enhancement processing be effectively reduced, but also the speech enhancement effect of the target speech signal can be effectively improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of speech processing, and in particular to a speech signal processing method, apparatus, and sound pickup device. Background Art

[0002] In noisy public spaces, such as subway stations and coffee shops, the environmental noise is very serious and has complex components, such as noisy human voices, coffee machine stirring sounds, and the friction of tables and chairs moving. If the voice interaction equipment deployed in such an environment does not perform noise reduction or voice enhancement processing on the collected voice signals, the signal-to-noise ratio of the voice signals will be very low, making it difficult for the voice recognition performance to reach a practical state.

[0003] In the prior art, microphone array technology is an important way to improve the sound pickup quality of voice interaction devices in such noisy environments. Specifically, microphone array technology first uses a linear array or circular array formed by omnidirectional microphones in the array topology, so that starting from the voice collection part, sounds from all directions will be received, including both the desired voice (such as the voice of a person speaking directly to the microphone array), and it is also inevitable to receive various interference noises from other directions and distances. After the sound is collected, it is necessary to use a very complex algorithm to perform signal feature analysis on the target sound and environmental interference, and perform signal enhancement processing based on the signal feature analysis results. However, this requires a lot of computing resources and is difficult to be practical on embedded devices. Summary of the invention

[0004] The purpose of the present application is to propose a method, device and sound pickup device for processing speech signals, so as to solve the problem existing in the prior art of how to effectively reduce the computing resources required for signal enhancement processing.

[0005] According to the first aspect of the embodiment of the present application, a method for processing a speech signal is provided. The method comprises: obtaining a first mixed speech signal and a second mixed speech signal respectively collected by a speech collection device and an audio collection device arranged in a speech interaction device, wherein the speech collection direction of the speech collection device is different from that of the speech collection device; using the environmental interference signal in the second mixed speech signal, performing signal enhancement processing on the target speech signal in the first mixed speech signal to obtain the target speech signal after signal enhancement.

[0006] According to the second aspect of the embodiment of the present application, a device for processing a speech signal is provided. The device comprises: an acquisition module for acquiring a first mixed speech signal and a second mixed speech signal respectively acquired by a speech acquisition device and an audio acquisition device arranged in a speech interaction device, wherein the speech acquisition direction of the speech acquisition device is different from that of the speech acquisition device; and a first processing module for performing signal enhancement processing on a target speech signal in the first mixed speech signal by utilizing an environmental interference signal in the second mixed speech signal, so as to obtain the target speech signal after signal enhancement.

[0007] According to the third aspect of the embodiment of the present application, a sound pickup device is provided. The device includes: a voice acquisition device, arranged on the voice interaction device, for collecting a first mixed voice signal; an audio acquisition device, arranged on the voice interaction device, for collecting a second mixed voice signal, wherein the voice acquisition direction of the voice acquisition device is different from that of the audio acquisition device; a processor, connected to the voice acquisition device and the audio acquisition device, for performing signal enhancement processing on the target voice signal in the first mixed voice signal using the environmental interference signal in the second mixed voice signal, so as to obtain the target voice signal after signal enhancement.

[0008] According to the fourth aspect of the embodiments of the present application, an electronic device is provided, comprising: one or more processors; a computer-readable medium configured to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method for processing speech signals as described in the first aspect of the above embodiments.

[0009] According to a fifth aspect of an embodiment of the present application, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the method for processing a speech signal as described in the first aspect of the above embodiment is implemented.

[0010] Through the technical solution provided in the embodiment of the present application, a first mixed voice signal and a second mixed voice signal respectively collected by a voice collection device and an audio collection device arranged in a voice interaction device are obtained, wherein the voice collection directions of the voice collection device and the audio collection device are different; and then, the environmental interference signal in the second mixed voice signal is used to perform signal enhancement processing on the target voice signal in the first mixed voice signal to obtain the target voice signal after signal enhancement. Compared with other existing methods, the environmental interference signal in the second mixed voice signal can be used to perform signal enhancement processing on the target voice signal in the first mixed voice signal, which can not only effectively reduce the computing resources required for signal enhancement processing, but also effectively improve the voice enhancement effect of the target voice signal. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0012] Figure 1A This is a flowchart of the steps of the method for processing a voice signal in Embodiment 1 of the present application;

[0013] Figure 1B A schematic diagram of the spatial response of a microphone provided according to Embodiment 1 of the present application;

[0014] Figure 2A This is a flowchart of the steps of the method for processing a speech signal in Embodiment 2 of the present application;

[0015] Figure 2B A schematic diagram of a signal separation step provided according to Embodiment 2 of the present application;

[0016] Figure 2C A schematic diagram of a method for processing a speech signal according to Embodiment 2 of the present application;

[0017] Figure 3 This is a structural diagram of a speech signal processing device in Embodiment 3 of the present application;

[0018] Figure 4 This is a structural diagram of a speech signal processing device in Embodiment 4 of the present application;

[0019] Figure 5 This is a schematic diagram of the structure of a speech signal processing device in Embodiment 5 of the present application;

[0020] Figure 6 This is a schematic diagram of the structure of an electronic device in Embodiment 6 of the present application;

[0021] Figure 7 This is the hardware structure of the electronic device in Example 7 of the present application. DETAILED DESCRIPTION

[0022] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only configured to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0023] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0024] Reference Figure 1A , showing a step flow chart of the method for processing a speech signal of Embodiment 1 of the present application.

[0025] Specifically, the method for processing a speech signal in this embodiment includes the following steps:

[0026] In step S101, a first mixed voice signal and a second mixed voice signal respectively collected by a voice collection device and an audio collection device arranged in a voice interaction device are obtained.

[0027] In this embodiment, the voice interaction device can be understood as a device that interacts through voice signals, such as an in-vehicle voice interaction device, a smart speaker, a smart TV, an automatic shopping machine, a food ordering machine, a ticket machine, etc. The number of the voice collection devices is 4, 6 or 8; and / or the number of the audio collection devices is 1 or 2. The voice collection device is specifically a microphone array or a single microphone. When the voice collection device is specifically a microphone array, the microphone array can be an array composed of omnidirectional microphones or an array composed of unidirectional microphones. When the voice collection device is specifically a single microphone, the single microphone can be an omnidirectional microphone or a unidirectional microphone. The audio collection device is specifically a microphone array or a single microphone. When the audio collection device is specifically a microphone array, the microphone array can be an array composed of omnidirectional microphones or an array composed of unidirectional microphones. When the audio collection device is specifically a single microphone, the single microphone can be an omnidirectional microphone or a unidirectional microphone. Among them, the microphone array can be understood as a sound pickup device formed by multiple microphones in a certain topological structure. The microphone can be a physical microphone or a virtual microphone obtained by combining software and a physical microphone. The first mixed voice signal and the second mixed voice signal can both be understood as mixed voice signals of voice signals from different directions. It can be understood that the above description is only exemplary, and the embodiments of the present application do not impose any limitation on this.

[0028] In a specific example, when the arrangement area of ​​the voice collection device in the voice interaction device is different from the arrangement area of ​​the audio collection device in the voice interaction device, and the voice collection device and the audio collection device are both arrays composed of unidirectional microphones, the voice collection direction of the voice collection device is different from that of the audio collection device. For example, when the voice collection device is arranged on the main viewing surface of the voice interaction device, the audio collection device is arranged on the side viewing surface of the voice interaction device, and the voice collection device and the audio collection device are both arrays composed of unidirectional microphones, the voice collection direction of the voice collection device is perpendicular to that of the audio collection device. When the arrangement area of ​​the voice collection device in the voice interaction device is different from that of the audio collection device in the voice interaction device, the voice collection device is an array composed of unidirectional microphones, and the audio collection device is a single unidirectional microphone, the voice collection direction of the voice collection device is different from that of the audio collection device. For example, when the voice collection device is arranged on the main viewing surface of the voice interaction device, the audio collection device is arranged on the side viewing surface of the voice interaction device, the voice collection device is an array composed of unidirectional microphones, and the audio collection device is a single unidirectional microphone, the voice collection direction of the voice collection device is perpendicular to the voice collection direction of the audio collection device. When the arrangement area of ​​the voice collection device in the voice interaction device is different from the arrangement area of ​​the audio collection device in the voice interaction device, the voice collection device is a single unidirectional microphone, and the audio collection device is an array composed of unidirectional microphones, the voice collection direction of the voice collection device is different from that of the audio collection device. For example, when the voice collection device is arranged on the main viewing surface of the voice interaction device, the audio collection device is arranged on the side viewing surface of the voice interaction device, the voice collection device is a single unidirectional microphone, and the audio collection device is an array composed of unidirectional microphones, the voice collection direction of the voice collection device is perpendicular to the voice collection direction of the audio collection device. It can be understood that the above description is only exemplary, and the embodiments of the present application do not impose any limitation on this.

[0029] In a specific example, when the voice collection device is arranged on the front surface of the voice interaction device, and the voice collection device is an array of unidirectional microphones or a single unidirectional microphone, by Figure 1B As shown in the schematic diagram of the spatial response of the microphone, the voice collection device collects the sound source in front of the voice interaction device. When the audio collection device is arranged on the rear surface of the voice interaction device, and the audio collection device is an array composed of unidirectional microphones or a single unidirectional microphone, by Figure 1B As can be seen from the schematic diagram of the spatial response of the microphone shown, the audio acquisition device collects the sound source behind the voice interaction device. In this way, the voice acquisition direction of the voice acquisition device is opposite to the voice acquisition direction of the audio acquisition device. Thereby, the difference between the first mixed voice signal collected by the voice acquisition device and the second mixed voice signal collected by the audio acquisition device can be used to perform signal enhancement processing, which can not only effectively reduce the computing resources required for signal enhancement processing, but also effectively improve the voice enhancement effect of the target voice signal. It can be understood that the above description is only exemplary, and the embodiments of the present application do not impose any limitation on this.

[0030] In step S102, the target speech signal in the first mixed speech signal is enhanced by using the environmental interference signal in the second mixed speech signal to obtain the target speech signal after signal enhancement.

[0031] In this embodiment, the environmental interference signal can be understood as a combination of a stationary noise signal, an interference signal and an echo residual signal. When the audio acquisition device collects the sound source behind the voice interaction device, the environmental interference signal is specifically a combination of a stationary noise signal, a rear interference signal and an echo residual signal, and the combination ratio is 1:1:1 by default. The default combination ratio can also be adjusted according to actual conditions. For example, the default combination ratio can be adjusted to 0.3:0.6:0.1, etc. By modeling the environmental interference signal as a combination of a stationary noise signal, an interference signal and an echo residual signal, it is not only convenient to calculate the signal power of the environmental interference signal, but also more noise in the target voice signal can be eliminated. The target voice signal can be understood as a voice signal for the user to interact with the voice interaction device. It can be understood that the above description is only exemplary and the embodiments of the present application do not impose any restrictions on this.

[0032] In some optional embodiments, when performing signal enhancement processing on the target voice signal in the first mixed voice signal using the environmental interference signal in the second mixed voice signal, the target voice signal and the environmental interference signal are respectively subjected to frame processing to obtain a first voice data frame of the target voice signal and a second voice data frame of the environmental interference signal; based on the signal power of the first voice data frame and the second voice data frame, the target voice signal is subjected to signal enhancement processing to obtain the target voice signal after signal enhancement. Wherein, when calculating the signal power of the second voice data frame, the signal power of the second voice data frame is calculated by modeling the environmental interference signal as a combination of a stationary noise signal, an interference signal, and an echo residual signal. It can be understood that the above description is only exemplary and the embodiments of the present application do not impose any limitation on this.

[0033] In some optional embodiments, when the target voice signal is enhanced based on the signal power of the first voice data frame and the second voice data frame, a first probability that the signal power of the first voice data frame is the noise signal power is determined based on the signal power of the second voice data frame and the amplitude of the second voice data frame within the frequency band; a second probability that the signal power of the first voice data frame is the voice signal power is determined based on the signal power of the first voice data frame, the amplitude of the first voice data frame within the frequency band, and the amplitude of the second voice data frame within the frequency band; based on the first probability and the second probability, the target voice signal is enhanced to obtain the target voice signal after signal enhancement. It can be understood that the above description is only exemplary and the embodiments of the present application do not impose any limitation on this.

[0034] In a specific example, when determining the first probability that the signal power of the first voice data frame is the noise signal power based on the signal power of the second voice data frame and the amplitude of the second voice data frame in the frequency band, the third probability that the signal power of the first voice data frame in the frequency band is the noise signal power is determined based on the average of the signal power of the second voice data frame in the frequency band and the amplitude of the second voice data frame in the frequency band to obtain the target voice signal after signal enhancement. Wherein, when calculating the signal power of the second voice data frame in the frequency band, the signal power of the second voice data frame in the frequency band is calculated by modeling the environmental interference signal as a combination of a stationary noise signal, an interference signal and an echo residual signal. It can be understood that the above description is only exemplary and the embodiments of the present application do not impose any limitation on this.

[0035] In a specific example, when determining the third probability that the signal power of the first voice data frame in the frequency band is the noise signal power, the third probability that the signal power of the first voice data frame in the frequency band is the noise signal power can be calculated by the following formula 1:

[0036]

[0037] Wherein, Y(k, l) represents the signal power of the first speech data frame l of the target speech signal in frequency band k, p(Y(k, l)|H0(k, l)) represents the third probability that the signal power of the first speech data frame l of the target speech signal in frequency band k is the noise signal power, λ d(k, l) represents the mean value of the amplitude of the second voice data frame l of the environmental interference signal in the frequency band k, and Y′(k, l) represents the signal power of the second voice data frame l of the environmental interference signal in the frequency band k. Wherein, Y′(k, l)=f(noise, interf, res-echo) represents that the signal power of the second voice data frame l of the environmental interference signal in the frequency band k is a combination of the signal power of the third voice data frame l of the stationary noise signal (noise) in the frequency band k, the signal power of the fourth voice data frame l of the backward interference signal (interf) in the frequency band k, and the signal power of the fifth voice data frame l of the echo residual signal (res-echo) in the frequency band k, and the combination ratio is 1:1:1 by default, and the default combination ratio can also be adjusted according to actual conditions, for example, the default combination ratio can be adjusted to 0.3:0.6:0.1, etc. It can be understood that the above description is only exemplary, and the embodiments of the present application do not impose any limitation on this.

[0038] In a specific example, when determining the second probability that the signal power of the first voice data frame is the voice signal power based on the signal power of the first voice data frame, the amplitude of the first voice data frame in the frequency band, and the amplitude of the second voice data frame in the frequency band, determine the fourth probability that the signal power of the first voice data frame in the frequency band is the voice signal power based on the signal power of the first voice data frame in the frequency band, the average of the amplitude of the first voice data frame in the frequency band, and the average of the amplitude of the second voice data frame in the frequency band, so as to obtain the target voice signal after signal enhancement. It can be understood that the above description is only exemplary and the embodiments of the present application do not impose any limitation on this.

[0039] In a specific example, when determining the fourth probability that the signal power of the first voice data frame in the frequency band is the voice signal power, the fourth probability that the signal power of the first voice data frame in the frequency band is the voice signal power can be calculated by the following formula 2:

[0040]

[0041] Wherein, Y(k, l) represents the signal power of the first speech data frame l of the target speech signal in frequency band k, p(Y(k, l)|H1(k, l)) represents the fourth probability that the signal power of the first speech data frame l of the target speech signal in frequency band k is the speech signal power, λ d (k, l) represents the mean value of the amplitude of the second voice data frame l of the environmental interference signal in frequency band k, λ x(k, l) represents the mean value of the amplitude of the first speech data frame l of the target speech signal in the frequency band k. It can be understood that the above description is only exemplary and the embodiment of the present application does not impose any limitation on this.

[0042] In a specific example, when the target speech signal is subjected to signal enhancement processing based on the first probability and the second probability to obtain the target speech signal after signal enhancement, the weight value of the signal power of the first speech data frame within the frequency band is determined based on the third probability and the fourth probability; the power spectrum of the target speech signal after signal enhancement is determined based on the signal power of the first speech data frame within the frequency band and the weight value; the target speech signal after signal enhancement is determined based on the power spectrum of the target speech signal after signal enhancement. It can be understood that the above description is only exemplary and the embodiments of the present application do not impose any limitation on this.

[0043] In a specific example, when determining the weight value of the signal power of the first voice data frame within the frequency band based on the third probability and the fourth probability, if the third probability is greater than the fourth probability, then based on the third probability, the weight value of the signal power of the first voice data frame within the frequency band is determined; if the third probability is less than or equal to the fourth probability, then the value of the fourth probability is determined as the weight value of the signal power of the first voice data frame within the frequency band. Wherein, when determining the weight value of the signal power of the first voice data frame within the frequency band based on the third probability, the difference between the value 1 and the third probability is determined as the weight value of the signal power of the first voice data frame within the frequency band. It can be understood that the above description is only exemplary and the embodiments of the present application do not impose any limitation on this.

[0044] In a specific example, when determining the power spectrum of the target voice signal after signal enhancement based on the signal power of the first voice data frame in the frequency band and the weight value, the signal power of the first voice data frame in the frequency band is multiplied by the weight value to obtain the signal power of the corresponding voice data frame of the target voice signal after signal enhancement in the frequency band; based on the signal power of the corresponding voice data frame of the target voice signal after signal enhancement in the frequency band, the power spectrum of the target voice signal after signal enhancement is determined. It can be understood that the above description is only exemplary and the embodiments of the present application do not impose any limitation on this.

[0045] Through the voice signal processing method provided in the embodiment of the present application, a first mixed voice signal and a second mixed voice signal respectively collected by a voice collection device and an audio collection device arranged in a voice interaction device are obtained, wherein the voice collection directions of the voice collection device and the audio collection device are different; and then, the environmental interference signal in the second mixed voice signal is used to perform signal enhancement processing on the target voice signal in the first mixed voice signal to obtain the target voice signal after signal enhancement. Compared with other existing methods, the environmental interference signal in the second mixed voice signal can be used to perform signal enhancement processing on the target voice signal in the first mixed voice signal, which can not only effectively reduce the computing resources required for signal enhancement processing, but also effectively improve the voice enhancement effect of the target voice signal.

[0046] The voice signal processing method of this embodiment can be executed by any appropriate device with data processing capabilities, including but not limited to: cameras, terminals, mobile terminals, PCs, servers, vehicle-mounted devices, entertainment equipment, advertising equipment, personal digital assistants (PDAs), tablet computers, laptop computers, handheld game consoles, smart glasses, smart watches, wearable devices, virtual display devices or display enhancement devices (such as Google Glass, Oculus Rift, Hololens, Gear VR), etc.

[0047] Reference Figure 2A , showing a step flow chart of the method for processing a speech signal of Embodiment 2 of the present application.

[0048] Specifically, the method for processing a speech signal in this embodiment includes the following steps:

[0049] In step S201, a first mixed voice signal and a second mixed voice signal respectively collected by a voice collection device and an audio collection device arranged in a voice interaction device are obtained.

[0050] Since step S201 is similar to the above step S101, it will not be described again here.

[0051] In step S202, speech signal separation is performed on the first mixed speech signal, and speech signal separation is performed on the second mixed speech signal to obtain the target speech signal and the environmental interference signal.

[0052] In this embodiment, the voice collection device and the audio collection device are both microphone arrays. Both the first mixed voice signal and the second mixed voice signal can be understood as mixed voice signals of voice signals from different directions. Based on this, beams can be formed in different directions, and voice signals within the beams can be picked up, noise outside the beams can be eliminated, and the required voice signals can be separated to achieve the purpose of voice signal enhancement. It can be understood that the above description is only exemplary, and the embodiments of the present application do not impose any limitations on this.

[0053] In some optional embodiments, performing voice signal separation on the first mixed voice signal and performing voice signal separation on the second mixed voice signal includes: determining first azimuth information of the target voice signal and second azimuth information of the environmental interference signal; performing voice signal separation on the first mixed voice signal based on the first azimuth information, and performing voice signal separation on the second mixed voice signal based on the second azimuth information, so as to obtain the target voice signal and the environmental interference signal. It can be understood that the above description is only exemplary, and the embodiments of the present application do not impose any limitation on this.

[0054] In a specific example, Figure 2BAs shown, after obtaining the first mixed voice signal collected by the voice collection device, the first mixed voice signal is first preprocessed, and after the preprocessing, the phase-transformed weighted generalized cross-correlation algorithm is used to obtain the delay difference of each voice signal relative to the reference voice signal, and finally, based on the calculated delay difference, a beam is formed by a delayed accumulation beamforming algorithm. Among them, the preprocessing includes framing, silence detection, and Hamming window addition. Since the voice signal is a non-steady-state signal, its characteristics change with time, but in a very short period of time, it can be considered that the voice signal has a relatively stable characteristic, that is, the voice signal has short-term stability. Therefore, when processing the first mixed voice signal, the first mixed voice signal is usually framed according to a short period of time. The purpose of silence detection is to eliminate the silence frame in the voice signal. Silence detection can not only eliminate the influence of the silence frame on the voice recognition of the previous and next frames, but also reduce unnecessary calculations and improve calculation efficiency. In addition, since the framing processing of the first mixed voice signal is equivalent to intercepting the time domain voice signal with a rectangular window, and since the time domain product is equivalent to the frequency domain convolution, in terms of the frequency domain, the rectangular window interception will cause the spectrum leakage of the voice signal. Therefore, it is necessary to add a Hamming window to alleviate the spectrum leakage. After preprocessing the first mixed voice signal, the voice enhancement based on the microphone array needs to locate the sound source to estimate the position or direction of the desired sound source, and then use the enhancement algorithm to obtain the enhanced voice signal. In an example, the sound source positioning based on the time delay difference (Time Difference of Arrival, TDOA) estimation method is used as an example for explanation. Commonly used time delay difference estimation methods include: Generalized Cross Correlation (Generalized Cross Correlation, GCC) method, Linear Regression (Linear Regression, LR) method, Least Mean Square (LMS) adaptive method, etc. The GCC method is used as an example for explanation below.

[0055] The GCC method first calculates the cross-power spectrum of a pair of microphone signals, then multiplies them by the corresponding weights, and finally performs an inverse Fourier transform to obtain the cross-correlation function of the signal. The time corresponding to the peak value is the arrival delay difference τ of this pair of microphones. i. The performance of the GCC method depends on the selected weight function, among which the most representative ones are maximum likelihood (ML) weighting and phase transform (PHAT) weighting. Ideally, maximum likelihood weighting can achieve optimal estimation, but maximum likelihood weighting requires the power spectrum of the sound source signal and noise to be known. In practical applications, this condition is difficult to meet. Phase transform weighting abandons the need for the power spectrum of the sound source signal and noise. By normalizing the cross-power spectrum function, the cross-correlation function is sharpened, making the peak prominent and better suppressing the interference of cross-correlation pseudo-peaks. In addition, in a reverberant environment, phase transform weighting has strong robustness. For an ideal free sound field environment, when the autocorrelation function of the sound source signal is at its maximum value, the cross-correlation function is also at its maximum value. Therefore, when calculating, you only need to find the maximum value of the cross-correlation function, and the corresponding time is the delay difference. In a reverberation environment, due to the superposition of countless reverberation signals, the function may have multiple peak points. To address this problem, the phase-shift weighted generalized cross-correlation GCC-PHAT algorithm can be used. The GCC-PHAT algorithm does not directly calculate the cross-correlation function in the time domain, but uses the correspondence between the cross-correlation function of the signal in the time domain and the cross-power spectrum function of the signal in the frequency domain. It first calculates the cross-power spectrum density between the two speech signals, then performs PHAT weighting, and finally obtains the generalized cross-correlation function through inverse Fourier transform, and then obtains the corresponding delay difference. The Delay-Sum Beamforming (DSB) algorithm uses the delay difference τ obtained by GCC-PHAT i , firstly, delay compensation is performed on the voice signals on each microphone channel so that the voice signals received by each microphone are aligned on the time axis, and then uniformly weighted and summed to obtain the output signal. In the process of beamforming, the azimuth information of each voice signal can be determined according to the relationship between the energy of each azimuth beam and the phase difference. In one example, after determining the azimuth information of the target voice signal in the first mixed voice signal relative to the microphone array, the target voice signal can be separated from the acquired first mixed voice signal based on the determined azimuth information. Similarly, after determining the azimuth information of the environmental interference signal in the second mixed voice signal relative to the microphone array, the environmental interference signal can be separated from the acquired second mixed voice signal based on the determined azimuth information. It can be understood that the above description is only exemplary, and the embodiments of the present application do not impose any limitation on this.

[0056] In step S203, the target speech signal in the first mixed speech signal is subjected to signal enhancement processing by utilizing the environmental interference signal in the second mixed speech signal to obtain the target speech signal after signal enhancement.

[0057] Since step S203 is similar to the above step S102, it will not be described again here.

[0058] In a specific example, Figure 2C As shown, the speech signal processing scheme provided in this embodiment includes a forward microphone array picking up sound on the front surface of the speech interaction device, a rearward microphone array picking up sound on the rear surface of the speech interaction device, speech processing of the forward microphone array (step S202), speech processing of the rearward microphone array (step S203), and dual-channel enhancement processing (step S102 or step S203). Specifically, Figure 2C The white circle in the figure represents the front microphone array, and the gray circle represents the rear microphone array. The front microphone array is composed of unidirectional microphones, which are physically directional spatially responded (such as Figure 1BAs shown in the figure, it can be seen that the sound source in front of the voice interaction device is mainly obtained. The rear microphone array is composed of unidirectional microphones, which mainly obtain the sound source behind the voice interaction device. Without loss of generality, omnidirectional microphones can also be used to implement it. Among them, the speech processing of the forward microphone array adopts signal processing technologies such as beamforming to further enhance the strength of the forward sound source (target voice signal). Beamforming (speech enhancement) refers to enhancing the target voice signal and suppressing interfering sound sources and noise. Speech enhancement based on a single microphone is not ideal, and it is difficult to obtain a significant performance improvement. In this case, by constructing a microphone array to suppress reverberation and noise, a higher quality target voice signal can be obtained. The microphone array forms a beam in the target direction, picks up the signal in the beam, eliminates the noise outside the beam, and separates the target voice signal to achieve the purpose of speech enhancement. The array processing algorithm with the lowest complexity is adopted: the delayed accumulation beamforming algorithm. The delay accumulation beamforming algorithm uses the delay difference obtained by the GCC-PHAT algorithm to first compensate the delay of the signals on each microphone channel so that the signals received by each microphone are aligned on the time axis, and then uniformly weighted and summed to obtain the output signal. Among them, the speech processing of the backward microphone array is specifically: the backward microphone array or the backward single microphone uses the unidirectional characteristics to obtain the sound source (mainly environmental noise and interference) behind the voice interaction device. If a microphone array is used, a method similar to the speech processing of the forward microphone array can be used for processing to obtain the separated environmental interference signal. If it is a single microphone, it can be processed without processing or simply single-microphone noise reduction. Among them, the dual-channel enhancement processing is specifically: using the separated target voice signal and environmental interference signal, the dual-channel voice noise reduction algorithm (the specific implementation method of step S102 or step S203) can further improve the voice enhancement effect, and the target voice signal after signal enhancement will be sent to the subsequent VAD (voice activity detector, voice endpoint detection) engine or voice wake-up engine (KWS, Keywordspooting) or automatic speech recognition engine (ASR, automatic speech recognition). It should be understood that the above description is merely exemplary and the embodiments of the present application do not impose any limitation thereto.

[0059] In practical applications, by using the difference in sound pickup between the forward microphone array and the backward microphone array and the speech processing solution provided in this embodiment, high-quality speech enhancement can be achieved with low computational effort in a noisy environment. Tests show that the speech recognition error rate is the same as that of a high-complexity algorithm, but the computational effort is reduced by more than 10 times.

[0060] Through the voice signal processing method provided in the embodiment of the present application, a first mixed voice signal and a second mixed voice signal respectively collected by a voice collection device and an audio collection device arranged in a voice interaction device are obtained, wherein the voice collection directions of the voice collection device and the audio collection device are different; and voice signal separation is performed on the first mixed voice signal, and voice signal separation is performed on the second mixed voice signal to obtain a target voice signal and an environmental interference signal, and then the environmental interference signal in the second mixed voice signal is used to perform signal enhancement processing on the target voice signal in the first mixed voice signal to obtain a target voice signal after signal enhancement. Compared with other existing methods, the environmental interference signal in the second mixed voice signal can be used to perform signal enhancement processing on the target voice signal in the first mixed voice signal, which can not only effectively reduce the computing resources required for signal enhancement processing, but also effectively improve the voice enhancement effect of the target voice signal.

[0061] The voice signal processing method of this embodiment can be executed by any appropriate device with data processing capabilities, including but not limited to: cameras, terminals, mobile terminals, PCs, servers, vehicle-mounted devices, entertainment equipment, advertising equipment, personal digital assistants (PDAs), tablet computers, laptop computers, handheld game consoles, smart glasses, smart watches, wearable devices, virtual display devices or display enhancement devices (such as Google Glass, Oculus Rift, Hololens, Gear VR), etc.

[0062] Reference Figure 3 , showing a schematic diagram of the structure of the speech signal processing device in Example 3 of the present application.

[0063] The speech signal processing device of this embodiment includes: an acquisition module 301, which is used to acquire a first mixed speech signal and a second mixed speech signal respectively collected by a speech collection device and an audio collection device arranged in a speech interaction device, wherein the speech collection directions of the speech collection device and the audio collection device are different; a first processing module 302, which is used to perform signal enhancement processing on a target speech signal in the first mixed speech signal by using an environmental interference signal in the second mixed speech signal, so as to obtain the target speech signal after signal enhancement.

[0064] The speech signal processing device of this embodiment is used to implement the corresponding speech signal processing methods in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be described in detail here.

[0065] Reference Figure 4 , showing a schematic diagram of the structure of the speech signal processing device in the fourth embodiment of the present application.

[0066] The speech signal processing device of this embodiment includes: an acquisition module 401, which is used to acquire a first mixed speech signal and a second mixed speech signal respectively collected by a speech collection device and an audio collection device arranged in a speech interaction device, wherein the speech collection directions of the speech collection device and the audio collection device are different; a first processing module 403, which is used to perform signal enhancement processing on a target speech signal in the first mixed speech signal by using an environmental interference signal in the second mixed speech signal, so as to obtain the target speech signal after signal enhancement.

[0067] Optionally, before the first processing module 403, the device further includes: a separation module 402, which is used to perform speech signal separation on the first mixed speech signal and to perform speech signal separation on the second mixed speech signal to obtain the target speech signal and the environmental interference signal.

[0068] Optionally, the separation module 402 is specifically used to: determine the first direction information of the target speech signal and the second direction information of the environmental interference signal; perform speech signal separation on the first mixed speech signal based on the first direction information, and perform speech signal separation on the second mixed speech signal based on the second direction information to obtain the target speech signal and the environmental interference signal.

[0069] The speech signal processing device of this embodiment is used to implement the corresponding speech signal processing methods in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be described in detail here.

[0070] Reference Figure 5 , showing a schematic diagram of the structure of the speech signal processing device in Example 5 of the present application.

[0071] The speech signal processing device of this embodiment includes: an acquisition module 501, which is used to acquire a first mixed speech signal and a second mixed speech signal respectively collected by a speech collection device and an audio collection device arranged in a speech interaction device, wherein the speech collection directions of the speech collection device and the audio collection device are different; a first processing module 502, which is used to perform signal enhancement processing on a target speech signal in the first mixed speech signal by using an environmental interference signal in the second mixed speech signal, so as to obtain the target speech signal after signal enhancement.

[0072] Optionally, the first processing module 502 includes: a second processing module 5021, used to perform frame processing on the target voice signal and the environmental interference signal respectively to obtain a first voice data frame of the target voice signal and a second voice data frame of the environmental interference signal; a third processing module 5022, used to perform signal enhancement processing on the target voice signal based on the signal power of the first voice data frame and the second voice data frame to obtain the target voice signal after signal enhancement.

[0073] Optionally, the third processing module 5022 includes: a first determination module 5023, used to determine a first probability that the signal power of the first voice data frame is the noise signal power based on the signal power of the second voice data frame and the amplitude of the second voice data frame within the frequency band; a second determination module 5024, used to determine a second probability that the signal power of the first voice data frame is the voice signal power based on the signal power of the first voice data frame, the amplitude of the first voice data frame within the frequency band and the amplitude of the second voice data frame within the frequency band; and a fourth processing module 5025, used to perform signal enhancement processing on the target voice signal based on the first probability and the second probability to obtain the target voice signal after signal enhancement.

[0074] Optionally, the first determination module 5023 is specifically used to determine the third probability that the signal power of the first voice data frame in the frequency band is the noise signal power based on the signal power of the second voice data frame in the frequency band and the average of the amplitude of the second voice data frame in the frequency band, so as to obtain the target voice signal after signal enhancement.

[0075] Optionally, the second determination module 5024 is specifically used to determine the fourth probability that the signal power of the first voice data frame within the frequency band is the voice signal power based on the signal power of the first voice data frame within the frequency band, the average of the amplitudes of the first voice data frame within the frequency band, and the average of the amplitudes of the second voice data frame within the frequency band, so as to obtain the target voice signal after signal enhancement.

[0076] Optionally, the fourth processing module 5025 includes: a third determination module 5026, used to determine the weight value of the signal power of the first voice data frame within the frequency band based on the third probability and the fourth probability; a fourth determination module 5027, used to determine the power spectrum of the target voice signal after signal enhancement based on the signal power of the first voice data frame within the frequency band and the weight value; and a fifth determination module 5028, used to determine the target voice signal after signal enhancement based on the power spectrum of the target voice signal after signal enhancement.

[0077] Optionally, the third determination module 5026 is specifically used to: if the third probability is greater than the fourth probability, determine the weight value of the signal power of the first voice data frame within the frequency band based on the third probability; if the third probability is less than or equal to the fourth probability, determine the numerical value of the fourth probability as the weight value of the signal power of the first voice data frame within the frequency band.

[0078] Optionally, the voice collection device and the audio collection device have voice collection directions opposite to each other.

[0079] The speech signal processing device of this embodiment is used to implement the corresponding speech signal processing methods in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be described in detail here.

[0080] The embodiment of the present application also provides a sound pickup device. The device includes: a voice collection device, arranged on the voice interaction device, for collecting a first mixed voice signal; an audio collection device, arranged on the voice interaction device, for collecting a second mixed voice signal, wherein the voice collection direction of the voice collection device is different from that of the audio collection device; a processor, connected to the voice collection device and the audio collection device, for performing signal enhancement processing on the target voice signal in the first mixed voice signal using the environmental interference signal in the second mixed voice signal, so as to obtain the target voice signal after signal enhancement.

[0081] Optionally, the voice collection device and the audio collection device are both microphone arrays.

[0082] Optionally, the voice collection device and the audio collection device have voice collection directions opposite to each other.

[0083] The sound pickup device of this embodiment is used to implement the corresponding voice signal processing methods in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be described in detail here.

[0084] Figure 6 This is a schematic diagram of the structure of an electronic device in Embodiment 6 of the present application; the electronic device may include:

[0085] One or more processors 601;

[0086] The computer readable medium 602 may be configured to store one or more programs,

[0087] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for processing a speech signal as described in the above-mentioned embodiment 1 or embodiment 2.

[0088] Figure 7 The hardware structure of the electronic device in the seventh embodiment of the present application; Figure 7 As shown, the hardware structure of the electronic device may include: a processor 701, a communication interface 702, a computer readable medium 703 and a communication bus 704;

[0089] The processor 701, the communication interface 702, and the computer readable medium 703 communicate with each other via a communication bus 704;

[0090] Optionally, the communication interface 702 may be an interface of a communication module, such as an interface of a GSM module;

[0091] Among them, the processor 701 can be specifically configured to: obtain a first mixed voice signal and a second mixed voice signal respectively collected by a voice collection device and an audio collection device arranged in the voice interaction device, wherein the voice collection directions of the voice collection device and the audio collection device are different; use the environmental interference signal in the second mixed voice signal to perform signal enhancement processing on the target voice signal in the first mixed voice signal to obtain the target voice signal after signal enhancement.

[0092] The processor 701 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0093] The computer readable medium 703 can be, but is not limited to, a random access memory medium (RAM), a read-only memory medium (ROM), a programmable read-only memory medium (PROM), an erasable programmable read-only memory medium (EPROM), an electrically erasable read-only memory medium (Electric Erasable Programmable Read-Only Memory, EEPROM), etc.

[0094] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code configured to execute the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through a communication part, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), the above functions defined in the method of the present application are executed. It should be noted that the computer-readable medium described in the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access storage medium (RAM), a read-only storage medium (ROM), an erasable programmable read-only storage medium (EPROM or flash memory), an optical fiber, a portable compact disk read-only storage medium (CD-ROM), an optical storage medium, a magnetic storage medium, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, an apparatus or a device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program configured to be used by or in combination with an instruction execution system, an apparatus or a device. The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0095] Computer program code configured to perform the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0096] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of the code contains one or more executable instructions configured to implement the specified logical function. There are specific sequential relationships in the above-mentioned specific embodiments, but these sequential relationships are only exemplary. When the specific implementation is implemented, these steps may be less, more, or the execution order may be adjusted. That is, in some implementations as replacements, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0097] The modules involved in the embodiments of the present application may be implemented by software or hardware. The modules described may also be arranged in a processor, for example, they may be described as: a processor includes an acquisition module and a first processing module. The names of these modules do not constitute a limitation on the modules themselves in some cases. For example, the acquisition module may also be described as "a module for acquiring a first mixed voice signal and a second mixed voice signal respectively collected by a voice acquisition device and an audio acquisition device arranged in a voice interaction device".

[0098] As another aspect, the present application further provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the method for processing a speech signal as described in the above-mentioned embodiment 1 or embodiment 2.

[0099] As another aspect, the present application also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by the device, the device: obtains a first mixed voice signal and a second mixed voice signal respectively collected by a voice collection device and an audio collection device arranged in a voice interaction device, wherein the voice collection directions of the voice collection device and the audio collection device are different; utilizes the environmental interference signal in the second mixed voice signal to perform signal enhancement processing on the target voice signal in the first mixed voice signal to obtain the target voice signal after signal enhancement.

[0100] The expressions "first", "second", "the first" or "the second" used in various embodiments of the present disclosure may modify various components regardless of order and / or importance, but these expressions do not limit the corresponding components. The above expressions are only configured for the purpose of distinguishing an element from other elements. For example, a first user device and a second user device represent different user devices, although both are user devices. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element without departing from the scope of the present disclosure.

[0101] When one element (e.g., a first element) is referred to as being "(operably or communicatively) coupled" or "(operably or communicatively) coupled to" or "connected to" another element (e.g., a second element), it is understood that the one element is directly connected to the other element or the one element is indirectly connected to the other element via yet another element (e.g., a third element). Conversely, it is understood that when an element (e.g., a first element) is referred to as being "directly connected" or "directly coupled" to another element (the second element), no element (e.g., a third element) is interposed between the two.

[0102] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A method for processing a speech signal, characterized in that: The method comprises: Acquire a first mixed voice signal and a second mixed voice signal respectively collected by a voice collection device and an audio collection device arranged in the voice interaction device, wherein the voice collection direction of the voice collection device is different from that of the audio collection device, and the voice collection device and the audio collection device are both microphone arrays; Using the environmental interference signal in the second mixed speech signal, performing signal enhancement processing on the target speech signal in the first mixed speech signal to obtain the target speech signal after signal enhancement; The step of performing signal enhancement processing on the target speech signal in the first mixed speech signal by utilizing the environmental interference signal in the second mixed speech signal includes: Performing frame processing on the target speech signal and the environmental interference signal respectively to obtain a first speech data frame of the target speech signal and a second speech data frame of the environmental interference signal; Based on the signal power of the second voice data frame and the amplitude of the second voice data frame in a frequency band, determine a first probability that the signal power of the first voice data frame is the noise signal power; based on the signal power of the first voice data frame, the amplitude of the first voice data frame in the frequency band and the amplitude of the second voice data frame in the frequency band, determine a second probability that the signal power of the first voice data frame is the voice signal power; based on the first probability and the second probability, perform signal enhancement processing on the target voice signal to obtain the target voice signal after signal enhancement, wherein the signal power of the second voice data frame is determined by modeling the environmental interference signal as a combination of a stationary noise signal, an interference signal and an echo residual signal.

2. The method according to claim 1, characterized in that: The method further comprises: The first mixed speech signal is subjected to speech signal separation, and the second mixed speech signal is subjected to speech signal separation to obtain the target speech signal and the environmental interference signal.

3. The method according to claim 2, characterized in that The performing speech signal separation on the first mixed speech signal and the performing speech signal separation on the second mixed speech signal include: Determining first position information of the target speech signal and second position information of the environmental interference signal; Based on the first orientation information, speech signal separation is performed on the first mixed speech signal, and based on the second orientation information, speech signal separation is performed on the second mixed speech signal to obtain the target speech signal and the environmental interference signal.

4. The method according to claim 1, characterized in that: The determining, based on the signal power of the second voice data frame and the amplitude of the second voice data frame in a frequency band, a first probability that the signal power of the first voice data frame is the noise signal power comprises: Based on the signal power of the second voice data frame in the frequency band and the average of the amplitude of the second voice data frame in the frequency band, determine the third probability that the signal power of the first voice data frame in the frequency band is the noise signal power to obtain the target voice signal after signal enhancement.

5. The method according to claim 4, characterized in that The determining, based on the signal power of the first voice data frame, the amplitude of the first voice data frame within the frequency band, and the amplitude of the second voice data frame within the frequency band, a second probability that the signal power of the first voice data frame is the voice signal power comprises: Based on the signal power of the first voice data frame in the frequency band, the average of the amplitudes of the first voice data frame in the frequency band, and the average of the amplitudes of the second voice data frame in the frequency band, determine the fourth probability that the signal power of the first voice data frame in the frequency band is the voice signal power to obtain the target voice signal after signal enhancement.

6. The method according to claim 5, characterized in that The performing signal enhancement processing on the target speech signal based on the first probability and the second probability to obtain the target speech signal after signal enhancement, comprises: Determine a weight value of the signal power of the first voice data frame within the frequency band based on the third probability and the fourth probability; Determining a power spectrum of the target speech signal after signal enhancement based on the signal power of the first speech data frame in the frequency band and the weight value; Based on the power spectrum of the target speech signal after signal enhancement, the target speech signal after signal enhancement is determined.

7. The method according to claim 6, characterized in that The step of determining a weight value of a signal power of the first voice data frame within the frequency band based on the third probability and the fourth probability includes: If the third probability is greater than the fourth probability, determining a weight value of the signal power of the first voice data frame within the frequency band based on the third probability; If the third probability is less than or equal to the fourth probability, the value of the fourth probability is determined as the weight value of the signal power of the first voice data frame in the frequency band.

8. The method according to any one of claims 1 to 7, characterized in that: The voice collecting device and the audio collecting device have voice collecting directions opposite to each other.

9. The method according to any one of claims 1 to 7, characterized in that: The number of the voice collection devices is 4, 6 or 8; and / or the number of the audio collection devices is 1 or 2.

10. A speech signal processing device, characterized in that: The device comprises: an acquisition module, used for acquiring a first mixed voice signal and a second mixed voice signal respectively collected by a voice collection device and an audio collection device arranged in the voice interaction device, wherein the voice collection direction of the voice collection device is different from that of the audio collection device, and the voice collection device and the audio collection device are both microphone arrays; A first processing module, configured to perform signal enhancement processing on a target speech signal in the first mixed speech signal by utilizing an environmental interference signal in the second mixed speech signal, so as to obtain the target speech signal after signal enhancement; Wherein, the first processing module includes: A second processing module, used for performing frame processing on the target voice signal and the environmental interference signal respectively, so as to obtain a first voice data frame of the target voice signal and a second voice data frame of the environmental interference signal; The third processing module is used to determine a first probability that the signal power of the first voice data frame is the noise signal power based on the signal power of the second voice data frame and the amplitude of the second voice data frame in a frequency band, determine a second probability that the signal power of the first voice data frame is the voice signal power based on the signal power of the first voice data frame, the amplitude of the first voice data frame in the frequency band and the amplitude of the second voice data frame in the frequency band, and perform signal enhancement processing on the target voice signal based on the first probability and the second probability to obtain the target voice signal after signal enhancement, wherein the signal power of the second voice data frame is determined by modeling the environmental interference signal as a combination of a stationary noise signal, an interference signal and an echo residual signal.

11. A sound pickup device, characterized in that: The device comprises: A voice collection device, arranged on the voice interaction device, for collecting a first mixed voice signal; an audio collection device, arranged on the voice interaction device, for collecting a second mixed voice signal, wherein the voice collection device and the audio collection device have different voice collection directions, and both the voice collection device and the audio collection device are microphone arrays; a processor connected to the voice collection device and the audio collection device, and configured to perform signal enhancement processing on the target voice signal in the first mixed voice signal by utilizing the environmental interference signal in the second mixed voice signal, so as to obtain the target voice signal after signal enhancement; The processor is further used to perform frame processing on the target speech signal and the environmental interference signal respectively to obtain a first speech data frame of the target speech signal and a second speech data frame of the environmental interference signal, and determine a first probability that the signal power of the first speech data frame is the noise signal power based on the signal power of the second speech data frame and the amplitude of the second speech data frame within the frequency band, and determine a second probability that the signal power of the first speech data frame is the speech signal power based on the signal power of the first speech data frame, the amplitude of the first speech data frame within the frequency band and the amplitude of the second speech data frame within the frequency band, and perform signal enhancement processing on the target speech signal based on the first probability and the second probability to obtain the target speech signal after signal enhancement, wherein the signal power of the second speech data frame is determined by modeling the environmental interference signal as a combination of a stationary noise signal, an interference signal and an echo residual signal.

12. The device according to claim 11, characterized in that The voice collecting device and the audio collecting device have voice collecting directions opposite to each other.