A speech enhancement method and device based on vector speech sensor array
By using mutually orthogonal vector speech sensors and omnidirectional microphones to perform voice signal processing in miniaturized devices, the noise interference problem is solved, and efficient voice enhancement effect is achieved in miniaturized devices.
Patent Information
- Application Number
- CN202111055145.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-09-09
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2041-09-09
AI Technical Summary
The prior art is difficult to achieve effective voice enhancement in miniaturized devices, especially in environments with severe noise interference. Traditional microphone arrays are due to the problems of large size and high power consumption.
Two vector speech sensors that are orthogonal to each other and an omnidirectional microphone are used as the acquisition unit. Through frame division, windowing, fast Fourier transformation, directional processing and filter design, spatial noise is removed and voice signal enhancement is achieved.
In miniaturized equipment, it effectively reduces noise interference and improves sound pickup effect. It uses the natural ability of vector voice sensors to suppress diffuse noise, and improves the quality and intelligibility of voice signals.
Smart Images

Figure CN113889133B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of signal processing technology, and in particular to a speech enhancement method and device based on a vector speech sensor array. Background Art
[0002] There are various types of noise in nature, including environmental noise, interfering speech, and noise within the device. During the sound pickup process, these noises are inevitably introduced into the sound pickup results, affecting the sound pickup effect and reducing the quality and intelligibility of the target speech. To reduce the impact of noise on the sound pickup results, speech enhancement technology is needed to remove these various types of noise. Speech enhancement technologies include single-microphone speech enhancement technology and microphone array speech enhancement technology, which can fully utilize the spatial information of speech and noise signals and has better results than single-microphone speech enhancement technology. However, microphone array speech enhancement technology requires the use of multiple microphones to pick up sound simultaneously, and the spacing between these microphones cannot be too small. Therefore, the microphone array is generally large in size, which is not conducive to the development of low-power, miniaturized sound pickup equipment. Summary of the Invention
[0003] The present invention provides a speech enhancement method and device based on a vector speech sensor array, so as to solve the problem in the prior art that the speech enhancement technology cannot be realized under the premise of miniaturization.
[0004] In the first aspect, the present invention provides a speech enhancement method based on a vector speech sensor array, the method comprising: using two mutually orthogonal vector speech sensors and an omnidirectional microphone as original sound signal acquisition units to collect noisy speech signals, wherein the vector speech sensor and the omnidirectional microphone are arranged at the same point; framing and windowing the noisy speech signal, and performing fast Fourier transform (FFT) of the single-frame signal, then calculating the direction of the signal at each frequency point, removing spatial noise, and finally performing inverse fast Fourier transform (IFFT) and time-domain enhanced speech generation to obtain an enhanced speech signal.
[0005] Optionally, the two mutually orthogonal vector voice sensors are a first vector voice sensor v1 and a second vector voice sensor v2, the sensitive direction of the first vector voice sensor v1 is 0°, the sensitive direction of the second vector voice sensor v2 is 90°, and the angle range of the plane where the first vector voice sensor v1 and the second vector voice sensor v2 are located is [-180, 180].
[0006] The first vector speech sensor v1 collects a first sound vector signal x, and the second vector speech sensor v2 collects a second sound vector signal y, wherein the sound vector signals collected by the first vector speech sensor v1 and the second vector speech sensor are orthogonal, and the omnidirectional microphone collects a sound scalar signal p.
[0007] Optionally, framing and windowing the noisy speech signal includes: framing and windowing the first sound vector signal x collected by the first vector speech sensor v1, the second sound vector signal y collected by the second vector speech sensor v2, and the sound scalar signal p collected by the omnidirectional microphone to obtain the first sound vector time domain single frame signal x win (1), the second sound vector time domain single frame signal y win (l), acoustic scalar time domain single frame signal p win (l), where l = 1, 2, ..., L, and L is the length of a single-frame time-domain noisy speech signal.
[0008] Optionally, the frame length of the frame windowing is 20 ms, the frame shift is 10 ms, and the window function is a Hanning window.
[0009] Optionally, performing fast Fourier transform of a single-frame signal includes:
[0010] For the first sound vector time domain single frame signal x win (1), the second sound vector time domain single frame signal y win (l), acoustic scalar time domain single frame signal p win (1) Perform the following processing:
[0011]
[0012]
[0013]
[0014] Among them, ceil means rounding up, x fwin (l), y fwin (l), p fwin (l) the first sound vector to be transformed time domain single frame signal, the second sound vector to be transformed time domain single frame signal, and the sound scalar to be transformed time domain single frame signal corresponding to v1, v2, and u respectively;
[0015] Get x fwin (l), y fwin (l), p fwin (l) is converted to the frequency domain using FFT to obtain a single-frame signal in the frequency domain.
[0016] X win (k) = fft(xfwin (l)) (4)
[0017] Y win (k) = fft(y fwin (l)) (5)
[0018] P win (k) = fft(p fwin (l)) (6)
[0019] Among them, X win (k),Y win (k),P win (k) are the first sound vector frequency domain single frame signal, the second sound vector frequency domain single frame signal, and the sound scalar frequency domain single frame signal corresponding to v1, v2, and u respectively, k is the frequency point number, and fft is the fast Fourier transform operator.
[0020] Optionally, calculating the direction of the signal at each frequency point includes:
[0021] Using the first sound vector to be transformed time domain single frame signal x fwin (1) The second sound vector is to be transformed into a single-frame time domain signal y fwin (l) The single-frame time-domain signal p to be transformed by the acoustic scalar fwin (l) The first single-frame signal X in the frequency domain of the acoustic vector after fast Fourier transform win (k), the second sound vector frequency domain single frame signal Y win (k), single-frame signal P in the frequency domain of acoustic scalar win (k) Orient the signals at each frequency point, determine the direction of the signals at each frequency point, and determine whether they belong to the target speech;
[0022] A complex sound intensity detector composed of a scalar sound field and a vector sound field is used for orientation to reduce the impact of ambient noise on the orientation result and improve the orientation accuracy. The definition of the complex sound intensity detector is:
[0023]
[0024]
[0025] Among them, I px (k),I py (k) represents the complex intensifier composed of p component and x component, p component and y component at frequency k, * represents the complex conjugate, Re represents the real part;
[0026] Get I px (k),I py (k), the direction of each frequency signal can be calculated by the following formula:
[0027]
[0028] Wherein, θ(k) represents the signal direction at frequency point k, and arctan represents the inverse tangent operator.
[0029] Optionally, the method also includes: removing spatial noise through a filter, and designing the filter based on the first sound vector signal x collected by the first vector speech sensor v1, the second sound vector signal y collected by the second vector speech sensor v2, and the sound scalar signal p collected by the omnidirectional microphone.
[0030] Optionally, designing the filter according to the first sound vector signal x collected by the first vector speech sensor v1, the second sound vector signal y collected by the second vector speech sensor v2, and the sound scalar signal p collected by the omnidirectional microphone includes:
[0031] All frequency signals with the signal direction within the range of [-15°, 15°] are retained. Based on this, the filter design formula is:
[0032]
[0033] Where G(k) is the filter gain function;
[0034] After obtaining the filter gain function G(k), the first sound vector frequency domain single frame signal X corresponding to v1 is converted to win Multiplying (k) with G(k) can obtain the frequency domain single-frame enhanced speech signal.
[0035] X enhwin (k) = X win (k)×G(ω) (11)
[0036] Among them, X enhwin (k) is a single-frame enhanced speech signal in the frequency domain.
[0037] Optionally, the inverse fast Fourier transform and time domain enhanced speech generation to obtain an enhanced speech signal includes: performing a frequency domain single frame enhanced speech signal X after removing spatial noise. enhwin (k) Perform IFFT to obtain the corresponding time domain single-frame enhanced speech signal,
[0038] x enhwin (k) = ifft(X enhwin (k)) (12)
[0039] Among them, x enhwin (k) is the time domain single-frame enhanced speech signal, ifft is the IFFT operator;
[0040] After all the enhanced speech in the time domain is obtained, it is synthesized according to the frame division and windowing parameters using the splicing and addition method to obtain the enhanced result of the entire noisy speech.
[0041] In a second aspect, the present invention provides a speech enhancement device based on a vector speech sensor array, the device comprising:
[0042] A collection unit for collecting noisy speech signals, wherein the collection unit is composed of two mutually orthogonal vector speech sensors and an omnidirectional microphone as the original sound signal, and the vector speech sensors and the omnidirectional microphone are arranged at the same point;
[0043] The processing unit is used to frame and window the noisy speech signal, perform fast Fourier transform (FFT) on the single-frame signal, calculate the direction of the signal at each frequency point, remove spatial noise, and finally perform inverse fast Fourier transform (IFFT) and time-domain enhanced speech generation to obtain an enhanced speech signal.
[0044] The beneficial effects of the present invention are as follows:
[0045] This invention utilizes an array of two mutually orthogonal vector speech sensors, enabling speech enhancement using microphone array speech enhancement technology. Because the array elements can be co-located, it offers the advantage of miniaturization. Furthermore, vector speech sensors inherently suppress diffuse noise, further enhancing sound pickup. Therefore, using a vector speech sensor array for sound pickup, combined with corresponding speech enhancement technology, can reduce noise interference while maintaining a compact design, effectively improving sound pickup.
[0046] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are specifically listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present invention. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:
[0048] Figure 1 1 is a flow chart of a speech enhancement method based on a vector speech sensor array provided by the first embodiment of the present invention;
[0049] Figure 2 is a schematic structural diagram of a collection unit provided by the first embodiment of the present invention;
[0050] Figure 3 This is a flow chart of a speech enhancement device based on a vector speech sensor array provided by the second embodiment of the present invention. DETAILED DESCRIPTION
[0051] The embodiments of the present invention address the problem that existing voice enhancement technologies cannot be implemented while miniaturizing. By utilizing two mutually orthogonal vector voice sensors to form an array, voice enhancement can be performed using microphone array voice enhancement technology. Furthermore, since the array elements can be co-located, miniaturization is possible. Furthermore, the vector voice sensor itself has a natural ability to suppress diffuse noise, which can further enhance the sound pickup effect. Therefore, using a vector voice sensor array for sound pickup, combined with corresponding voice enhancement technology, can reduce noise interference while miniaturizing and effectively enhance the sound pickup effect. The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely intended to explain the present invention and do not limit the present invention.
[0052] The first embodiment of the present invention provides a method for speech enhancement based on a vector speech sensor array, see Figure 1 , the method comprising:
[0053] S101, using two mutually orthogonal vector speech sensors and an omnidirectional microphone as original sound signal acquisition units to collect noisy speech signals, wherein the vector speech sensors and the omnidirectional microphone are co-located;
[0054] That is, the two mutually orthogonal vector voice sensors in the embodiment of the present invention are respectively a first vector voice sensor v1 and a second vector voice sensor v2, the sensitive direction of the first vector voice sensor v1 is 0°, the sensitive direction of the second vector voice sensor v2 is 90°, and the angle range of the plane in which the first vector voice sensor v1 and the second vector voice sensor v2 are located is [-180, 180].
[0055] The first vector speech sensor v1 collects a first sound vector signal x, and the second vector speech sensor v2 collects a second sound vector signal y, wherein the sound vector signals collected by the first vector speech sensor v1 and the second vector speech sensor are orthogonal, and the omnidirectional microphone collects a sound scalar signal p.
[0056] S102, framing and windowing the noisy speech signal;
[0057] Specifically, the embodiment of the present invention is to perform frame windowing on the first sound vector signal x collected by the first vector speech sensor v1, the second sound vector signal y collected by the second vector speech sensor v2, and the sound scalar signal p collected by the omnidirectional microphone to obtain the corresponding single-frame time-domain noisy speech signal, that is, the first sound vector single-frame signal x win (1), the second sound vector single frame signal y win (l), sound scalar single frame signal p win (l), where l = 1, 2, ..., L, and L is the length of a single-frame time-domain noisy speech signal.
[0058] Wherein, the frame length of the framing windowing in the embodiment of the present invention is 20ms, the frame shift is 10ms, and the window function is a Hanning window. Of course, those skilled in the art can also adopt other framing windowing parameters according to actual needs during specific implementation, and the present invention does not make detailed restrictions on this.
[0059] S103, performing fast Fourier transform (FFT) of a single frame signal;
[0060] That is, it is a single frame of time domain noisy speech signal x win (l), y win (l), p win (1) Perform the following processing:
[0061]
[0062]
[0063]
[0064] Among them, ceil means rounding up, x fwin (l), y fwin (l), p fwin (l) the first sound vector to be transformed time domain single frame signal, the second sound vector to be transformed time domain single frame signal, and the sound scalar to be transformed time domain single frame signal corresponding to v1, v2, and u respectively;
[0065] Get x fwin (l), y fwin (l), p fwin (l) is converted to the frequency domain using FFT to obtain a single-frame frequency domain noisy speech signal.
[0066] X win (k) = fft(x fwin (l)) (4)
[0067] Y win (k) = fft(y fwin (l)) (5)
[0068] P win (k) = fft(p fwin (l)) (6)
[0069] Among them, X win (k),Y win (k),P win (k) are single-frame frequency-domain noisy speech signals corresponding to v1, v2, and u, respectively. k is the frequency point number, and fft is the fast Fourier transform operator.
[0070] S104, calculating the direction of the signal at each frequency point;
[0071] In specific implementation, the embodiment of the present invention uses the first sound vector to be transformed time domain single frame signal x fwin (1) The second sound vector is to be transformed into a single-frame time domain signal y fwin (l) The single-frame time-domain signal p to be transformed by the acoustic scalar fwin (l) The first single-frame signal X in the frequency domain of the acoustic vector after fast Fourier transform win (k), the second sound vector frequency domain single frame signal Y win (k), single-frame signal P in the frequency domain of acoustic scalar win (k) Orient the signals at each frequency point, determine the direction of the signals at each frequency point, and determine whether they belong to the target speech;
[0072] A complex sound intensity detector composed of a scalar sound field and a vector sound field is used for orientation to reduce the impact of ambient noise on the orientation result and improve the orientation accuracy. The definition of the complex sound intensity detector is:
[0073]
[0074]
[0075] Among them, I px (k),I py (k) represents the complex intensifier composed of p component and x component, p component and y component at frequency k, * represents the complex conjugate, Re represents the real part;
[0076] Get I px (k),I py (k), the direction of each frequency signal can be calculated by the following formula:
[0077]
[0078] Wherein, θ(k) represents the signal direction at frequency point k, and arctan represents the inverse tangent operator.
[0079] S105, removing spatial noise;
[0080] An embodiment of the present invention removes spatial noise through a filter, and designs the filter based on the first sound vector signal x collected by the first vector speech sensor v1, the second sound vector signal y collected by the second vector speech sensor v2, and the sound scalar signal p collected by the omnidirectional microphone.
[0081] Specifically, the embodiment of the present invention retains all frequency signals whose directions are within the range of [-15°, 15°]. Based on this, the filter design formula is:
[0082]
[0083] Where G(k) is the filter gain function;
[0084] After obtaining the filter gain function G(k), the single-frame frequency domain noisy speech signal X corresponding to v1 is converted to win Multiplying (k) with G(k) can obtain the frequency domain single-frame enhanced speech signal.
[0085] X enhwin (k) = X win (k)×G(ω) (11)
[0086] Among them, X enhwin (k) is a single-frame enhanced speech signal in the frequency domain.
[0087] S106: Perform inverse fast Fourier transform (IFFT) and time-domain enhanced speech generation to obtain an enhanced speech signal.
[0088] This step specifically includes: enhancing the speech signal X in the frequency domain by removing the spatial noise enhwin (k) Perform IFFT to obtain the corresponding time domain single-frame enhanced speech signal,
[0089] x enhwin (k) = ifft(X enhwin (k)) (12)
[0090] Among them, x enhwin (k) is the time domain single-frame enhanced speech signal, ifft is the IFFT operator;
[0091] After all the enhanced speech in the time domain is obtained, it is synthesized according to the frame division and windowing parameters using the splicing and addition method to obtain the enhanced result of the entire noisy speech.
[0092] Specifically, the embodiments of the present invention utilize an array of two mutually orthogonal vector voice sensors, enabling the use of microphone array voice enhancement technology for voice enhancement. Furthermore, since the array elements can be co-located, they offer the advantage of miniaturization. Furthermore, vector voice sensors inherently have the ability to suppress diffuse noise, further enhancing sound pickup. Therefore, employing a vector voice sensor array for sound pickup, combined with corresponding voice enhancement technology, can reduce noise interference while maintaining a compact design, effectively improving sound pickup.
[0093] The method of the present invention will be explained and illustrated in detail below through a specific example:
[0094] This method utilizes two mutually orthogonal vector speech sensors and an omnidirectional microphone as the original sound signal acquisition unit (three array elements are arranged at the same point). Based on the collected sound vector and scalar signals, a filter is designed to remove spatial noise. Specifically, the method includes: 1) system layout and noisy speech signal acquisition; 2) noisy speech signal framing and windowing; 3) single-frame signal FFT (Fast Fourier Transform); 4) calculation of the signal direction at each frequency point; 5) filter design and noise removal; and 6) IFFT (Inverse Fast Fourier Transform) and time-domain enhanced speech generation.
[0095] Module 1 performs signal acquisition:
[0096] like Figure 2 The layout diagram of the acquisition unit of the present invention is shown as follows: Figure 2 It can be seen that the angular range of the plane where the vector speech sensor array of the embodiment of the present invention is located is defined as [-180, 180); v1 and v2 respectively represent two vector speech sensors in the vector speech sensor array, which are used to collect sound vector signals, and their sensitive directions are 0° and 90° respectively; u represents the omnidirectional microphone in the vector speech sensor array, which is used to collect scalar sound fields; the target sound source direction angle is 0°. The purpose of adding an omnidirectional microphone to the array is to use a complex sound intensifier to perform sound source orientation, so as to achieve the purpose of improving the orientation accuracy. The purpose of the entire system is to retain the target speech signal from the target sound source direction (0°) as much as possible, and suppress the non-target speech signal (noise) in other angle ranges.
[0097] according to Figure 2 After the arrangement shown is completed, the noisy speech signal is collected using the vector speech sensor array. The signals collected by v1, v2, and u are recorded as x, y, and p respectively, and then the next step is entered.
[0098] Module 2: Framing and windowing of noisy speech signals:
[0099] Framing and windowing are basic preprocessing processes in speech signal processing, and their purpose is to ensure that the processed speech signal is approximately a stationary signal. Frame and window x, y, and p in module 1 to obtain the corresponding single-frame time-domain noisy speech signal x win (l), y win (l), p win (l) (l = 1, 2, ..., L, where L is the length of a single frame of time-domain noisy speech signal). The frame-splitting and windowing parameters are selected as follows: frame length: 20 ms; frame shift: 10 ms; window function: Hanning window.
[0100] Module 3 is single frame signal FFT:
[0101] The purpose of this module is to convert a single-frame time-domain noisy speech signal into the frequency domain. Since FFT requires the signal length to be an integer power of 2, and for signals with different sampling rates, the length of a single-frame time-domain noisy speech signal may not meet this condition according to the frame length parameter selection of module 2. Therefore, for a single-frame time-domain noisy speech signal x win (l), y win (l), p win (1) Perform the following processing:
[0102]
[0103]
[0104]
[0105] Among them, ceil means rounding up, x fwin (l), y fwin (l), p fwin (l) are the first sound vector time domain single frame signal to be transformed, the second sound vector time domain single frame signal to be transformed, and the sound scalar time domain single frame signal to be transformed corresponding to v1, v2, and u respectively.
[0106] Get x fwin (l), y fwin (l), p fwin (l), we can use FFT to convert it into the frequency domain and obtain a single-frame frequency domain noisy speech signal.
[0107] X win (k) = fft(x fwin (l)) (4)
[0108] Y win (k) = fft(y fwin (l)) (5)
[0109] P win(k) = fft(p fwin (l)) (6)
[0110] Among them, X win (k),Y win (k),P win (k) are the first sound vector frequency domain single frame signal, the second sound vector frequency domain single frame signal, and the sound scalar frequency domain single frame signal corresponding to v1, v2, and u respectively, k is the frequency point number, and fft is the fast Fourier transform operator.
[0111] Module 4 calculates the signal of each frequency point to be:
[0112] The purpose of this module is to obtain X based on the results of modules 1, 2 and 3. win (k),Y win (k),P win (k) Orient the signals at each frequency point, determine the direction of the signals at each frequency point, and judge whether they belong to the target speech.
[0113] In order to reduce the influence of environmental noise on the directional results and improve the directional accuracy, a complex sound intensity device composed of a scalar sound field and a vector sound field is used for directional control. The definition of the complex sound intensity device is:
[0114]
[0115]
[0116] Among them, I px (k),I py (k) represents the complex intensifier composed of p component and x component, p component and y component at frequency k, * Re stands for the real part.
[0117] Get I px (k),I py (k), the direction of each frequency signal can be calculated by the following formula:
[0118]
[0119] Wherein, θ(k) represents the signal direction at frequency point k, and arctan represents the inverse tangent operator.
[0120] Module 5 is filter design and noise removal: After obtaining the signal direction of each frequency point through modules 1, 2, 3, and 4, the filter can be designed based on the signal direction. The design principle of the filter is: in actual operation, there will be errors in the directional results, and the target sound source may not be completely located in the 0° direction. Therefore, in order to improve the robustness of the system, a 15° buffer is set, that is, all frequency point signals with a signal direction within the range of [-15°, 15°] are retained. Based on this, the filter design can be represented by the following formula:
[0121]
[0122] Where G(k) is the filter gain function.
[0123] After obtaining the filter gain function G(k), the single-frame frequency domain noisy speech signal X corresponding to v1 is converted to win Multiplying (k) with G(k) can obtain the frequency domain single-frame enhanced speech signal.
[0124] X enhwin (k) = X win (k)×G(ω) (11)
[0125] Among them, X enhwin (k) is a single-frame enhanced speech signal in the frequency domain.
[0126] Module 6 is IFFT and time domain enhanced speech generation: through modules 1, 2, 3, 4 and 5, the frequency domain single frame enhanced speech signal X is obtained. enhwin (k) after X enhwin (k) Perform IFFT to obtain the corresponding single-frame time-domain enhanced speech signal.
[0127] x enhwin (k) = ifft(X enhwin (k)) (12)
[0128] Among them, x enhwin (k) is a single-frame time-domain enhanced speech signal, and ifft is the IFFT operator.
[0129] After all single-frame time-domain enhanced speech is obtained, it is synthesized according to the frame division and windowing parameters using the overlap-and-add method (OLA) to obtain the enhancement result of the entire noisy speech.
[0130] In general, the embodiments of the present invention utilize a vector speech sensor array to collect speech signals, use a complex intensifier to orient the acoustic signal at each frequency point, and design a filter based on the orientation results to achieve noise removal. Compared to single-microphone speech enhancement technology, the present invention can remove non-stationary noise in space; compared to traditional microphone array speech enhancement technology, the present invention has a smaller size. Furthermore, the present invention fully utilizes the inherent noise suppression properties of the vector speech sensor to further improve sound pickup.
[0131] The second embodiment of the present invention provides a speech enhancement device based on a vector speech sensor array, see Figure 3 , including: an acquisition unit for acquiring noisy speech signals, wherein the acquisition unit is composed of two mutually orthogonal vector speech sensors and an omnidirectional microphone as original sound signals, and the vector speech sensors and the omnidirectional microphone are arranged at the same point; a processing unit for framing and windowing the noisy speech signals, performing fast Fourier transform (FFT) of a single frame signal, then calculating the direction of the signal at each frequency point, removing spatial noise, and finally performing inverse fast Fourier transform (IFFT) and time domain enhanced speech generation to obtain an enhanced speech signal.
[0132] In summary, the embodiments of the present invention utilize an array of two mutually orthogonal vector voice sensors, enabling the use of microphone array voice enhancement technology for voice enhancement. Furthermore, since the array elements can be co-located, they offer the advantage of miniaturization. Furthermore, vector voice sensors inherently have the ability to suppress diffuse noise, further enhancing sound pickup. Therefore, employing a vector voice sensor array for sound pickup, combined with corresponding voice enhancement technology, can reduce noise interference while maintaining a compact design, effectively improving sound pickup.
[0133] The relevant contents of the embodiments of the present invention can be understood by referring to the first embodiment of the present invention, and will not be discussed in detail here.
[0134] Although the preferred embodiments of the present invention have been disclosed for illustrative purposes, those skilled in the art will appreciate that various modifications, additions and substitutions are possible, and thus, the scope of the present invention should not be limited to the above embodiments.
Claims
1. A speech enhancement method based on a vector speech sensor array, characterized in that: include: Using two mutually orthogonal vector speech sensors and an omnidirectional microphone as an original sound signal acquisition unit to collect noisy speech signals, wherein the vector speech sensors and the omnidirectional microphone are arranged at the same point; The noisy speech signal is framed and windowed, and a single-frame signal Fast Fourier Transform FFT is performed, and then the direction of the signal at each frequency point is calculated, the spatial noise is removed, and finally an Inverse Fast Fourier Transform IFFT and time-domain enhanced speech generation are performed to obtain an enhanced speech signal.
2. The method according to claim 1, characterized in that The two mutually orthogonal vector voice sensors are respectively a first vector voice sensor v1 and a second vector voice sensor v2, the sensitive direction of the first vector voice sensor v1 is 0°, the sensitive direction of the second vector voice sensor v2 is 90°, and the angle range of the plane where the first vector voice sensor v1 and the second vector voice sensor v2 are located is [-180, 180]; The first vector speech sensor v1 collects a first sound vector signal x, and the second vector speech sensor v2 collects a second sound vector signal y, wherein the sound vector signals collected by the first vector speech sensor v1 and the second vector speech sensor are orthogonal, and the omnidirectional microphone collects a sound scalar signal p.
3. The method according to claim 2, characterized in that The noisy speech signal is framed and windowed, including: The first sound vector signal x collected by the first vector speech sensor v1, the second sound vector signal y collected by the second vector speech sensor v2, and the sound scalar signal p collected by the omnidirectional microphone are framed and windowed to obtain a first sound vector time domain single frame signal x win (l), the second sound vector time domain single frame signal y win (l), single-frame signal p in the acoustic scalar time domain win (l), where l = 1, 2, ..., L, and L is the length of a single frame of time-domain noisy speech signal.
4. The method according to claim 3, characterized in that The frame length of the frame-splitting and windowing is 20 ms, the frame shift is 10 ms, and the window function is a Hanning window.
5. The method according to claim 3, characterized in that: The single-frame signal fast Fourier transform comprises: For the first sound vector time domain single frame signal x win (l), the second sound vector time domain single frame signal y win (l), single-frame signal p in the acoustic scalar time domain win (l) Perform the following processing: Among them, ceil means rounding up, x fwin (l), y fwin (l), p fwin (l) respectively correspond to v1, v2, and u, the first sound vector to be transformed time domain single frame signal, the second sound vector to be transformed time domain single frame signal, and the sound scalar to be transformed time domain single frame signal; Get x fwin (l), y fwin (l), p fwin (l), and then use FFT to convert it into the frequency domain to obtain a single frame signal in the frequency domain. X win (k)=fft(x fwin (l)) (4) Y win (k)=fft(y fwin (l)) (5) P win (k) = fft(p fwin (l)) (6) Where X win (k),Y win (k),P win (k) are the first sound vector frequency domain single frame signal, the second sound vector frequency domain single frame signal, and the sound scalar frequency domain single frame signal corresponding to v1, v2, and u respectively, k is the frequency point number, and fft is the fast Fourier transform operator.
6. The method according to claim 5, characterized in that The calculating of the direction of the signal at each frequency point includes: Using the first sound vector to be transformed time domain single frame signal x fwin (l), the second sound vector to be transformed time domain single frame signal y fwin (l), the single-frame signal p in the time domain to be transformed by the acoustic scalar fwin (l) The first single-frame signal X in the frequency domain of the sound vector after fast Fourier transformation win (k), the second sound vector frequency domain single frame signal Y win (k), single frame signal P in the frequency domain of acoustic scalar win (k) orienting the signals at each frequency point, determining the direction of the signals at each frequency point, and judging whether they belong to the target speech; A complex sound intensity device composed of a scalar sound field and a vector sound field is used for orientation to reduce the impact of environmental noise on the orientation result and improve the orientation accuracy. The definition of the complex sound intensity device is: Among them, I px (k),I py (k) represents the complex sound intensifier composed of p component and x component, p component and y component at frequency k, respectively. * represents the complex conjugate, Re represents the real part; Get I px (k),I py (k), the direction of the signal at each frequency point can be calculated by the following formula: Among them, θ(k) represents the signal direction at frequency point k, and arctan represents the inverse tangent operator.
7. The method according to claim 2, characterized in that: The method further comprises: The spatial noise is removed by a filter, and the filter is designed according to the first sound vector signal x collected by the first vector speech sensor v1, the second sound vector signal y collected by the second vector speech sensor v2, and the sound scalar signal p collected by the omnidirectional microphone.
8. The method according to claim 7, characterized in that The filter is designed according to the first sound vector signal x collected by the first vector speech sensor v1, the second sound vector signal y collected by the second vector speech sensor v2, and the sound scalar signal p collected by the omnidirectional microphone, including: All frequency signals with a signal direction within the range of [-15°, 15°] are retained. Based on this, the filter design formula is: Where G(k) is the filter gain function; After obtaining the filter gain function G(k), the first sound vector frequency domain single frame signal X corresponding to v1 is converted into win By multiplying (k) with G(k), we can get the frequency domain single-frame enhanced speech signal. X enhwin (k)=X win (k)×G(ω) (11) Among them, X enhwin (k) is a single-frame enhanced speech signal in the frequency domain.
9. The method according to claim 8, characterized in that The method of performing inverse fast Fourier transform and time domain enhanced speech generation to obtain an enhanced speech signal includes: The frequency domain single frame enhanced speech signal X is removed from the spatial noise enhwin (k) Perform IFFT to obtain the corresponding single-frame enhanced speech signal in the time domain, x enhwin (k)=ifft(X enhwin (k)) (12) Among them, x enhwin (k) is a single-frame enhanced speech signal in the time domain, and ifft is an IFFT operator; After all the enhanced speech of single frames in the time domain are obtained, they are synthesized according to the frame division and windowing parameters using the splicing addition method to obtain the enhanced result of the entire noisy speech.
10. A speech enhancement device based on a vector speech sensor array, characterized in that: include: A collection unit, used for collecting noisy speech signals, wherein the collection unit is composed of two mutually orthogonal vector speech sensors and an omnidirectional microphone as original sound signals, and the vector speech sensors and the omnidirectional microphone are arranged at the same point; The processing unit is used to frame and window the noisy speech signal, perform fast Fourier transform (FFT) on the single-frame signal, calculate the direction of the signal at each frequency point, remove the spatial noise, and finally perform inverse fast Fourier transform (IFFT) and time-domain enhanced speech generation to obtain an enhanced speech signal.
