Speech enhancement method based on planar quaternary microphone array and electronic equipment

Abstract

The invention belongs to the field of voice signal processing, and provides a voice enhancement method based on a planar quaternary microphone array and electronic equipment. The method comprises the following steps: collecting signals received by four microphones of a planar quaternary microphone array to obtain four paths of noisy voice signals; performing voice differential wave beam forming pairwise to obtain four paths of enhanced differential voice signals; performing summation average beam forming processing on the four paths of enhanced differential voice signals to obtain voice enhanced signals of which the noise is further attenuated; carrying out weighted difference operation on every two of the four paths of noisy voice signals to obtain four paths of differential noise signals; a multi-path adaptive noise cancellation (MANC) algorithm based on voice activity detection VAD improvement is adopted for the voice enhancement signals and the four paths of differential noise signals, and voice signals enhanced again are obtained; and performing recovery processing on the re-enhanced voice signal by using a time domain recovery algorithm to obtain a normal enhanced voice signal. The speech enhancement effect is good.

Description

Technical Field

[0001] This application belongs to the field of speech signal processing technology, and in particular relates to a speech enhancement method based on a planar four-element microphone array, a computer-readable storage medium, and an electronic device. Background Technology

[0002] Speech enhancement is a crucial technology in electronic devices such as hearing aids, smart speakers, and headphones. Its main purpose is to suppress noise from noisy speech to obtain clean speech, thereby improving speech clarity and intelligibility. Hearing aids mainly include hearing aids, cochlear implants, middle ear implants, and assistive hearing devices.

[0003] Speech enhancement is crucial for speech communication and speech recognition in complex environments. However, existing speech enhancement methods based on planar four-element microphone arrays do not provide satisfactory results. Summary of the Invention

[0004] The purpose of this application is to provide a speech enhancement method, a computer-readable storage medium, and an electronic device based on a planar four-element microphone array, aiming to solve the problem that the speech enhancement effect of existing speech enhancement methods based on planar four-element microphone arrays is not good enough.

[0005] In a first aspect, this application provides a speech enhancement method based on a planar four-element microphone array, the method comprising the following steps: S101. Collect the signals received by the four microphones of the planar four-element microphone array to obtain four noisy speech signals. S102. Perform differential beamforming on each pair of the four noisy speech signals to obtain four enhanced differential speech signals. S103. Perform summation and average beamforming processing on the four enhanced differential speech signals to obtain a speech enhancement signal with further noise attenuation. S104. Perform weighted differential operations on each pair of the four noisy speech signals to obtain four differential noise signals. S105. By applying the improved multi-channel adaptive noise cancellation (MANC) algorithm based on sound activity detection (VAD) to the speech enhancement signal and the four differential noise signals, the residual noise in the speech enhancement signal is further eliminated, and a re-enhanced speech signal is obtained. S106. The enhanced speech signal is restored using a time-domain recovery algorithm to obtain a normal enhanced speech signal.

[0006] Secondly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the speech enhancement method based on a planar four-element microphone array as described above.

[0007] Thirdly, this application provides an electronic device, comprising: One or more processors; Memory; and One or more computer programs, wherein the processor and the memory are connected via a bus, wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, wherein the processors, when executing the computer programs, implement the steps of the speech enhancement method based on a planar four-element microphone array as described above.

[0008] In this application, four noisy speech signals are weighted and differentially analyzed pairwise to obtain four differential noise signals. Then, differential beamforming is performed on each pair of the four noisy speech signals to obtain four enhanced differential speech signals. These enhanced differential speech signals are then subjected to summation-average beamforming to obtain a speech enhancement signal with further noise attenuation. Finally, a modified multi-channel adaptive noise cancellation (MANC) algorithm based on voice activity detection (VAD) is applied to the speech enhancement signal and the four differential noise signals to further eliminate residual noise in the speech enhancement signal, resulting in a re-enhanced speech signal. This re-enhanced speech signal is then restored using a time-domain recovery algorithm to obtain a normal enhanced speech signal. Therefore, the speech enhancement effect is excellent. Attached Figure Description

[0009] Figure 1 This is a flowchart of a speech enhancement method based on a planar four-element microphone array provided in an embodiment of this application.

[0010] Figure 2 This is a schematic diagram of a planar four-element microphone array for signal acquisition.

[0011] Figure 3 It is a beammap of speech differential beamforming.

[0012] Figure 4 This is a schematic diagram of an improved multi-path adaptive noise cancellation algorithm.

[0013] Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0014] To make the objectives, technical solutions, and beneficial effects of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0015] To illustrate the technical solution described in this application, specific embodiments are provided below.

[0016] Please see Figure 1 This is a flowchart of a speech enhancement method based on a planar four-element microphone array provided in an embodiment of this application. This embodiment mainly uses the application of this speech enhancement method based on a planar four-element microphone array to an electronic device as an example for illustration. The speech enhancement method based on a planar four-element microphone array provided in an embodiment of this application includes the following steps: S101. Collect the signals received by the four microphones of the planar four-element microphone array to obtain four noisy speech signals.

[0017] In one embodiment of this application, the microphone may be a MEMS microphone (i.e., a microelectromechanical system microphone), an ECM microphone (i.e., an electret condenser microphone), or the like.

[0018] Please see Figure 2 In one embodiment of this application, the side length of one side of the rectangle formed by the planar four-element microphone array The calculation formula is as follows: (1) in, The sampling frequency of the speech signal. The speed at which sound travels through the air. In one embodiment of this application, , ,but .

[0019] like Figure 2 As shown, ideally, the voice source is in front of the wearer of the electronic device, and the noise source is to the left or right of the wearer; that is, the voice source is located on one side of the rectangle formed by the planar four-element microphone array. On the extension line, and close to the adjacent edge At one end, the noise source is located on the vertical line of the rectangle formed by the planar four-element microphone array.

[0020] In one embodiment of this application, step S101 may specifically be: Suppose that the noisy speech signal obtained by sampling the i-th microphone of the planar four-element microphone array at sampling time n is: The voice signal is The interference noise signal is ,but (2) (3) (4) The side length of the side parallel to the direction of the speech source in the rectangle formed by the planar four-element microphone array. hour, (5) in, The sampling frequency of the speech signal. The speed at which sound travels through the air. For the speech signal, the first microphone To the microphone spread Signal amplitude attenuation factor at distance, In one embodiment of this application .

[0021] S102. Perform differential beamforming on each pair of the four noisy speech signals to obtain four enhanced differential speech signals.

[0022] Differential beamforming for speech is used to enhance the signal from the direction of the speech source and suppress signals from other directions.

[0023] In one embodiment of this application, the enhanced differential speech signal In the process, interference noise signals are attenuated while speech signals are preserved. However, differential beamforming will cause distortion of the speech signal. The distorted speech signal will not be restored for the time being.

[0024] In one embodiment of this application, the step of performing differential beamforming on each pair of four noisy speech signals to obtain four enhanced differential speech signals may specifically include: By any two noisy speech signals A differential operation is performed to obtain an enhanced differential speech signal. , : (6) (7) (8) (9) As can be seen from equation (6), for the first microphone and the fourth microphone After the noisy speech signal is picked up and differentially processed, the interference noise signal is removed. It was subtracted, while the voice signal It was preserved, but the voice signal was distorted. Distortion .

[0025] However, in practical applications, the noise source may not be located on the vertical line of the rectangle formed by the planar four-element microphone array, or due to the influence of the reflection and refraction path of noise in the actual environment, etc., speech differential beamforming often cannot completely eliminate the interference noise signal. That is, there is still residual noise in equation (6). The residual noise can be determined according to the beam pattern of speech differential beamforming.

[0026] Please see Figure 3 This is the beam pattern of differential beamforming for speech at a frequency of 1kHz. The speech signal is in or The direction (forward or backward) remains intact, but there is attenuation in other directions, especially in... or The attenuation is greatest in the direction (i.e., left or right). When a planar quad microphone array is used in headphones, the signal is usually attenuated due to the action of the auricle or other devices. It is also somewhat suppressed when moving in the direction (backwards).

[0027] The enhanced differential speech signal represented by equation (6) Just as residual noise is still present in the signal, equations (7), (8), and (9) respectively represent the enhanced differential speech signal. , , It also usually contains residual noise.

[0028] S103. Perform summation and average beamforming processing on the four enhanced differential speech signals to obtain a speech enhancement signal with further noise attenuation.

[0029] In one embodiment of this application, S103 specifically uses the following formula to calculate the speech enhancement signal. : (10) S104. Perform weighted differential operations on each pair of the four noisy speech signals to obtain four differential noise signals.

[0030] In one embodiment of this application, in the differential noise signal, the speech signal is attenuated while the interference noise signal is retained, providing an input signal for subsequent multi-path adaptive noise cancellation.

[0031] In one embodiment of this application, S104 may specifically include: By any two noisy speech signals Weighted differential operation is performed to obtain one differential noise signal. : (11) (12) (13) (14) S105. By applying a multi-channel adaptive noise cancellation (MANC) algorithm based on sound activity detection (VAD) to the speech enhancement signal and the four differential noise signals, residual noise in the speech enhancement signal is further eliminated, and a re-enhanced speech signal is obtained.

[0032] Despite the speech enhancement signal The residual noise in the signal is reduced further, but it usually still exists. In practice, it can be assumed that the interfering noise signal is uncorrelated with the speech signal, so the MANC algorithm can be applied to further cancel the residual noise in the speech enhancement signal.

[0033] In practical applications, the voice source may not be located exactly on one side of the rectangle formed by the planar four-element microphone array. On the extension line, or due to factors such as the reflection and refraction paths of the speech signal in the actual environment, the reference input signal of the MANC algorithm... It is difficult to guarantee that the signal is purely interference noise. It is inevitable that some speech will be mixed in. This will inevitably cause partial cancellation of speech while canceling noise. In order to avoid or reduce the cancellation of speech, the coefficients of the adaptive filter in the MANC algorithm are only updated during silent periods, while the coefficients of the adaptive filter remain unchanged during speech periods.

[0034] Please see Figure 4 In one embodiment of this application, S105 may specifically be: In the MANC algorithm based on Voice Activity Detection (VAD), the main input signal is the speech enhancement signal. The multiple reference input signals are four differential noise signals. To obtain an enhanced speech signal VAD enhances speech signals The algorithm detects periods of speech or silence and uses the detection results to control whether to update the coefficients of the adaptive filter A in the MINC algorithm. That is, it only updates during silent periods, while keeping the coefficients of the adaptive filter A unchanged during periods of speech. The coefficients of the adaptive filter A are updated using the Least Mean Square (LMS) algorithm.

[0035] The specific formula for S105 can be: Using MINC's adaptive algorithm, therefore (15) (16) (17) (18) in, Differential noise signal A vector composed of values ​​taken at different times; The output signal of the adaptive filter A; The error signal is derived from the speech enhancement signal. and the output signal of adaptive filter A The result of subtraction is a further enhanced speech signal; Let A be the coefficient vector of the adaptive filter A. The number of delayed sampling points for each input signal of the adaptive filter A can be, for example, taken as... By adjusting the coefficient vector of the adaptive filter A To make the error signal The mean squared expected value is minimized; The update step size representing the coefficients of the adaptive filter A is, in one embodiment of this application, taken as... .

[0036] S106. The enhanced speech signal is restored using a time-domain recovery algorithm to obtain a normal enhanced speech signal.

[0037] In one embodiment of this application, S106 may specifically be: For the enhanced speech signal An L-order finite-time impulse response (FIR) filtering algorithm is used for the reconstruction operation to obtain the normal enhanced speech signal at output. In one embodiment of this application, L is 64.

[0038] As can be seen from equation (10), the speech enhancement signal Not a voice signal It is not its distorted signal, but its distorted signal. It is necessary to perform time-domain recovery operations on it.

[0039] This application further enhances the speech signal. The recovery operation is performed using an L-order finite-time impulse response (FIR) filtering algorithm as follows: Taking the Z-transform of equation (10) yields (19) Equation (20) can be obtained from equation (19). (20) Transforming equation (20) from the Z-domain to the time domain, and when the positive integer L is sufficiently large, we have (twenty one) Pick (twenty two) (twenty three) but (twenty four) in, Let L be the coefficient vector of the recovery filter, and L be the order of the recovery filter.

[0040] Because the MANC algorithm can enhance speech signals The residual noise in the speech signal is suppressed, i.e., the speech signal is enhanced again in equation (17). It has a higher signal enhancement than speech. A higher signal-to-noise ratio, therefore recovery filtering can be targeted To proceed, that is (25) in (26) The algorithm for speech signal recovery using equation (25) only requires L-order finite impulse response (FIR) filtering.

[0041] In one embodiment of this application, the following steps may be included after S106: S107. Using a single-channel speech enhancement algorithm to enhance the normal speech signal Further enhancement processing is performed to obtain the final enhanced speech signal.

[0042] In one embodiment of this application, the single-channel speech enhancement algorithm is the log-minimum mean square error (logMMSE) algorithm.

[0043] S107 can specifically be: Assuming a normal enhanced speech signal ,in, It is a pure voice signal. This is a residual noise signal. and Independent of each other; for normal enhanced speech signals Adding a Hamming window for frame segmentation, let the frame length be... Frame shift is In one embodiment of this application, take ;make , and They represent , and The result of the Fast Fourier Transform (FFT) of the frame length points is given by the expression where, For the first One analysis window, For the first Each frequency component point is denoted as... (27) (28) in, yes The range, yes phase, yes The range, yes The phase; denote the final enhanced speech signal at the . The first analysis window The amplitude of each frequency component point is According to the logMMSE criterion, it is necessary to make Minimum, in speech signal and residual noise signal Under the assumption that all conform to the Gaussian statistical model, we can obtain The estimated value : (29) in , , (30) in, and These are pure voice signals. and residual noise signal In the The variance of each frequency component point; in equation (30) and Its estimated value can be obtained through simple iterative operations between the previous and next frames; it can be calculated using equation (29). After that, with For amplitude, with For the phase, perform an inverse fast Fourier transform (IFFT) of the frame length points to obtain the th... The enhanced speech of each window is obtained by superimposing and adding the enhanced speech of all windows together.

[0044] One embodiment of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the speech enhancement method based on a planar four-element microphone array provided in one embodiment of this application.

[0045] Figure 5The diagram illustrates a specific structural block diagram of an electronic device provided in an embodiment of this application. An electronic device 100 includes: one or more processors 101, a memory 102, and one or more computer programs. The processors 101 and the memory 102 are connected via a bus. The one or more computer programs are stored in the memory 102 and configured to be executed by the one or more processors 101. When the processors 101 execute the computer programs, they implement the steps of the speech enhancement method based on a planar four-element microphone array provided in an embodiment of this application.

[0046] The electronic device may be a smart speaker, headphones, or hearing aid.

[0047] In this application, four noisy speech signals are weighted and differentially analyzed pairwise to obtain four differential noise signals. Then, differential beamforming is performed on each pair of the four noisy speech signals to obtain four enhanced differential speech signals. These enhanced differential speech signals are then subjected to summation-average beamforming to obtain a speech enhancement signal with further noise attenuation. Finally, a modified multi-channel adaptive noise cancellation (MANC) algorithm based on voice activity detection (VAD) is applied to the speech enhancement signal and the four differential noise signals to further eliminate residual noise in the speech enhancement signal, resulting in a re-enhanced speech signal. This re-enhanced speech signal is then restored using a time-domain recovery algorithm to obtain a normal enhanced speech signal. Therefore, the speech enhancement effect is excellent.

[0048] It should be understood that the steps in the various embodiments of this application are not necessarily executed sequentially according to the order indicated by the step numbers. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in each embodiment may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0049] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0050] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0051] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A speech enhancement method based on a planar four-element microphone array, characterized in that, The method includes the following steps: S101. Collect the signals received by the four microphones of the planar four-element microphone array to obtain four noisy speech signals. S102. Perform differential beamforming on each pair of the four noisy speech signals to obtain four enhanced differential speech signals. S103. Perform summation and average beamforming processing on the four enhanced differential speech signals to obtain a speech enhancement signal with further noise attenuation. S104. Perform weighted differential operations on each pair of the four noisy speech signals to obtain four differential noise signals. S105. By applying the improved multi-channel adaptive noise cancellation (MANC) algorithm based on sound activity detection (VAD) to the speech enhancement signal and the four differential noise signals, the residual noise in the speech enhancement signal is further eliminated, and a re-enhanced speech signal is obtained. S106. The enhanced speech signal is restored using a time-domain recovery algorithm to obtain a normal enhanced speech signal.

2. The method as described in claim 1, characterized in that, Specifically, S101 is: Suppose that the noisy speech signal obtained by sampling the i-th microphone of the planar four-element microphone array at sampling time n is: The voice signal is The interference noise signal is ,but (2) Ideally, the voice source is in front of the wearer of the electronic device, and the noise source is to the wearer's left or right. (3) (4) The side length of the side parallel to the direction of the speech source in the rectangle formed by the planar four-element microphone array. hour, (5)； in, The sampling frequency of the speech signal. The speed at which sound travels through the air. For the speech signal, the first microphone To the microphone spread Signal amplitude attenuation factor at distance, ; S104 specifically includes: By any two noisy speech signals Weighted differential operation is performed to obtain one differential noise signal. : （11） （12） （13） （14）。 3. The method as described in claim 2, characterized in that, S102 specifically includes: By any two noisy speech signals A differential operation is performed to obtain an enhanced differential speech signal. , : (6) (7) (8) (9)。 4. The method as described in claim 3, characterized in that, S103 specifically uses the following formula to calculate the speech enhancement signal. : （10）。 5. The method as described in claim 4, characterized in that, Specifically, S105 is: In the VAD-based MANC algorithm, the main input signal is the speech enhancement signal. The multiple reference input signals are four differential noise signals. To obtain an enhanced speech signal VAD enhances speech signals The algorithm detects periods of sound or silence and uses the detection results to control whether to update the coefficients of the adaptive filter in the MINC algorithm. That is, the update is only performed during silent periods. The coefficients of the adaptive filter are implemented using the Least Mean Square (LMS) algorithm.

6. The method as described in claim 5, characterized in that, The specific formula for S105 is as follows: Using MINC's adaptive algorithm, therefore （15） （16） （17） （18） in, Differential noise signal A vector composed of values ​​taken at different times; The output signal of the adaptive filter; The error signal is derived from the speech enhancement signal. and the output signal of the adaptive filter The result of subtraction is a further enhanced speech signal; Let's consider the coefficient vector of the adaptive filter. The number of delayed sampling points for each input signal of the adaptive filter is determined by adjusting the coefficient vector of the adaptive filter. To make the error signal The mean squared expected value is minimized; This represents the update step size of the adaptive filter coefficients.

7. The method as described in claim 6, characterized in that, S106 specifically refers to: For the enhanced speech signal An L-order time-domain finite impulse response (FIR) filtering algorithm is used for the reconstruction operation to obtain the normal enhanced speech signal at output. ; For the enhanced speech signal The recovery operation is performed using an L-order time-domain finite impulse response (FIR) filtering algorithm as follows: Taking the Z-transform of equation (10) yields (19) Equation (20) can be obtained from equation (19). (20) Transforming equation (20) from the Z-domain to the time domain, and when the positive integer L is sufficiently large, we have （21） Pick （22） （23） but （24） in, Let L be the coefficient vector of the recovery filter, and L be the order of the recovery filter. against Recovery filtering, i.e. (25) in （26）。 8. The method as described in claim 1, characterized in that, Following S106, the following is also included: The normal enhanced speech signal is further enhanced using a single-channel speech enhancement algorithm to obtain the final enhanced speech signal.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the speech enhancement method based on a planar four-element microphone array as described in any one of claims 1 to 8.

10. An electronic device, comprising: One or more processors; Memory; as well as One or more computer programs, wherein the processor and the memory are connected via a bus, wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, characterized in that, when the processor executes the computer programs, it implements the steps of the speech enhancement method based on a planar four-element microphone array as described in any one of claims 1 to 8.

Images