Self-adaptive differential beam forming method and system based on Kalman filtering

Abstract

The invention provides an adaptive differential beam forming method and system based on Kalman filtering. The method comprises the following steps: acquiring a forward beam and a backward beam; inputting the forward wave beam and the backward wave beam into a preset Kalman filter, so that the Kalman filter updates a current first Kalman filtering parameter group according to the forward wave beam and the backward wave beam to obtain a second Kalman filtering parameter group, according to the second Kalman filter parameter set, differential beam forming frequency domain signals corresponding to the forward beam and the backward beam are generated, and the Kalman filter is constructed and obtained based on a neural network; and converting the differential beam forming frequency domain signal into a corresponding time domain voice signal to be output, thereby improving the accuracy and adaptability of differential beam forming.

Description

Technical Field

[0001] The present application relates to the field of signal processing technology, and in particular to an adaptive differential beamforming method and system based on Kalman filtering. Background Art

[0002] Adaptive Differential Beamforming (ADBF) is a technology widely used in antenna arrays and Multi Input Multi Output (MIMO) systems, and can also be used in audio signal processing. ADBF enhances the target signal while suppressing interference signals by adaptively adjusting the amplitude and phase of the received signal. Currently, ADBF mainly uses the Least Mean Square (LMS) or Normalized Least Mean Square (NLMS) algorithm for adaptive adjustment. The LMS and NLMS algorithms are simple and practical, but in some cases, such as when the surrounding environment is complex or the noise environment changes greatly, the performance of LMS and NLMS may deteriorate.

[0003] Kalman filter is a filter based on state space model, which provides the optimal estimation of system state by fusing the dynamic model and observation data of the system. The state space model of Kalman filter consists of state equation and observation equation, where the state equation describes the evolution of system state and the observation equation describes the relationship between data and system state. Kalman filter assumes that system state and observation data are linear and satisfy Gaussian distribution. The core idea of ​​Kalman filter is to predict state estimation by using state prediction equation and covariance prediction equation through prediction and update steps, and then correct state estimation by Kalman gain calculation, state correction equation and covariance correction equation. Kalman filter has good robustness and adaptability, and can adapt to different environments and noise conditions by adjusting system model and covariance matrix. Kalman filter has a wide range of applications in estimation problems, especially in control systems, signal processing and navigation. Summary of the invention

[0004] In response to the above technical problems, the present application provides an adaptive differential beamforming method and system based on Kalman filtering to improve the accuracy and adaptability of differential beamforming.

[0005] In a first aspect, the present application provides an adaptive differential beamforming method based on Kalman filtering, comprising:

[0006] Acquire forward beam and backward beam;

[0007] Inputting the forward beam and the backward beam into a preset Kalman filter, so that the Kalman filter updates the current first Kalman filter parameter group according to the forward beam and the backward beam, obtains a second Kalman filter parameter group, and then generates a differential beamforming frequency domain signal corresponding to the forward beam and the backward beam according to the second Kalman filter parameter group, wherein the Kalman filter is obtained based on a neural network;

[0008] The differential beamforming frequency domain signal is converted into a corresponding time domain speech signal output.

[0009] The embodiment of the present application provides an adaptive differential beamforming method based on Kalman filtering, wherein the forward beam and the backward beam are processed by the Kalman filter to generate a corresponding differential beamforming frequency domain signal, and the differential beamforming frequency domain signal is converted into a corresponding time domain speech signal output. Each time a new forward beam and a backward beam are input to the Kalman filter, the Kalman filter updates the current first Kalman filter parameter group according to the forward beam and the backward beam, obtains a second Kalman filter parameter group, and generates a corresponding differential beamforming frequency domain signal according to the second Kalman filter parameter group. Compared with the prior art, the embodiment of the present application uses the Kalman filter as an alternative to the adaptive filter. The Kalman filter can more accurately suppress the noise in the interference direction and provide a better quality beamforming effect. In addition, the Kalman filter in the embodiment of the present application is obtained based on the construction of a neural network, combining the nonlinear modeling capability of the neural network and the real-time signal processing capability of the Kalman filter, realizing the adaptive update of the Kalman filter, further improving the suppression effect of echo and nonlinear distortion, and improving the accuracy and adaptability of differential beamforming.

[0010] Further, the acquiring of the forward beam and the backward beam includes:

[0011] Acquire a first time domain signal through a first microphone;

[0012] Acquire a second time domain signal through a second microphone;

[0013] Converting the first time domain signal and the second time domain signal into corresponding first frequency domain signals and second frequency domain signals respectively through a Hanning window and a fast Fourier transform;

[0014] According to the time delay between the first microphone and the second microphone, the first frequency domain signal and the second frequency domain signal are respectively converted into the corresponding forward beam and the backward beam.

[0015] The embodiment of the present application provides a method for acquiring a forward beam and a backward beam, wherein a first time domain signal and a second time domain signal are acquired respectively by a first microphone and a second microphone, and then the first time domain signal and the second time domain signal received by the microphone are transformed into the frequency domain through a Hanning window and a fast Fourier transform (FFT), and converted into corresponding first frequency domain signals and second frequency domain signals, so as to facilitate subsequent processing. Finally, according to the time delay between the first microphone and the second microphone, the first frequency domain signal and the second frequency domain signal are respectively converted into the corresponding forward beam and the backward beam, realizing the conversion from the time domain signal to the forward beam and the backward beam, and providing a data basis for subsequent differential beamforming. In practical applications, omnidirectional microphones greater than or equal to 2 are usually placed in a certain geometric structure to construct a microphone array to acquire time domain signals. In differential beamforming, the spacing between microphones has a certain influence on the performance of the system. Differential beamforming realizes signal enhancement and interference suppression by introducing time delay and weighting in the microphone array, and the microphone spacing determines the time delay difference of the signal reaching different microphones. Typically, the spacing between microphones should be small enough to ensure that the phase difference of the signals does not exceed half the wavelength in the frequency range of interest, because when the phase difference exceeds half the wavelength, the phase difference of the signal will cause aliasing in the beamforming process, resulting in the attenuation of the beamforming effect.

[0016] In a possible implementation manner, the Kalman filter updates a current first Kalman filter parameter group according to the forward beam and the backward beam to obtain a second Kalman filter parameter group, and then generates a differential beamforming frequency domain signal corresponding to the forward beam and the backward beam according to the second Kalman filter parameter group, including:

[0017] Generate a first differential beamforming frequency domain signal according to the first Kalman filter parameter group, the forward beam and the backward beam, wherein the first Kalman filter parameter group includes a first transfer factor, a first far-end nonlinear distortion function, a first nonlinear transfer function, a first state covariance matrix, a first Kalman filter gain and a first beam weight parameter;

[0018] Inputting the first differential beamforming frequency domain signal into the neural network, so that the neural network generates a corresponding second transfer factor, a second far-end nonlinear distortion function, and a second nonlinear transfer function according to the first differential beamforming frequency domain signal;

[0019] updating the first state covariance matrix, the first Kalman filter gain, and the first beam weight parameter according to the second transfer factor to obtain a second Kalman filter parameter group;

[0020] A differential beamforming frequency domain signal corresponding to the forward beam and the backward beam is generated according to the second Kalman filter parameter group.

[0021] An embodiment of the present application provides a method for updating a Kalman filter parameter group and generating a differential beamforming frequency domain signal. In practical applications, sound signals are continuously acquired and there is great similarity between the sound signal of the previous frame and the sound signal of the current frame. Therefore, the current first Kalman filter parameter group can be used to generate a first differential beamforming frequency domain signal according to the forward beam and the backward beam, and then the first Kalman filter parameter group is updated by the first differential beamforming frequency domain signal to obtain a second Kalman filter parameter group. In this process, the estimation function of the neural network is utilized, and the first differential beamforming frequency domain signal is input into the neural network to estimate some parameters in the first Kalman filter parameter group, generate the corresponding second transfer factor, the second far-end nonlinear distortion function and the second nonlinear transfer function, and further update the remaining parameters in the first Kalman filter parameter group through the second transfer factor to obtain the second Kalman filter parameter group. Finally, according to the second Kalman filter parameter group, the differential beamforming frequency domain signal corresponding to the forward beam and the backward beam is generated, and the real-time update of the Kalman filter parameter group is realized while performing differential beamforming, thereby improving the accuracy and adaptability of differential beamforming.

[0022] Further, the updating of the first state covariance matrix, the first Kalman filter gain and the first beam weight parameter according to the second transfer factor to obtain a second Kalman filter parameter group includes:

[0023] updating the first state covariance matrix according to the second transfer factor and the backward beam to obtain a second state covariance matrix;

[0024] Update the first Kalman filter gain according to the second state covariance matrix and the backward beam to obtain a second Kalman filter gain;

[0025] updating the first beam weight parameter according to the second Kalman filter gain and the first differential beamforming frequency domain signal to obtain a second beam weight parameter;

[0026] The second transfer factor, the second far-end nonlinear distortion function, the second nonlinear transfer function, the second beam weight parameter, the second Kalman filter gain and the second state covariance matrix are constructed to obtain the second Kalman filter parameter group.

[0027] Further, generating a differential beamforming frequency domain signal corresponding to the forward beam and the backward beam according to the second Kalman filter parameter group includes:

[0028] generating a nonlinearly distorted far-end signal according to the second far-end nonlinear distortion function and the backward beam;

[0029] generating a far-end signal according to the second nonlinear transfer function and the second beam weight parameter;

[0030] A differential beamforming frequency domain signal corresponding to the forward beam and the backward beam is generated according to the nonlinearly distorted far-end signal, the far-end signal, and the forward beam.

[0031] Further, the differential beamforming frequency domain signal corresponding to the forward beam and the backward beam is generated according to the nonlinearly distorted far-end signal, the far-end signal and the forward beam. The specific formula is:

[0032] z(k)=c f (k)-t(We(k))*Xe(k)

[0033] Wherein, z(k) is the frequency domain signal of the differential beamforming, c f (k) is the forward beam, We(k) is the second beam weight parameter, t(We(k)) is the far-end signal, and Xe(k) is the nonlinearly distorted far-end signal.

[0034] In a possible implementation, converting the differential beamforming frequency domain signal into a corresponding time domain speech signal output includes:

[0035] Using a preset low-pass filter to perform low-pass compensation filtering on the differential beamforming frequency domain signal, and converting the differential beamforming frequency domain signal into a low-pass filtered frequency domain signal;

[0036] Performing an inverse Fourier transform on the low-pass filtered frequency domain signal to convert the low-pass filtered frequency domain signal into a low-pass filtered time domain signal;

[0037] The low-pass filtered time domain signal is subjected to windowing and overlapping storage operations, and the low-pass filtered time domain signal is converted into the time domain speech signal and outputted.

[0038] The embodiment of the present application provides a method for converting the differential beamforming frequency domain signal into a corresponding time domain speech signal. Since the differential operation has a small response to the low-frequency signal and a large response to the high-frequency signal, the adaptive differential beamforming has a high-pass feature. In order to achieve a distortion-free response of the differential beam at the target angle, that is, unit gain and no phase shift, a low-pass filter is required to further low-pass compensate the signal after the differential operation. In the embodiment of the present application, a preset low-pass filter is used to perform low-pass compensation filtering on the differential beamforming frequency domain signal, and the differential beamforming frequency domain signal is converted into a low-pass filtered frequency domain signal. Finally, the low-pass filtered frequency domain signal after low-pass filtering is converted from the frequency domain back to the time domain using an inverse Fourier transform (IFFT), and the final time domain speech signal is generated after windowing and overlapping storage (Overlap and Save) operations. The embodiment of the present application realizes a distortion-free response of the differential beam at the target angle by performing signal processing on the differential beamforming frequency domain signal, thereby improving the accuracy of the differential beamforming.

[0039] In a second aspect, accordingly, the present application provides an adaptive differential beamforming system based on Kalman filtering, comprising an acquisition module, a filtering module and an output module;

[0040] Wherein, the acquisition module is used to acquire the forward beam and the backward beam;

[0041] The filtering module is used to input the forward beam and the backward beam into a preset Kalman filter, so that the Kalman filter updates the current first Kalman filter parameter group according to the forward beam and the backward beam, obtains the second Kalman filter parameter group, and then generates a differential beamforming frequency domain signal corresponding to the forward beam and the backward beam according to the second Kalman filter parameter group, wherein the Kalman filter is obtained based on a neural network construction;

[0042] The output module is used to convert the differential beamforming frequency domain signal into a corresponding time domain speech signal for output.

[0043] Further, the filtering module includes a first difference unit, a generation unit, a parameter updating unit and a second difference unit;

[0044] The first differential unit is used to generate a first differential beamforming frequency domain signal according to the first Kalman filter parameter group, the forward beam and the backward beam, wherein the first Kalman filter parameter group includes a first transfer factor, a first far-end nonlinear distortion function, a first nonlinear transfer function, a first state covariance matrix, a first Kalman filter gain and a first beam weight parameter;

[0045] The generating unit is used to input the first differential beamforming frequency domain signal into the neural network, so that the neural network generates a corresponding second transfer factor, a second far-end nonlinear distortion function and a second nonlinear transfer function according to the first differential beamforming frequency domain signal;

[0046] The parameter updating unit is used for updating the first state covariance matrix, the first Kalman filter gain and the first beam weight parameter according to the second transfer factor to obtain a second Kalman filter parameter group;

[0047] The second differential unit is used to generate a differential beamforming frequency domain signal corresponding to the forward beam and the backward beam according to the second Kalman filter parameter group.

[0048] Further, the parameter updating unit updates the first state covariance matrix, the first Kalman filter gain and the first beam weight parameter according to the second transfer factor to obtain a second Kalman filter parameter group, including:

[0049] updating the first state covariance matrix according to the second transfer factor and the backward beam to obtain a second state covariance matrix;

[0050] Update the first Kalman filter gain according to the second state covariance matrix and the backward beam to obtain a second Kalman filter gain;

[0051] updating the first beam weight parameter according to the second Kalman filter gain and the first differential beamforming frequency domain signal to obtain a second beam weight parameter;

[0052] The second transfer factor, the second far-end nonlinear distortion function, the second nonlinear transfer function, the second beam weight parameter, the second Kalman filter gain and the second state covariance matrix are constructed to obtain the second Kalman filter parameter group. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 : A flow chart of an adaptive differential beamforming method based on Kalman filtering provided in an embodiment of the present application.

[0054] Figure 2 : A waveform diagram of a forward beam in an adaptive differential beamforming method based on Kalman filtering provided in an embodiment of the present application.

[0055] Figure 3 : A waveform diagram of a backward beam in an adaptive differential beamforming method based on Kalman filtering provided in an embodiment of the present application.

[0056] Figure 4: A schematic diagram of the signal processing flow of an adaptive differential beamforming method based on Kalman filtering provided in an embodiment of the present application.

[0057] Figure 5 : A schematic diagram of the Kalman filtering process in an adaptive differential beamforming method based on Kalman filtering provided in an embodiment of the present application.

[0058] Figure 6 : A schematic diagram of the internal structure of a Kalman filter in an adaptive differential beamforming method based on Kalman filtering provided in an embodiment of the present application.

[0059] Figure 7 : It is a schematic diagram comparing the implementation effects of the embodiments of the present application with those of the prior art.

[0060] Figure 8 : A structural diagram of an adaptive differential beamforming system based on Kalman filtering provided in an embodiment of the present application.

[0061] Fig. 9 : A structural schematic diagram of a filtering module in an adaptive differential beamforming system based on Kalman filtering provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0063] It should be noted that the step numbers in the text are only for the convenience of explanation of the specific embodiments and do not serve to limit the order in which the steps are executed. In the description of this application, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0064] Embodiment 1:

[0065] like Figure 1 As shown, embodiment 1 provides an adaptive differential beamforming method based on Kalman filtering, including steps S1-S3:

[0066] Step S1, obtaining a forward beam and a backward beam;

[0067] Step S2, inputting the forward beam and the backward beam into a preset Kalman filter, so that the Kalman filter updates the current first Kalman filter parameter group according to the forward beam and the backward beam, obtains a second Kalman filter parameter group, and then generates a differential beamforming frequency domain signal corresponding to the forward beam and the backward beam according to the second Kalman filter parameter group, wherein the Kalman filter is obtained based on a neural network construction;

[0068] Step S3: convert the differential beamforming frequency domain signal into a corresponding time domain speech signal for output.

[0069] The embodiment of the present application provides an adaptive differential beamforming method based on Kalman filtering, wherein the forward beam and the backward beam are processed by the Kalman filter to generate a corresponding differential beamforming frequency domain signal, and the differential beamforming frequency domain signal is converted into a corresponding time domain speech signal output. Each time a new forward beam and a backward beam are input to the Kalman filter, the Kalman filter updates the current first Kalman filter parameter group according to the forward beam and the backward beam, obtains a second Kalman filter parameter group, and generates a corresponding differential beamforming frequency domain signal according to the second Kalman filter parameter group. Compared with the prior art, the embodiment of the present application uses the Kalman filter as an alternative to the adaptive filter. The Kalman filter can more accurately suppress the noise in the interference direction and provide a better quality beamforming effect. In addition, the Kalman filter in the embodiment of the present application is obtained based on the construction of a neural network, combining the nonlinear modeling capability of the neural network and the real-time signal processing capability of the Kalman filter, realizing the adaptive update of the Kalman filter, further improving the suppression effect of echo and nonlinear distortion, and improving the accuracy and adaptability of differential beamforming.

[0070] Further, in step S1, the acquiring of the forward beam and the backward beam includes:

[0071] Acquire a first time domain signal through a first microphone;

[0072] Acquire a second time domain signal through a second microphone;

[0073] Converting the first time domain signal and the second time domain signal into corresponding first frequency domain signals and second frequency domain signals respectively through a Hanning window and a fast Fourier transform;

[0074] According to the time delay between the first microphone and the second microphone, the first frequency domain signal and the second frequency domain signal are respectively converted into the corresponding forward beam and the backward beam.

[0075] In a preferred embodiment, taking a unilateral hearing aid as an example, each unilateral hearing aid uses two microphones, which are arranged in a straight line with a distance of 7 mm (millimeter). Considering the difference in the wearing method of the hearing aid, the line where the two microphones are located may be at a certain angle to the horizontal plane, but generally not far from the horizontal plane. According to the arrangement structure of the microphone array, the time delay of the output signals of the two microphones is obtained. When the angle between the incident direction of the target signal source and the axis of the microphone is 0 degrees, the time delay of the two microphones receiving the signal is the largest. The time delay calculation formula is: τ = d / c, where d is the distance between the two microphones. According to the previous design, d = 7mm, and the sound propagation speed is 334m / s when c. The time domain signal received by the microphone is transformed into the frequency domain for subsequent processing. The digital signal sampling rate of the microphone is 16kHz. We set the frame length to 128 points (8 milliseconds). The specific frame length setting can be set according to the system requirements. The frame shift is set to 64 points (4 milliseconds), that is, 64 samples are moved each time to generate a new frame of data, that is, there will be 64 samples overlapped between two adjacent frames. Then, a window function is added to each frame signal, and we use the Hanning window. The audio signal after frame division and windowing is transformed from the time domain to the frequency domain through the fast Fourier transform (FFT) to obtain the spectrum representation of each frame. The output after FFT is a complex number of 65 frequency points, and the amplitude and phase of the frequency domain signal can be obtained. According to the two frequency domain signals and time delay after FFT transformation, the signals after the forward beam and the backward beam are calculated according to the following formulas:

[0076] c f (k) = Y1(k) - e -j2πkT Y2(k)

[0077] c b (k) = Y2(k) - e -j2πkT Y1(k)

[0078] Among them, c f (k) is the forward beam, c b (k) is the backward beam, Y1(k) is the first frequency domain signal corresponding to the first microphone, Y2(k) is the second frequency domain signal corresponding to the second microphone, k represents the frequency index, j represents a complex number, and T represents the time delay between microphones.

[0079] The waveform diagram of the forward beam is as follows: Figure 2 As shown in the figure, the gain at 0 degrees is 1, keeping the sound at 0 degrees unsuppressed, while a zero point is formed at 180 degrees, the sound is completely suppressed, and the beam pattern is similar to a heart shape, retaining the speech between -90 and 90 degrees as much as possible, and suppressing the speech between 90 and 270 degrees as much as possible. The waveform diagram of the backward beam pattern is shown in Figure 3As shown, the waveform diagram of the forward beam is exactly the opposite, the speech is completely retained in the 180 degree direction, and a null point is formed in the 0 degree direction.

[0080] In a preferred embodiment, Figure 4 As shown, two time domain signals are acquired through two microphones and processed and converted into a forward beam and a backward beam, and then the forward beam and the backward beam are input into the Kalman filter provided in the present application to obtain the final time domain speech signal.

[0081] The embodiment of the present application provides a method for acquiring a forward beam and a backward beam, wherein a first time domain signal and a second time domain signal are acquired respectively by a first microphone and a second microphone, and then the first time domain signal and the second time domain signal received by the microphone are transformed into the frequency domain through a Hanning window and a fast Fourier transform (FFT), and converted into corresponding first frequency domain signals and second frequency domain signals, so as to facilitate subsequent processing. Finally, according to the time delay between the first microphone and the second microphone, the first frequency domain signal and the second frequency domain signal are respectively converted into the corresponding forward beam and the backward beam, realizing the conversion from the time domain signal to the forward beam and the backward beam, and providing a data basis for subsequent differential beamforming. In practical applications, omnidirectional microphones greater than or equal to 2 are usually placed in a certain geometric structure to construct a microphone array to acquire time domain signals. In differential beamforming, the spacing between microphones has a certain influence on the performance of the system. Differential beamforming realizes signal enhancement and interference suppression by introducing time delay and weighting in the microphone array, and the microphone spacing determines the time delay difference of the signal reaching different microphones. Typically, the spacing between microphones should be small enough to ensure that the phase difference of the signals does not exceed half the wavelength in the frequency range of interest, because when the phase difference exceeds half the wavelength, the phase difference of the signal will cause aliasing in the beamforming process, resulting in the attenuation of the beamforming effect.

[0082] After obtaining the forward beam and the backward beam, a beam pattern that generates a null point in any direction in the rear hemisphere (i.e., between 90 degrees and 270 degrees) can be formed by combining the forward beam and the backward beam. The angular position of the null point depends on w. The formula for adaptive beamforming is as follows:

[0083] z(k)=c f (k)-w(k)cb(k)

[0084] By adjusting the w value, the power spectrum of the output of the adaptive beamforming can be minimized. Therefore, the above formula can be regarded as minimizing the mean square error by adjusting w. The prior art usually uses adaptive algorithms such as LMS or NLMS to update w. LMS-type algorithms only update w based on the error between the input and output signals, which may be affected by signal and noise transformation in a non-stationary environment, resulting in a slower convergence speed or failure to achieve the expected beamforming effect. Therefore, in this application, we use a neural network-based Kalman filter to update w and other Kalman filter parameter groups:

[0085] The covariance matrix calculation formula of the error spectrum is:

[0086]

[0087] ZZ Z.z H

[0088] The update formula for each parameter in the Kalman filter parameter group is:

[0089]

[0090] W t+1 =W t +K*z

[0091]

[0092] Among them, K represents the Kalman filter gain, P represents the state covariance matrix, W represents the beam weight parameter, and A represents the transfer factor.

[0093] In one possible implementation, in step S2, the Kalman filter updates the current first Kalman filter parameter group according to the forward beam and the backward beam to obtain a second Kalman filter parameter group, and then generates a differential beamforming frequency domain signal corresponding to the forward beam and the backward beam according to the second Kalman filter parameter group, such as Figure 5 As shown, steps S201-S204 are included:

[0094] Step S201, generating a first differential beamforming frequency domain signal according to the first Kalman filter parameter group, the forward beam and the backward beam, wherein the first Kalman filter parameter group includes a first transfer factor, a first far-end nonlinear distortion function, a first nonlinear transfer function, a first state covariance matrix, a first Kalman filter gain and a first beam weight parameter;

[0095] Step S202: inputting the first differential beamforming frequency domain signal into the neural network, so that the neural network generates a corresponding second transfer factor, a second far-end nonlinear distortion function, and a second nonlinear transfer function according to the first differential beamforming frequency domain signal;

[0096] Step S203: updating the first state covariance matrix, the first Kalman filter gain and the first beam weight parameter according to the second transfer factor to obtain a second Kalman filter parameter group;

[0097] Step S204: Generate a differential beamforming frequency domain signal corresponding to the forward beam and the backward beam according to the second Kalman filter parameter group.

[0098] An embodiment of the present application provides a method for updating a Kalman filter parameter group and generating a differential beamforming frequency domain signal. In practical applications, sound signals are continuously acquired and there is great similarity between the sound signal of the previous frame and the sound signal of the current frame. Therefore, the current first Kalman filter parameter group can be used to generate a first differential beamforming frequency domain signal according to the forward beam and the backward beam, and then the first Kalman filter parameter group is updated by the first differential beamforming frequency domain signal to obtain a second Kalman filter parameter group. In this process, the estimation function of the neural network is utilized, and the first differential beamforming frequency domain signal is input into the neural network to estimate some parameters in the first Kalman filter parameter group, generate the corresponding second transfer factor, the second far-end nonlinear distortion function and the second nonlinear transfer function, and further update the remaining parameters in the first Kalman filter parameter group through the second transfer factor to obtain the second Kalman filter parameter group. Finally, according to the second Kalman filter parameter group, the differential beamforming frequency domain signal corresponding to the forward beam and the backward beam is generated, and the real-time update of the Kalman filter parameter group is realized while performing differential beamforming, thereby improving the accuracy and adaptability of differential beamforming.

[0099] In a preferred embodiment, the parameters in the Kalman parameter group are first initialized: w is initialized to 0 so that its zero point direction points to 180 degrees, P is randomly initialized to a value between 0.5 and 0.9, and A^2 is set to 0.6.

[0100] In a preferred embodiment, in step S202, the present application uses a neural network to estimate the transfer factor A, the far-end nonlinear distortion function g() and the nonlinear transfer function t(), such as Figure 6As shown. Among them, the transfer factor A is a key parameter in the Kalman filter state equation, reflecting the change of the filter state. Traditional FDKF (frequency domain Kalman filter) usually uses a fixed transfer factor, which may not be flexible enough when the environment changes. By using a neural network to estimate a time-varying transfer factor, the system can better adapt to sudden changes in the echo path and improve the stability and convergence of the filter. The far-end signal introduces nonlinear distortion when it is transmitted through the speaker. If this nonlinear distortion is not processed, it will affect the effect of echo cancellation. By using a neural network to estimate this nonlinear distortion, this distortion can be more accurately modeled and compensated, thereby improving the effect of echo cancellation. In traditional Kalman filters, the state update equation is usually linear. However, the echo path in practical applications may be nonlinear. By introducing a nonlinear transfer function t estimated by a neural network, the nonlinear change of the state can be better modeled, thereby improving the performance and training speed of the filter. These improvements enable the Kalman model to handle nonlinear distortion and environmental changes more effectively. Since the echo path and nonlinear distortion information can be retrieved from the microphone and far-end signals, the transfer factor A and nonlinear distortion g(·) are estimated using the input features of the complex STFT calculation of the microphone and far-end signals. Figure 6 As shown, a recurrent neural network (GRU) obtains the calculated input Mel features and then divides into two branches to estimate the nonlinearly distorted far-end signal Xe(k) and the transfer factor A(k), respectively. The shared RNN is a 4-layer long short-term memory (GRU) network with 128 hidden units in each layer. In this way, the framework combines the nonlinear modeling capabilities of neural networks and the real-time signal processing capabilities of Kalman filters, improving the suppression of echoes and nonlinear distortions.

[0101] Further, in step S203, the updating of the first state covariance matrix, the first Kalman filter gain and the first beam weight parameter according to the second transfer factor to obtain a second Kalman filter parameter group includes:

[0102] updating the first state covariance matrix according to the second transfer factor and the backward beam to obtain a second state covariance matrix;

[0103] Update the first Kalman filter gain according to the second state covariance matrix and the backward beam to obtain a second Kalman filter gain;

[0104] updating the first beam weight parameter according to the second Kalman filter gain and the first differential beamforming frequency domain signal to obtain a second beam weight parameter;

[0105] The second transfer factor, the second far-end nonlinear distortion function, the second nonlinear transfer function, the second beam weight parameter, the second Kalman filter gain and the second state covariance matrix are constructed to obtain the second Kalman filter parameter group.

[0106] Specifically, the transfer factor A in the range [0,1] describes the change of the Kalman filter and is usually manually adjusted to a value close to 1. In order to take into account the impact of possible changes in the echo path on the transfer factor, a neural network is used to estimate the time-varying transfer factor. The branch that estimates the frame-based transfer factor A(k) consists of a linear layer and a Sigmoid activation function. Here, a new second transfer factor A~ is obtained, and the first state covariance matrix is ​​updated by the following formula:

[0107]

[0108] Among them, P t+1 is the second state covariance matrix, P t is the first state covariance matrix, c b Then, the first Kalman filter gain is updated according to the second state covariance matrix and the backward beam to obtain the second Kalman filter gain, and the specific formula is:

[0109]

[0110]

[0111] Ψ ZZ Z.z H

[0112] Where z is the first differential beamforming frequency domain signal, K t+1 is the second Kalman filter gain.

[0113] The first beam weight parameter is updated according to the second Kalman filter gain and the first differential beamforming frequency domain signal to obtain a second beam weight parameter, and the specific formula is:

[0114] We t+1 =We t +K t+1 *z

[0115] Among them, We t is the first beam weight parameter, We t+1 is the second beam weight parameter.

[0116] Finally, the second transfer factor A, the second far-end nonlinear distortion function g(), the second nonlinear transfer function t(), and the second beam weight parameter W are obtained. t+1, the second Kalman filter gain K t+1 And the second state covariance matrix P t+1 .

[0117] Further, in step S204, generating a differential beamforming frequency domain signal corresponding to the forward beam and the backward beam according to the second Kalman filter parameter group includes:

[0118] generating a nonlinearly distorted far-end signal according to the second far-end nonlinear distortion function and the backward beam;

[0119] generating a far-end signal according to the second nonlinear transfer function and the second beam weight parameter;

[0120] A differential beamforming frequency domain signal corresponding to the forward beam and the backward beam is generated according to the nonlinearly distorted far-end signal, the far-end signal, and the forward beam.

[0121] Specifically, in order to solve the nonlinear distortion introduced by the speaker, we use a deep neural network (DNN) to estimate the far-end nonlinear distortion and use the nonlinear far-end signal as a reference signal to update the Kalman filter. Figure 6 As shown in Figure 1, the subnetwork responsible for estimating g() consists of a linear layer with a ReLU activation function and a one-dimensional convolutional layer. By applying a complex filter to the microphone signal c b (k), we can obtain the nonlinearly distorted far-end signal Xe(k), the specific formula is:

[0122] Xe(k)=c b (k).*g(k)

[0123] Where, Xe(k) is the nonlinear distortion far-end signal, c b (k) is the backward beam, g(k) is the second far-end nonlinear distortion function, and .* represents element-by-element multiplication.

[0124] Our observations also suggest that incorporating covariance modeling can improve performance. However, we find that introducing a nonlinear transfer function in the state equation can achieve similar performance gains with less computational effort. More specifically, we replace the linear transfer function with a nonlinear transfer function, i.e., Figure 6 The second beam weight parameter We t+1 It passes through a GRU network with 256 hidden units and outputs the far-end signal t(We(k)).

[0125] Finally, according to the nonlinear distortion far-end signal Xe(k), the far-end signal t(We(k)) and the forward beam c f(k), generating the differential beamforming frequency domain signal z(k):

[0126] z(k)=c f (k)-t(We(k))*Xe(k)

[0127] In a possible implementation, converting the differential beamforming frequency domain signal into a corresponding time domain speech signal output includes:

[0128] Using a preset low-pass filter to perform low-pass compensation filtering on the differential beamforming frequency domain signal, and converting the differential beamforming frequency domain signal into a low-pass filtered frequency domain signal;

[0129] Performing an inverse Fourier transform on the low-pass filtered frequency domain signal to convert the low-pass filtered frequency domain signal into a low-pass filtered time domain signal;

[0130] The low-pass filtered time domain signal is subjected to windowing and overlapping storage operations, and the low-pass filtered time domain signal is converted into the time domain speech signal and outputted.

[0131] In a preferred embodiment, the low-pass compensation filter formula is expressed as:

[0132]

[0133] z t+1 =H L *z t

[0134] Among them, eps is a very small value used to correct the filter response at the zero frequency point.

[0135] The embodiment of the present application provides a method for converting the differential beamforming frequency domain signal into a corresponding time domain speech signal. Since the differential operation has a small response to low-frequency signals and a large response to high-frequency signals, the adaptive differential beamforming has a high-pass feature. In order to achieve a distortion-free response of the differential beam at the target angle, that is, unit gain and no phase shift, a low-pass filter is required to further low-pass compensate the signal after the differential operation. In the embodiment of the present application, a preset low-pass filter is used to perform low-pass compensation filtering on the differential beamforming frequency domain signal, and the differential beamforming frequency domain signal is converted into a low-pass filtered frequency domain signal. Finally, the low-pass filtered frequency domain signal after low-pass filtering is converted from the frequency domain back to the time domain using an inverse Fourier transform (IFFT), and the final time domain speech signal is generated after windowing and overlapping storage (Overlap and Save) operations. The embodiment of the present application realizes a distortion-free response of the differential beam at the target angle by performing signal processing on the differential beamforming frequency domain signal, thereby improving the accuracy of the differential beamforming.

[0136] In a preferred embodiment, the loss function of the neural network is defined as jointly optimizing the mean absolute error (MAE) of the SI-SDR and the target signal and the estimated near-end signal amplitude spectrum in the time domain:

[0137] L=-SI-SDR(s,s^)+α*MAE(|S|,|S^|)

[0138] Among them, s^ and S^ are the estimated near-end signals in time domain and frequency domain, respectively. In order to balance the numerical range of the two losses, we set α to 10000 in our implementation.

[0139] SI-SDR, the full name of which is Scale-Invariant Signal-to-Distortion Ratio, is an evaluation index used to evaluate the performance of signal separation or enhancement algorithms. Its definition is as follows:

[0140]

[0141] Where s is the target signal, is the estimated signal, and α is a scaling factor such that αs and Specifically, the scaling factor α is determined by the following formula:

[0142]

[0143] The schematic diagram comparing the implementation effects of the present application embodiment and the prior art NLMS is shown in FIG. Figure 7 As shown, from Figure 7 It can be found that the use of the adaptive differential beamforming method provided in the present application can effectively suppress interference signals and enhance target signals, and compared with the prior art NLMS, the adaptive differential beamforming method provided in the present application achieves better results.

[0144] Embodiment 2:

[0145] like Figure 8 As shown, accordingly, embodiment 2 provides an adaptive differential beamforming system based on Kalman filtering, including an acquisition module 10, a filtering module 20 and an output module 30;

[0146] Wherein, the acquisition module 10 is used to acquire the forward beam and the backward beam;

[0147] The filtering module 20 is used to input the forward beam and the backward beam into a preset Kalman filter, so that the Kalman filter updates the current first Kalman filter parameter group according to the forward beam and the backward beam, obtains the second Kalman filter parameter group, and then generates a differential beamforming frequency domain signal corresponding to the forward beam and the backward beam according to the second Kalman filter parameter group, wherein the Kalman filter is obtained based on a neural network.

[0148] The output module 30 is used to convert the differential beamforming frequency domain signal into a corresponding time domain speech signal for output.

[0149] Further, such as Fig. 9 As shown, the filtering module 20 includes a first difference unit 201, a generation unit 202, a parameter updating unit 203 and a second difference unit 204;

[0150] The first differential unit 201 is used to generate a first differential beamforming frequency domain signal according to the first Kalman filter parameter group, the forward beam and the backward beam, wherein the first Kalman filter parameter group includes a first transfer factor, a first far-end nonlinear distortion function, a first nonlinear transfer function, a first state covariance matrix, a first Kalman filter gain and a first beam weight parameter;

[0151] The generating unit 202 is used to input the first differential beamforming frequency domain signal into the neural network, so that the neural network generates a corresponding second transfer factor, a second far-end nonlinear distortion function and a second nonlinear transfer function according to the first differential beamforming frequency domain signal;

[0152] The parameter updating unit 203 is used to update the first state covariance matrix, the first Kalman filter gain and the first beam weight parameter according to the second transfer factor to obtain a second Kalman filter parameter group;

[0153] The second difference unit 204 is used to generate a differential beamforming frequency domain signal corresponding to the forward beam and the backward beam according to the second Kalman filter parameter group.

[0154] Further, the parameter updating unit 203 updates the first state covariance matrix, the first Kalman filter gain and the first beam weight parameter according to the second transfer factor to obtain a second Kalman filter parameter group, including:

[0155] updating the first state covariance matrix according to the second transfer factor and the backward beam to obtain a second state covariance matrix;

[0156] Update the first Kalman filter gain according to the second state covariance matrix and the backward beam to obtain a second Kalman filter gain;

[0157] updating the first beam weight parameter according to the second Kalman filter gain and the first differential beamforming frequency domain signal to obtain a second beam weight parameter;

[0158] The second transfer factor, the second far-end nonlinear distortion function, the second nonlinear transfer function, the second beam weight parameter, the second Kalman filter gain and the second state covariance matrix are constructed to obtain the second Kalman filter parameter group.

[0159] The embodiment of the present application provides an adaptive differential beamforming system based on Kalman filtering, which processes the forward beam and the backward beam through the Kalman filter to generate a corresponding differential beamforming frequency domain signal, and converts the differential beamforming frequency domain signal into a corresponding time domain speech signal output. Each time a new forward beam and a backward beam are input to the Kalman filter, the Kalman filter will update the current first Kalman filter parameter group according to the forward beam and the backward beam, obtain the second Kalman filter parameter group, and generate the corresponding differential beamforming frequency domain signal according to the second Kalman filter parameter group. Compared with the prior art, the embodiment of the present application uses the Kalman filter as an alternative to the adaptive filter. The Kalman filter can more accurately suppress the noise in the interference direction and provide a better quality beamforming effect. In addition, the Kalman filter in the embodiment of the present application is obtained based on the construction of a neural network, combining the nonlinear modeling capability of the neural network and the real-time signal processing capability of the Kalman filter, realizing the adaptive update of the Kalman filter, further improving the suppression effect of echo and nonlinear distortion, and improving the accuracy and adaptability of differential beamforming.

[0160] The more detailed working principle and step flow of this embodiment can refer to, but are not limited to, the relevant records of Embodiment 1.

[0161] The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the scope of protection of the present application. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of protection of the present application.

Claims

1. An adaptive differential beamforming method based on Kalman filtering, characterized in that: include: Acquire forward beam and backward beam; Inputting the forward beam and the backward beam into a preset Kalman filter, so that the Kalman filter updates the current first Kalman filter parameter group according to the forward beam and the backward beam, obtains a second Kalman filter parameter group, and then generates a differential beamforming frequency domain signal corresponding to the forward beam and the backward beam according to the second Kalman filter parameter group, wherein the Kalman filter is obtained based on a neural network; The differential beamforming frequency domain signal is converted into a corresponding time domain speech signal output.

2. The adaptive differential beamforming method based on Kalman filtering according to claim 1, characterized in that: The acquiring of the forward beam and the backward beam comprises: Acquire a first time domain signal through a first microphone; Acquire a second time domain signal through a second microphone; Converting the first time domain signal and the second time domain signal into corresponding first frequency domain signals and second frequency domain signals respectively through a Hanning window and a fast Fourier transform; According to the time delay between the first microphone and the second microphone, the first frequency domain signal and the second frequency domain signal are respectively converted into the corresponding forward beam and the backward beam.

3. The adaptive differential beamforming method based on Kalman filtering according to claim 1, characterized in that: The Kalman filter updates a current first Kalman filter parameter group according to the forward beam and the backward beam to obtain a second Kalman filter parameter group, and then generates a differential beamforming frequency domain signal corresponding to the forward beam and the backward beam according to the second Kalman filter parameter group, including: Generate a first differential beamforming frequency domain signal according to the first Kalman filter parameter group, the forward beam and the backward beam, wherein the first Kalman filter parameter group includes a first transfer factor, a first far-end nonlinear distortion function, a first nonlinear transfer function, a first state covariance matrix, a first Kalman filter gain and a first beam weight parameter; Inputting the first differential beamforming frequency domain signal into the neural network, so that the neural network generates a corresponding second transfer factor, a second far-end nonlinear distortion function, and a second nonlinear transfer function according to the first differential beamforming frequency domain signal; updating the first state covariance matrix, the first Kalman filter gain, and the first beam weight parameter according to the second transfer factor to obtain a second Kalman filter parameter group; A differential beamforming frequency domain signal corresponding to the forward beam and the backward beam is generated according to the second Kalman filter parameter group.

4. The adaptive differential beamforming method based on Kalman filtering according to claim 3, characterized in that: The updating of the first state covariance matrix, the first Kalman filter gain, and the first beam weight parameter according to the second transfer factor to obtain a second Kalman filter parameter group includes: updating the first state covariance matrix according to the second transfer factor and the backward beam to obtain a second state covariance matrix; Update the first Kalman filter gain according to the second state covariance matrix and the backward beam to obtain a second Kalman filter gain; updating the first beam weight parameter according to the second Kalman filter gain and the first differential beamforming frequency domain signal to obtain a second beam weight parameter; The second Kalman filter parameter group is obtained by constructing the second transfer factor, the second far-end nonlinear distortion function, the second nonlinear transfer function, the second beam weight parameter, the second Kalman filter gain and the second state covariance matrix.

5. The adaptive differential beamforming method based on Kalman filtering according to claim 4, characterized in that: The step of generating, according to the second Kalman filter parameter group, a differential beamforming frequency domain signal corresponding to the forward beam and the backward beam comprises: generating a nonlinearly distorted far-end signal according to the second far-end nonlinear distortion function and the backward beam; generating a far-end signal according to the second nonlinear transfer function and the second beam weight parameter; A differential beamforming frequency domain signal corresponding to the forward beam and the backward beam is generated according to the nonlinearly distorted far-end signal, the far-end signal, and the forward beam.

6. The adaptive differential beamforming method based on Kalman filtering according to claim 5, characterized in that: The differential beamforming frequency domain signal corresponding to the forward beam and the backward beam is generated according to the nonlinearly distorted far-end signal, the far-end signal and the forward beam. The specific formula is: z(k)=c f (k)-t(We(k))*Car(k) Wherein, z(k) is the frequency domain signal of the differential beamforming, c f (k) is the forward beam, We(k) is the second beam weight parameter, t(We(k)) is the far-end signal, and Xe(k) is the nonlinearly distorted far-end signal.

7. The adaptive differential beamforming method based on Kalman filtering according to claim 1, characterized in that: The step of converting the differential beamforming frequency domain signal into a corresponding time domain speech signal output comprises: Using a preset low-pass filter to perform low-pass compensation filtering on the differential beamforming frequency domain signal, and converting the differential beamforming frequency domain signal into a low-pass filtered frequency domain signal; Performing an inverse Fourier transform on the low-pass filtered frequency domain signal to convert the low-pass filtered frequency domain signal into a low-pass filtered time domain signal; The low-pass filtered time domain signal is subjected to windowing and overlapping storage operations, and the low-pass filtered time domain signal is converted into the time domain speech signal and outputted.

8. An adaptive differential beamforming system based on Kalman filtering, characterized in that: It includes an acquisition module, a filtering module and an output module; Wherein, the acquisition module is used to acquire the forward beam and the backward beam; The filtering module is used to input the forward beam and the backward beam into a preset Kalman filter, so that the Kalman filter updates the current first Kalman filter parameter group according to the forward beam and the backward beam, obtains the second Kalman filter parameter group, and then generates a differential beamforming frequency domain signal corresponding to the forward beam and the backward beam according to the second Kalman filter parameter group, wherein the Kalman filter is obtained based on a neural network construction; The output module is used to convert the differential beamforming frequency domain signal into a corresponding time domain speech signal for output.

9. The Kalman filter-based adaptive differential beamforming system according to claim 8, characterized in that: The filtering module includes a first difference unit, a generation unit, a parameter updating unit and a second difference unit; The first differential unit is used to generate a first differential beamforming frequency domain signal according to the first Kalman filter parameter group, the forward beam and the backward beam, wherein the first Kalman filter parameter group includes a first transfer factor, a first far-end nonlinear distortion function, a first nonlinear transfer function, a first state covariance matrix, a first Kalman filter gain and a first beam weight parameter; The generating unit is used to input the first differential beamforming frequency domain signal into the neural network, so that the neural network generates a corresponding second transfer factor, a second far-end nonlinear distortion function and a second nonlinear transfer function according to the first differential beamforming frequency domain signal; The parameter updating unit is used for updating the first state covariance matrix, the first Kalman filter gain and the first beam weight parameter according to the second transfer factor to obtain a second Kalman filter parameter group; The second differential unit is used to generate a differential beamforming frequency domain signal corresponding to the forward beam and the backward beam according to the second Kalman filter parameter group.

10. The adaptive differential beamforming system based on Kalman filtering according to claim 9, characterized in that: The parameter updating unit updates the first state covariance matrix, the first Kalman filter gain, and the first beam weight parameter according to the second transfer factor to obtain a second Kalman filter parameter group, including: updating the first state covariance matrix according to the second transfer factor and the backward beam to obtain a second state covariance matrix; Update the first Kalman filter gain according to the second state covariance matrix and the backward beam to obtain a second Kalman filter gain; updating the first beam weight parameter according to the second Kalman filter gain and the first differential beamforming frequency domain signal to obtain a second beam weight parameter; The second Kalman filter parameter group is obtained by constructing the second transfer factor, the second far-end nonlinear distortion function, the second nonlinear transfer function, the second beam weight parameter, the second Kalman filter gain and the second state covariance matrix.

Images