A method for suppressing strong noise in air-based command and control environment

Abstract

The present application belongs to the field of aviation display and control technology, and particularly relates to a method for suppressing strong noise in an air-based command and control environment. It includes: step one, optimizing the microphone array of air-based command and control, collecting a first voice signal and engine mechanical noise through the microphone array, processing the first voice signal through a super-directional beamformer, and obtaining a second voice signal after noise reduction directed to the pilot's mouth; step two, removing the engine mechanical noise in the second voice signal through an adaptive filter to obtain a third voice signal; step three, training a deep learning noise reduction model, and performing cabin noise suppression processing on the third voice signal through a deep learning noise reduction model to obtain a fourth voice signal, and sending the fourth voice signal to the recognition engine. The method for suppressing strong noise in an air-based command and control environment of the present application can improve the quality of the collected human voice data, thereby improving the accuracy of operator voice recognition and promoting the engineering application of voice interaction related technologies.

Description

Technical Field

[0001] The present application belongs to the field of aviation display and control technology, and in particular relates to a method for suppressing strong noise in an air-based command and control environment. Background Art

[0002] The air-based command and control system is deployed in the cockpit of a manned aircraft. The operator completes the command and control functions of the UAV through voice recognition in a high-dynamic environment. The strong noise environment of the manned cockpit causes the audio data obtained by the traditional helmet sound pickup device to be unable to meet the needs of voice recognition. Simple physical noise reduction or traditional active noise reduction methods at the back end cannot eliminate the typical aerodynamic noise in the cockpit.

[0003] Therefore, it is desired to have a technical solution to overcome or at least alleviate at least one of the above-mentioned defects of the prior art. Summary of the invention

[0004] The purpose of the present application is to provide a method for suppressing strong noise in an air-based command and control environment to solve at least one problem existing in the prior art.

[0005] The technical solution of this application is:

[0006] A method for suppressing strong noise in an air-based command and control environment, comprising:

[0007] Step 1: Optimize the microphone array of air-based command and control, collect a first voice signal and engine mechanical noise through the microphone array, process the first voice signal through a super-directional beamformer, and obtain a second voice signal after noise reduction directed to the pilot's mouth;

[0008] Step 2: removing the engine mechanical noise in the second voice signal through an adaptive filter to obtain a third voice signal;

[0009] Step three: perform deep learning noise reduction model training, perform cabin noise suppression processing on the third voice signal through the deep learning noise reduction model to obtain a fourth voice signal, and send the fourth voice signal to the recognition engine.

[0010] In at least one embodiment of the present application, in step one, the microphone array for optimizing air-based command includes: four microphones evenly distributed in a linear pattern are arranged at the pickup position of the driver's helmet earphone, and a microphone is installed on the side of the driver's helmet.

[0011] In at least one embodiment of the present application, in step 1, the super-directional beamformer is:

[0012] Assume that there is a uniform linear array of M omnidirectional microphones with an array element spacing of δ. In the presence of isotropic noise, a super-directional beamformer is designed, whose array gain in the end-fire direction reaches M2 , the array gain is defined as follows:

[0013]

[0014]

[0015] δ ij =(ij)δ

[0016] Among them, d L (ω,θ) is the array steering vector, θ is the desired direction;

[0017] So that the array gain ζ L,dn The filter with the largest [h(ω)] is obtained by the following formula:

[0018]

[0019] in, is an arbitrary complex number. By the distortion-free criterion constraint, the maximum signal-to-noise ratio filter, that is, the super-directional beamformer, is obtained as follows:

[0020]

[0021] In at least one embodiment of the present application, the method further includes processing the super-directional beamformer to obtain a robust super-directional beamformer:

[0022] Get the white noise gain:

[0023]

[0024] Maximizing the directivity factor under the premise of constraining the white noise gain is equivalent to minimizing the following formula:

[0025]

[0026] Among them, ε is the Lagrange multiplier. By constraining the distortion-free criterion, a robust super-directional beamformer is obtained:

[0027]

[0028] In at least one embodiment of the present application, in step 2, removing the engine mechanical noise in the second voice signal by an adaptive filter to obtain a third voice signal includes:

[0029] Get adaptive filter;

[0030] The engine mechanical noise in the second voice signal is removed by an adaptive filter to obtain a third voice signal:

[0031] e(n)=d(n)-y(n)

[0032]

[0033] Wherein, x(n) is the engine mechanical noise, d(n) is the second speech signal, h(n) is the adaptive filter, e(n) is the third speech signal, and N is the filter length.

[0034] In at least one embodiment of the present application, in step three, the deep learning denoising model training includes:

[0035] The deep learning denoising model is trained using the mixed speech y and the corresponding pure speech s output by the signal processing module:

[0036] Extract the acoustic features F of the mixed speech y:

[0037] F = log(mel(STFT(y)))

[0038] Among them, STFT is short-time Fourier transform, mel is Mel spectrum feature;

[0039] Normalize the acoustic feature F:

[0040] F=(F-mean) / var

[0041] Among them, mean is the average value of the training set features, and var is the standard deviation of the training set features;

[0042] The standardized acoustic features F are sent as input to the deep learning denoising model, and the MSE error e between the output s_ of the deep learning denoising model and the actual pure speech s is used to guide the update of the model parameters;

[0043] e=||s_-ISTFT(STFT(y)*model(F))|| 2

[0044] Among them, ISTFT is the inverse short-time Fourier transform, model is the model, and * is the dot product;

[0045] Until the value of the MSE error e tends to be stable and the deep learning denoising model converges, the deep learning denoising model is saved.

[0046] In at least one embodiment of the present application, the deep learning denoising model is further compressed, specifically:

[0047] Use 16-bit fixed-point method for all parameters of deep learning denoising model to speed up calculation;

[0048] SVD decomposition is used on the matrix to reduce the amount of calculation.

[0049] The invention has at least the following beneficial technical effects:

[0050] The strong noise suppression method for the air-based command and control environment of the present application can improve the quality of the collected human voice data, thereby improving the accuracy of operator voice recognition and promoting the engineering application of voice interaction related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a flow chart of a method for suppressing strong noise in an air-based command environment according to an embodiment of the present application;

[0052] Figure 2 is a super-directional beam pattern of an embodiment of the present application;

[0053] Figure 3 is a flowchart of an adaptive filter processing according to an embodiment of the present application;

[0054] Figure 4 is an adaptive filter according to an embodiment of the present application;

[0055] Figure 5 It is a frequency domain block adaptive filter of one embodiment of the present application;

[0056] Figure 6 This is a deep learning denoising model training flowchart for one implementation of the present application. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical scheme and advantages of the implementation of this application clearer, the technical scheme in the embodiment of this application will be described in more detail below in conjunction with the drawings in the embodiment of this application. In the drawings, the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The described embodiments are part of the embodiments of this application, not all of them. The embodiments described below with reference to the drawings are exemplary and are intended to be used to explain this application, and should not be construed as limitations on this application. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. The embodiments of this application are described in detail below in conjunction with the drawings.

[0058] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the scope of protection of the present application.

[0059] The following is combined with Figures 1 to 6 This application is described in further detail.

[0060] This application provides a method for suppressing strong noise in an air-based command environment, such as Figure 1 As shown, the following steps are included:

[0061] Step 1: Optimize the microphone array of air-based command and control, collect the first voice signal and the engine mechanical noise through the microphone array, process the first voice signal through the super-directional beamformer, and obtain the second voice signal with noise reduction directed to the pilot's mouth;

[0062] Step 2: removing the engine mechanical noise in the second voice signal by an adaptive filter to obtain a third voice signal;

[0063] Step three: perform deep learning noise reduction model training, perform cabin noise suppression processing on the third voice signal through the deep learning noise reduction model to obtain a fourth voice signal, and send the fourth voice signal to the recognition engine.

[0064] The method for suppressing strong noise in an air-based command environment of the present application, in step 1, optimizes the microphone array of air-based command, including: setting four microphones evenly distributed in a linear manner at the pickup position of the pilot's helmet earphone to collect the pilot's voice control instructions, and installing a microphone on the side of the pilot's helmet to collect the mechanical noise of the aircraft engine. The voice signal collected by the microphone array is processed by a super-directional beamformer to filter out the wind noise after the friction between the front end of the aircraft and the air, and a noise-reduced signal pointing to the pilot's mouth can be obtained.

[0065] Super-directional beamforming is a type of fixed beamforming, which has a narrower main lobe and stronger noise suppression capability than conventional beamforming. In one embodiment of the present application, in step 1, the super-directional beamformer is:

[0066] Assume that there is a uniform linear array of M omnidirectional microphones with an array element spacing of δ. In the presence of isotropic noise, a super-directional beamformer is designed, whose array gain in the end-fire direction reaches M 2 , the array gain is defined as follows:

[0067]

[0068]

[0069] δ ij =(ij)δ

[0070] Among them, d L (ω,θ) is the array steering vector, θ is the desired direction;

[0071] So that the array gain ζ L,dn The filter with the largest [h(ω)] is obtained by the following formula:

[0072]

[0073] in, is an arbitrary complex number. By the distortion-free criterion constraint, the maximum signal-to-noise ratio filter, that is, the super-directional beamformer, is obtained as follows:

[0074]

[0075] The maximum signal-to-noise ratio filter is a super-directional beamformer (the main lobe direction is θ). However, the white noise gain of the super-directional beamformer at low frequencies is much less than 1, so it will lead to the amplification of white noise in practical applications, especially at low frequencies. Therefore, in this embodiment, preferably, the super-directional beamformer is processed to obtain a robust super-directional beamformer:

[0076] Get the white noise gain:

[0077]

[0078] Maximizing the directivity factor under the premise of constraining the white noise gain is equivalent to minimizing the following formula:

[0079]

[0080] Among them, ε is the Lagrange multiplier. By constraining the distortion-free criterion, a robust super-directional beamformer is obtained:

[0081]

[0082] On the basis of the robust super-directional beamformer, the low-frequency sidelobe constraint is further added to make the beam pointing narrower at low frequencies and enhance the suppression of noise. After the beamforming processing at this level, the overall signal-to-noise ratio is improved by about 10 to 15 dB.

[0083] Figure 2The super-directional beam pattern designed in this embodiment is given, and it can be seen that the super-directional beamformer has limited noise suppression capability at low frequencies. In this regard, the noise will be further suppressed by combining the adaptive noise cancellation method.

[0084] The strong noise suppression method for air-based command and control environment of the present application is that in the aircraft cockpit environment, the frequency spectrum occupied by wind noise is relatively wide and does not have obvious spectral characteristics, but the engine mechanical noise usually has more obvious spectral characteristics, which makes it possible to use adaptive noise cancellation processing for engine mechanical noise in the cockpit environment. The present application uses a mic arranged on the side of the pilot's helmet to collect reference noise. The collected reference noise includes both wind noise and engine mechanical noise, but the spectral structure of the engine mechanical noise is more critical for learning and tracking the adaptive filter processing. Figure 3 As shown, the noisy signal obtained by mixing the voice command and the engine mechanical noise collected by the reference microphone will be collected by the microphone array placed in the helmet headset. Therefore, as long as the change of H can be accurately calculated and tracked, the engine mechanical noise collected by the microphone can be well eliminated.

[0085] In this embodiment, in step 2, removing the engine mechanical noise in the second voice signal by an adaptive filter to obtain a third voice signal includes:

[0086] Get adaptive filter;

[0087] The engine mechanical noise in the second voice signal is removed by an adaptive filter to obtain a third voice signal:

[0088] e(n)=d(n)-y(n)

[0089]

[0090] Where x(n) is the engine mechanical noise, d(n) is the second speech signal, h(n) is the adaptive filter, e(n) is the third speech signal, and N is the filter length. The block diagram of the adaptive cancellation algorithm can be used Figure 4 represents, s(n) represents the near-end speaker's speech.

[0091] In the preferred embodiment of the present application, in the actual complex use environment of the cockpit, it is necessary to consider the fast tracking and convergence of the filter. Time domain processing usually produces unacceptable delays, while frequency domain processing has the advantages of low computational complexity and high convergence rate compared to time domain processing. Each frequency band in the convergence process can be accurately controlled to achieve overall optimal filtering. Therefore, in this embodiment, the structure of the frequency domain block adaptive filter (PBFDAF) is used for adaptive noise cancellation. Its structure is as follows: Figure 5 shown.

[0092] The coefficient vector of the time-domain adaptive filter can be expressed as:

[0093] w(n)=[w 0 (n)…w M-1 (n)] T

[0094] The error vector in the time domain is:

[0095] e(n)=[e(n)…e(n+M-1)] T

[0096] The input signal matrix in the frequency domain is:

[0097] X(k)=diag{X 0 (k)…X 2M-1 (k)}

[0098] =diag{F[x(kM-M)…x(kM+M-1)] T}

[0099] The filter coefficients and error signal vector in the frequency domain are:

[0100] W(k)=[w 0 (k)…w 2M-1 (k)] T =F[w T (kM)0…0]

[0101] E(k)=[E 0 (k)…E 2M-1 (k)] T =F[0…0e(kM)] T

[0102] Among them, F and F -1 They are 2M×2M DFT and IDFT matrices respectively. Therefore, the frequency domain adaptive iteration expression of FDAF can be written as:

[0103]

[0104] Where, μ(k)=diag{μ 0 (k)…μ 2M-1 (k)} is the normalized step size matrix, Λ(k)=diag{P 0 (k)…P 2M-1 (k)} is the input signal power matrix, and They are:

[0105]

[0106]

[0107] After being processed by the adaptive filter, the overall signal-to-noise ratio can be improved by about 15 to 20 dB.

[0108] The method for suppressing strong noise in the air-based command and control environment of the present application also performs cabin noise suppression processing on the third voice signal through a deep learning noise reduction model. Single-channel noise reduction algorithms can be divided into two categories: traditional methods and methods based on deep learning. Traditional methods can be divided into parameter methods, non-parametric methods, and methods based on statistical models. The typical representative of deep learning methods is the method based on deep neural network (DNN). The advantages of traditional methods are simple methods, easy implementation, small amount of calculation, and good processing effect on stationary noise in an environment with high signal-to-noise ratio. However, when the signal-to-noise ratio is low, the algorithm performance drops sharply, and the processing effect on non-stationary noise is very poor. The noise reduction algorithm based on DNN just makes up for the shortcomings of the traditional noise reduction signal algorithm, but because this method is a data-driven supervised learning method, a large amount of data training model is required to achieve the expected effect. Both data preparation and model training take a lot of time, and compared with traditional methods, the method has a large amount of calculation, and the method of model compression will be used to reduce the amount of calculation while ensuring performance.

[0109] In a preferred embodiment of the present application, in step 3, performing deep learning denoising model training includes:

[0110] The deep learning denoising model is trained using the mixed speech y and the corresponding pure speech s output by the signal processing module:

[0111] The input speech of this module is the speech output by the signal processing module, and the output speech is sent to the next module for subsequent operations.

[0112] Extract the acoustic features F of the mixed speech y:

[0113] F = log(mel(STFT(y)))

[0114] Among them, STFT is short-time Fourier transform, mel is Mel spectrum feature;

[0115] Normalize the acoustic feature F:

[0116] F=(F-mean) / var

[0117] Among them, mean is the average value of the training set features, and var is the standard deviation of the training set features;

[0118] The standardized acoustic features F are sent as input to the deep learning denoising model, and the MSE error e between the output s_ of the deep learning denoising model and the actual pure speech s is used to guide the update of the model parameters;

[0119] e=||s_-ISTFT(STFT(y)*model(F))|| 2

[0120] Among them, ISTFT is the inverse short-time Fourier transform, model is the model, and * is the dot product;

[0121] Until the value of the MSE error e tends to be stable and the deep learning denoising model converges, the deep learning denoising model is saved.

[0122] In this embodiment, further, a model test is included. After the mixed speech passes through the signal processing module, y is obtained, and features are extracted from y, and the features are input into the model to obtain a corresponding output mask, and the mask is multiplied by y to reconstruct the speech to obtain a noise-reduced signal and output it to the next module. The model framework is as follows Figure 6 shown.

[0123] The method for suppressing strong noise in an air-based command and control environment of the present application needs to compress the model to reduce the amount of calculation because the parameters of the original model are large and the computing resources of the mobile device are limited. In this embodiment, the deep learning noise reduction model is compressed, specifically: a 16-bit fixed-point method is used for all parameters of the deep learning noise reduction model to speed up the calculation speed; SVD decomposition is used for the matrix to reduce the amount of calculation. The amount of calculation is greatly reduced while the loss of model performance is very small. The signal after the previous signal processing is processed by the model again, and the expected signal-to-noise ratio can be increased by about 10dB.

[0124] The strong noise suppression method for air-based command and control environment of the present application can improve the noise suppression by about 30-45db, and the optimization rate is as high as 100%.

[0125] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for suppressing strong noise in an air-based command and control environment, characterized in that: include: Step 1: Optimize the microphone array of air-based command and control, collect a first voice signal and engine mechanical noise through the microphone array, process the first voice signal through a super-directional beamformer, and obtain a second voice signal after noise reduction directed to the pilot's mouth; Step 2: removing the engine mechanical noise in the second voice signal through an adaptive filter to obtain a third voice signal; Step 3: training a deep learning noise reduction model, performing cabin noise suppression processing on the third voice signal through the deep learning noise reduction model to obtain a fourth voice signal, and sending the fourth voice signal to a recognition engine; In step 1, the super-pointing beamformer is: Assume that there is a uniform linear array of M omnidirectional microphones with an array element spacing of δ. In the presence of isotropic noise, a super-directional beamformer is designed, whose array gain in the end-fire direction reaches M. 2 , the array gain is defined as follows: d ij =(ij)δ Among them, d L (ω,θ) is the array steering vector, θ is the desired direction; So that the array gain ζ L,dn The filter with the largest [h(ω)] is obtained by the following formula: in, is an arbitrary complex number. By the distortion-free criterion constraint, the maximum signal-to-noise ratio filter, that is, the super-directional beamformer, is obtained as follows:

2. The method for suppressing strong noise in an air-based command and control environment according to claim 1, characterized in that: In step one, the microphone array for optimizing air-based command includes: setting four microphones evenly distributed in a linear pattern at the pickup position of the driver's helmet earphone, and installing a microphone on the side of the driver's helmet.

3. The method for suppressing strong noise in an air-based command and control environment according to claim 1, characterized in that: Also included is processing the super-directional beamformer to obtain a robust super-directional beamformer: Get the white noise gain: Maximizing the directivity factor under the premise of constraining the white noise gain is equivalent to minimizing the following formula: Among them, ε is the Lagrange multiplier. By constraining the distortion-free criterion, a robust super-directional beamformer is obtained:

4. The method for suppressing strong noise in an air-based command and control environment according to claim 3 is characterized in that: In step 2, removing the engine mechanical noise in the second voice signal by an adaptive filter to obtain a third voice signal includes: Get adaptive filter; The engine mechanical noise in the second voice signal is removed by an adaptive filter to obtain a third voice signal: e(n)=d(n)-y(n) Wherein, x(n) is the engine mechanical noise, d(n) is the second speech signal, h(n) is the adaptive filter, e(n) is the third speech signal, and N is the filter length.

5. The method for suppressing strong noise in an air-based command and control environment according to claim 1, characterized in that: In step 3, the deep learning denoising model training includes: The deep learning denoising model is trained using the mixed speech y and the corresponding pure speech s output by the signal processing module: Extract the acoustic features F of the mixed speech y: F = log(mel(STFT(y))) Among them, STFT is short-time Fourier transform, mel is Mel spectrum feature; Normalize the acoustic feature F: F=(F-mean) / var Among them, mean is the average value of the training set features, and var is the standard deviation of the training set features; The standardized acoustic features F are sent as input to the deep learning denoising model, and the MSE error e between the output s_ of the deep learning denoising model and the actual pure speech s is used to guide the update of the model parameters; e=||s_-ISTFT(STFT(y)*model(F))|| 2 Among them, ISTFT is the inverse short-time Fourier transform, model is the model, and * is the dot product; Until the value of the MSE error e tends to be stable and the deep learning denoising model converges, the deep learning denoising model is saved.

6. The method for suppressing strong noise in an air-based command and control environment according to claim 5, characterized in that: It also includes model compression of the deep learning denoising model, specifically: Use 16-bit fixed-point method for all parameters of deep learning denoising model to speed up calculation; SVD decomposition is used on the matrix to reduce the amount of calculation.

Images