An active noise control system and method based on a convolutional recurrent neural network

Abstract

The present invention discloses an active noise control system and method based on a convolutional recurrent neural network. The system consists of a preprocessing module, a convolutional recurrent neural network, a signal reconstruction module, and a network loss module. The preprocessing module is used to preprocess a reference signal and a sound signal and extract corresponding features. The convolutional recurrent neural network consists of two convolution modules, a recurrent neural network, and two deconvolution modules. The reference signal features and the sound signal features are used as network inputs to predict the elimination signal features. The signal reconstruction module is used to reconstruct the elimination signal features in the frequency domain into an elimination signal in the time domain. The elimination signal is played through a loudspeaker to achieve the purpose of noise control. The network loss module consists of a psychoacoustic model and a loss calculation module. The masking threshold is obtained through the psychoacoustic model, and the total network loss is calculated. The gradient is back-propagated and used to update the weights of the network module.

Description

Technical Field

[0001] The present invention relates to the field of speech processing technology, and in particular to an active noise control system and method based on a convolutional recurrent neural network. Background Art

[0002] Sound is an important means of information exchange. However, the presence of various types of ambient noise can interfere with the transmission of sound information, reducing communication efficiency. Therefore, efforts are underway to control noise, one of which is active noise control (ANC). Active noise control, also known as active noise control, cancels out ambient noise by introducing a secondary sound source—sound waves with the same amplitude but opposite phase as the ambient noise. ANC systems are generally categorized as feedforward, feedback, and hybrid. Feedforward systems offer significant noise suppression effectiveness and low complexity, making them the most commonly used in practice. Feedforward ANC systems typically include a reference microphone, an error microphone, and a loudspeaker. The reference microphone collects ambient noise, or the reference signal. The error microphone, typically located near the user's ears, collects the error signal, which reflects the user's hearing perception and serves as the basis for adaptive filter adjustments. The loudspeaker primarily plays the cancellation signal. Because the reference microphone, error microphone, and loudspeaker are typically located some distance apart, the effects of sound propagation must still be considered in ANC processing. The propagation path between the reference microphone and the error microphone is called the primary path, while the path between the loudspeaker and the error microphone is called the secondary path. In practical applications, the nonlinear effects of the loudspeaker's internal electronic components can cause linear algorithms to inaccurately estimate the secondary path, leading to deviations in the calculation of the cancellation signal and affecting the effectiveness of noise control. To address this nonlinear effect, researchers have proposed methods such as using Volterra series, estimating the nonlinear coefficients of the secondary path, and using neural networks as filters. These methods have demonstrated effective noise cancellation regardless of audio playback.

[0003] In practice, speakers often play both a sound signal and a cancellation signal simultaneously, as in active noise-canceling headphones and hearing aids. Systems that simultaneously play both a sound signal and a cancellation signal are called Audio Integrated Active Noise Control (AIANC) systems. AIANC systems typically employ an ANC algorithm, but replace the adaptive algorithm's error signal with an estimated error signal. The estimated error signal is derived by adding the sound signal that has passed through a secondary path to the error signal. For a linear AIANC system, as long as the secondary path is accurately estimated, sound interference can be cleanly eliminated from the error signal. However, when nonlinearities exist in the AIANC system, simple additive compensation cannot eliminate sound signal interference. The estimated error signal contains residual sound, causing the adaptive filtering process to damage the sound and degrade the listening experience. Furthermore, an AIANC system must ensure both noise reduction and sound quality. However, the migrated ANC algorithm only considers noise cancellation, which does not meet practical requirements. In response to the need to ensure sound quality, the academic community has proposed introducing automatic gain controllers and psychoacoustic models to improve sound quality. However, the research on these two methods has not yet involved nonlinear issues and has not evaluated the integrity of the processed sound.

[0004] In summary, the current research on AIANC systems still has the following deficiencies:

[0005] (1) Less consideration is given to the impact of nonlinear effects on secondary path modeling;

[0006] (2) Only the noise elimination effect after AIANC processing is considered, ignoring the need to ensure sound quality, which easily causes sound distortion. Summary of the Invention

[0007] The purpose of the present invention is to address the shortcomings of active noise control methods in existing AIANC systems and to provide an active noise control system and method based on a convolutional recurrent neural network (CRN). This system and method addresses the problems of inaccurate secondary path estimation and severe sound signal distortion in current nonlinear AIANC systems by providing an improved convolutional recurrent neural network model that effectively preserves sound signals while taking into account nonlinear effects, thereby eliminating noise and significantly improving voice quality. The present invention can be applied to factory workshops, vehicle interiors, hands-free calls, and other situations.

[0008] The first object of the present invention can be achieved by adopting the following technical solutions:

[0009] An active noise control system based on convolutional recurrent neural network, the active noise control system includes a preprocessing module, a convolutional recurrent neural network, a signal reconstruction module, and a network loss module, wherein:

[0010] The convolutional recurrent neural network includes a first convolution module, a second convolution module, a recurrent neural model, a first deconvolution module, and a second deconvolution module. The network loss module includes a psychoacoustic model and a loss calculation module. The preprocessing module is connected to the first convolution module and the second convolution module respectively. The first convolution module is connected to the second convolution module, the recurrent neural network, and the first deconvolution module. The second convolution module is connected to the first convolution module, the recurrent neural network, and the second deconvolution module. The first deconvolution module and the second deconvolution module are connected to the signal reconstruction module. The network loss module is connected to the convolutional recurrent neural network.

[0011] The preprocessing module is used to preprocess the signal. After the reference signal passes through the preprocessing module, the reference signal feature is output and input into the first convolution module. After the sound signal passes through the preprocessing module, the sound signal feature is output and input into the second convolution module and the network loss module.

[0012] The first convolution module and the second convolution module are used as encoders to perform convolution processing on signal features. The first convolution module and the second convolution module each include M convolution units connected in sequence. The output of the Mth convolution unit of the first convolution module and the output of the Mth convolution unit of the second convolution module are feature-concatenated and used as the input of the recurrent neural network.

[0013] The first deconvolution module and the second deconvolution module are used as decoders to perform deconvolution processing on signal features. The first deconvolution module and the second deconvolution module both include M sequentially connected deconvolution units. The inputs of the first deconvolution module and the second deconvolution module are both obtained by splicing the output of the recurrent neural network, the output of the Mth convolution unit of the first convolution module, and the output of the Mth convolution unit of the second convolution module. The input of the mth deconvolution unit of the first deconvolution module is obtained by splicing the output of the (M-m+1)th convolution unit of the first convolution module, the output of the (M-m+1)th convolution unit of the second convolution module, and the output of the (m-1)th deconvolution unit of the first deconvolution module, where m∈(1,M]; the input of the mth deconvolution unit of the second deconvolution module is obtained by splicing the output of the (M-m+1)th convolution unit of the first convolution module, the output of the (M-m+1)th convolution unit of the second convolution module, and the output of the (m-1)th deconvolution unit of the second deconvolution module;

[0014] The signal reconstruction module is used to restore the signal characteristics to a time domain signal, the input is the elimination signal characteristics obtained by splicing the outputs of the first deconvolution module and the second deconvolution module, and the output is a time domain elimination signal;

[0015] The psychoacoustic model is used to calculate the sound masking threshold, and its input is the sound signal characteristics and the secondary path label. The output is connected to the loss calculation module, and the secondary path label is obtained from the data set; the loss calculation module is used to calculate the network loss, and its input is the output of the signal reconstruction module, the output of the psychoacoustic model, the secondary path label, the expected signal label and the nonlinear coefficient label. The output is connected to the convolutional recurrent neural network, and the secondary path label, the expected signal label and the nonlinear coefficient label are obtained from the data set.

[0016] The second object of the present invention can be achieved by adopting the following technical solutions:

[0017] An active noise control method based on a convolutional recurrent neural network, the noise control method comprising the following steps:

[0018] S1. Set noise environment parameters and calculate propagation paths, select nonlinear coefficients, signal-to-noise ratios, and sound signals to generate reference and expected signals, and construct noise control datasets.

[0019] S2. Input the sound signal and reference signal in the data set into the preprocessing module to obtain the sound signal feature set and the reference signal feature set respectively, and construct the training data set and the test data set;

[0020] S3, inputting the sound signal features and the reference signal features into the convolutional recurrent neural network to obtain the elimination signal features;

[0021] S4, inputting the cancellation signal feature into the signal reconstruction module to obtain the cancellation signal;

[0022] S5. Input the elimination signal, sound signal characteristics, nonlinear coefficient label, secondary path label, and expected signal label into the network loss module to calculate the network loss;

[0023] S6. Calculate the gradient of the network loss and use backpropagation to update the weights of the convolutional recurrent neural network.

[0024] S7, repeat steps S3, S4, S5, and S6 until the loss of the convolutional recurrent neural network is lower than a preset threshold, and save the network parameters at this time;

[0025] S8: Input the sound signal and reference signal into the preprocessing module to obtain the corresponding signal features, input them into the convolutional recurrent neural network saved in S7 to obtain the elimination signal features, and obtain the elimination signal after processing by the signal reconstruction module to eliminate the environmental noise.

[0026] Furthermore, the dataset construction process in step S1 is as follows:

[0027] S1.1. Set the length, width, and height of the room model, the speaker positions, and the positions of the reference and error microphones. Build a noise environment model and use the image method to calculate the discrete impulse response (p(n)) between the reference microphone and the speaker, and the discrete impulse response (s(n)) between the error microphone and the speaker. The path between the reference microphone and the speaker is the primary path, and the path between the error microphone and the speaker is the secondary path.

[0028] S1.2. Select the signal-to-noise ratio snr, the nonlinear coefficient η, and select the sound signal a(n) from the clean speech library. Construct the reference signal x(n) and the expected signal d(n) based on the primary and secondary paths in S1.1. The x(n) constructed according to the selected signal-to-noise ratio snr and the nonlinear coefficient η is:

[0029]

[0030] a s (n) = f thf (a(n))*s(n) (2)

[0031] d0(n)=v(n)*p(n) (3)

[0032]

[0033] Where T is the signal length, a s (n) is the sound signal through the secondary path, d0(n) is the original noise through the primary path, v(n) is the original noise, f thf (·) represents the nonlinear effect of the secondary path, which is represented by the hyperbolic tangent function and the nonlinear coefficient η, and * represents the convolution operation;

[0034] The expected signal is calculated as follows, d(n) = x(n)*p(n) (5)

[0035] S1.3. A noise control dataset is constructed using the secondary path discrete impulse response s(n), nonlinear coefficient η, and desired signal d(n) as noise scene labels, and the sound signal a(n) and reference signal x(n) as noise control data. Different primary paths, secondary paths, and nonlinear coefficients create different noise scenes, which helps the convolutional recurrent network learn to eliminate the relationship between the signal and the noise environment as much as possible, achieving certain generalization and robustness.

[0036] Furthermore, the process of obtaining the feature set in step S2 is as follows:

[0037] S2.1. Perform short-time Fourier transform on the reference signal x(n) and the sound signal a(n) to obtain frequency domain features X(i,k) and A(i,k), respectively, where i is the frame index, i∈[0,N), N is the total number of frames, and k is the frequency index.

[0038] S2.2. Extract the real and imaginary parts of the signal frequency domain features and reconstruct them as follows to obtain the features X(i, k, c) and A(i, k, c):

[0039] X(i,k,c)=[Re{X(i,k)},Im{X(i,k)}],c∈[0,1] (6)

[0040] A(i,k,c)=[Re{A(i,k)},Im{A(i,k)}],c∈[0,1] (7)

[0041] Where c is the channel index. A channel index of 0 represents the real part of the signal in the frequency domain, and a channel index of 1 represents the imaginary part of the signal in the frequency domain. Reorganizing the frequency domain features of the signal is done to convert them from complex features to dual-channel real features. This allows the network to learn amplitude and phase features simultaneously, while also converting complex operations into multi-dimensional real operations.

[0042] Furthermore, the signal reconstruction process in step S4 is as follows: S4.1, the output of the convolutional recurrent neural network, that is, the eliminated signal feature Y(i, k, c), is reconstructed into a frequency domain signal according to the following formula:

[0043] Y(i,k)=Re{Y(i,k)}+j·Im{Y(i,k)}=Y(i,k,0)+j·Y(i,k,1) (8)

[0044] S4.2. Calculate the cancellation signal y(n) by performing an inverse short-time Fourier transform on the frequency-domain signal Y(i,k). Converting the cancellation signal features into a time-domain cancellation signal facilitates both the calculation of network loss during training and the practical use of the model.

[0045] Furthermore, the network loss in step S5 is calculated as follows: S5.1. Calculate the estimated error signal ε(n) based on the expected signal label, the nonlinear coefficient label, the secondary path label, the sound signal characteristics, and the cancellation signal:

[0046] ε(n)=d(n)-f thf (y(n)+a(n))*s(n)+f thf (a(n))*s(n) (9)

[0047] S5.2. Calculate the sound signal a passing through the secondary path based on the secondary path label and sound signal characteristics. sThe power spectrum of (n) is obtained by the Johnston psychoacoustic model to obtain the masking threshold power spectrum P th (k), the process is as follows:

[0048] S5.2.1, obtain a by Fourier transform s (n) the frequency domain signal A s (k), calculate the power spectrum P s (k) and the energy B of the i-th critical band within the critical band i :

[0049] P s (k)=Re 2 {A s (k)}+Im 2 {A s (k)} (10)

[0050]

[0051] Among them, bl i represents the lower frequency limit of the i-th critical band, bh i represents the upper frequency limit of the i-th critical band.

[0052] S5.2.2. Calculate the propagation function S ij And the diffuse power spectrum C i :

[0053]

[0054] C i =S ij ×B i (13)

[0055] S5.2.3. Calculate the masking threshold T i :

[0056]

[0057] σ i =α SFM (14.5+i)+5.5(1-α SFM ) (15)

[0058]

[0059]

[0060] Among them, σ i is the bias, α SFM is the pitch factor, SFM dB is the spectral flatness, SFM dBmaxTake [-55dB, -65dB], G m is the geometric mean of the power spectrum, A m is the arithmetic mean of the power spectrum;

[0061] S5.2.4. Compare the masking threshold with the absolute hearing threshold of the human ear, T abs Compare and take the larger value:

[0062] T i =max(T i ,T abs ) (18)

[0063]

[0064] Where f is the sampling rate of the signal in kHz, T abs The unit is dB;

[0065] S5.2.5, set the masking threshold T i Convert to linear frequency domain and get the power spectrum P of the masking threshold th (k);

[0066] S5.3. Calculate the network loss L:

[0067] L=mean(H·P ε (k)) (20)

[0068]

[0069] Among them, H is the weight of loss, P ε (k) is the power spectrum of the estimated error signal ε(n). Calculating the network loss can be used to guide the optimization direction of the network. The introduced psychoacoustic masking threshold can enable the network to retain as many sound components as possible while ensuring noise elimination.

[0070] Furthermore, the calculation process of the elimination signal in step S8 is as follows:

[0071] S8.1. Process the reference signal x(n) and the sound signal a(n) through a preprocessing module to obtain features X(i, k, c) and A(i, k, c);

[0072] S8.2. Input features X(i, k, c) and A(i, k, c) into the convolutional recurrent neural network to obtain the eliminated signal features Y(i, k, c);

[0073] S8.3. Input the feature Y(i,k,c) into the signal reconstruction module and calculate the frequency-domain cancellation signal Y(i,k) according to Equation (8). Then, perform an inverse short-time Fourier transform to obtain the time-domain signal y(n). This step allows the performance of the model to be evaluated by predicting the cancellation signal using the reference signal and the sound signal in the test dataset.

[0074] The present invention has the following advantages and effects compared to the prior art:

[0075] (1) The present invention adopts a convolutional recurrent network as a filter for active noise control. Unlike existing methods that only use the estimation error under linear conditions as the basis for adaptive method adjustment, the error calculation in training adds nonlinear conditions, allowing the network to simulate the nonlinear effects in the system to a certain extent. Therefore, in active noise control, the influence of secondary path estimation errors and nonlinear effects on the noise elimination effect can be significantly reduced.

[0076] (2) The present invention simultaneously inputs the reference signal and the sound signal into the convolutional recurrent network, performs deep feature extraction in each convolution module, and then performs feature reorganization learning within the recurrent neural network, so that the network can simultaneously learn the characteristics of the two and the relationship between the two, thereby predicting a better cancellation signal.

[0077] (3) The present invention adopts a psychoacoustic model in the calculation of network loss. The masking threshold is used as the loss weight to guide the network parameter adjustment for noise of different intensities, so that the optimized model can filter out noise to a certain extent while ensuring the integrity and comfort of the sound signal. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0079] Figure 1 is a structural block diagram of an active noise control system based on a convolutional recurrent network disclosed in an embodiment of the present invention;

[0080] Figure 2 is a structural diagram of a convolution unit in an embodiment of the present invention;

[0081] Figure 3 is a structural diagram of a deconvolution unit in an embodiment of the present invention;

[0082] Figure 4 This is a training and testing flow chart of the active noise control method based on the convolutional recurrent network in an embodiment of the present invention;

[0083] Figure 5Schematic diagram of generating training set reference signals and sound signals in an embodiment of the present invention. DETAILED DESCRIPTION

[0084] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0085] Example 1

[0086] This embodiment discloses an active noise control system based on a convolutional recurrent network. The structure of the active noise control system is as follows: Figure 1 As shown,

[0087] The system consists of a preprocessing module, a convolutional recurrent neural network, a signal reconstruction module, and a network loss module.

[0088] The convolutional recurrent neural network includes a first convolution module, a second convolution module, a recurrent neural network, a first deconvolution module, and a second deconvolution module. The network loss module includes a psychoacoustic model and a loss calculation module. The preprocessing module is connected to the first convolution module and the second convolution module respectively. The first convolution module is connected to the second convolution module, the recurrent neural network, and the first deconvolution module. The second convolution module is connected to the first convolution module, the recurrent neural network, and the second deconvolution module. The first deconvolution module and the second deconvolution module are connected to the signal reconstruction module. The network loss module is connected to the convolutional recurrent neural network.

[0089] The preprocessing module is used to preprocess the signal. After the reference signal passes through the preprocessing module, the reference signal feature is output and input into the first convolution module. After the sound signal passes through the preprocessing module, the sound signal feature is output and input into the second convolution module and the network loss module.

[0090] The first convolution module and the second convolution module are used as encoders to perform convolution processing on signal features. The first convolution module and the second convolution module each include 5 convolution units connected in sequence. The structure of each convolution unit is as follows: Figure 2As shown in the figure, it consists of two convolutional layers, one of which is followed by a Sigmoid activation function, followed by a batch normalization layer and an exponential linear activation unit. The input of the recurrent neural network is obtained by concatenating the output of the fifth convolution unit of the first convolution module and the output of the fifth convolution unit of the second convolution module. The output is connected to the first deconvolution module and the second deconvolution module respectively. The recurrent neural network includes two layers of grouped unidirectional long short-term memory (GLSTM) networks. Within each GLSTM, the last dimension of the feature is divided into two groups, which are respectively fed into the LSTM layer. The output features are obtained through stacking, transposition, and dimensionality reduction.

[0091] The first deconvolution module and the second deconvolution module are used as decoders to perform deconvolution processing on the signal features. The first deconvolution module and the second deconvolution module each include 5 deconvolution units connected in sequence. The structure of each deconvolution unit is as follows: Figure 3 As shown, the network consists of two deconvolution layers, one of which is followed by a sigmoid activation function, followed by a batch normalization layer, and an exponential linear activation unit. The output channel number of the first deconvolution module and the fifth deconvolution unit of the second deconvolution module is 1. The inputs of the first and second deconvolution modules are both the output of the recurrent neural network, the output of the fifth convolution unit of the first convolution module, and the output of the fifth convolution unit of the second convolution module. The input of the mth deconvolution unit of the first deconvolution module is obtained by concatenating the output of the (M-m+1)th convolution unit of the first convolution module, the output of the (M-m+1)th convolution unit of the second convolution module, and the output of the (m-1)th deconvolution unit of the first deconvolution module, where m∈(1,M], M=5. The input of the mth deconvolution unit of the second deconvolution module is obtained by concatenating the output of the (M-m+1)th convolution unit of the first convolution module, the output of the (M-m+1)th convolution unit of the second convolution module, and the output of the (m-1)th deconvolution unit of the second deconvolution module;

[0092] The signal reconstruction module is used to restore the signal features to the time domain signal. The input is the output of the first deconvolution module and the second deconvolution module. After processing by the fully connected layer, the signal features are concatenated to obtain the eliminated signal features, and the output is the time domain eliminated signal.

[0093] The psychoacoustic model is used to calculate the sound masking threshold, and its input is the sound signal characteristics and the secondary path label. The output is connected to the loss calculation module, and the secondary path label is obtained from the data set; the loss calculation module is used to calculate the network loss, and its input is the output of the signal reconstruction module, the output of the psychoacoustic model, the secondary path label, the expected signal label and the nonlinear coefficient label. The output is connected to the convolutional recurrent neural network, and the secondary path label, the expected signal label and the nonlinear coefficient label are obtained from the data set.

[0094] Example 2

[0095] This embodiment discloses an active noise control system based on a convolutional recurrent network. The structure of the active noise control system is as follows: Figure 1 As shown,

[0096] The system consists of a preprocessing module, a convolutional recurrent neural network, a signal reconstruction module, and a network loss module.

[0097] The convolutional recurrent neural network includes a first convolution module, a second convolution module, a recurrent neural network, a first deconvolution module, and a second deconvolution module. The network loss module includes a psychoacoustic model and a loss calculation module. The preprocessing module is connected to the first convolution module and the second convolution module respectively. The first convolution module is connected to the second convolution module, the recurrent neural network, and the first deconvolution module. The second convolution module is connected to the first convolution module, the recurrent neural network, and the second deconvolution module. The first deconvolution module and the second deconvolution module are connected to the signal reconstruction module. The network loss module is connected to the convolutional recurrent neural network.

[0098] The preprocessing module is used to preprocess the signal. After the reference signal passes through the preprocessing module, the reference signal feature is output and input into the first convolution module. After the sound signal passes through the preprocessing module, the sound signal feature is output and input into the second convolution module and the network loss module.

[0099] The first convolution module and the second convolution module are used as encoders to perform convolution processing on signal features. The first convolution module and the second convolution module each include 5 convolution units connected in sequence. The structure of each convolution unit is as follows: Figure 2As shown in the figure, it consists of two convolutional layers, one of which is followed by a Sigmoid activation function, followed by a batch normalization layer and an exponential linear activation unit. The input of the recurrent neural network is obtained by concatenating the output of the fifth convolution unit of the first convolution module and the output of the fifth convolution unit of the second convolution module. The output is connected to the first deconvolution module and the second deconvolution module respectively. The recurrent neural network includes two layers of grouped unidirectional long short-term memory (GLSTM) networks. Within each GLSTM, the last dimension of the feature is divided into two groups, which are respectively fed into the LSTM layer. The output features are obtained through stacking, transposition, and dimensionality reduction.

[0100] The first deconvolution module and the second deconvolution module are used as decoders to perform deconvolution processing on the signal features. The first deconvolution module and the second deconvolution module each include 5 deconvolution units connected in sequence. The structure of each deconvolution unit is as follows: Figure 3 As shown, the network consists of two deconvolution layers, one of which is followed by a sigmoid activation function, followed by a batch normalization layer, and then an exponential linear activation unit. The output channels of the first deconvolution module and the fifth deconvolution unit of the second deconvolution module are two, and are followed by a fully connected layer for dimensionality reduction. The inputs of the first and second deconvolution modules are the concatenation of the output of the recurrent neural network, the output of the fifth convolution unit of the first convolution module, and the output of the fifth convolution unit of the second convolution module. The input of the mth deconvolution unit of the first deconvolution module is obtained by concatenating the output of the (M-m+1)th convolution unit of the first convolution module, the output of the (M-m+1)th convolution unit of the second convolution module, and the output of the (m-1)th deconvolution unit of the first deconvolution module, where m∈(1,M], M=5. The input of the mth deconvolution unit of the second deconvolution module is obtained by concatenating the output of the (M-m+1)th convolution unit of the first convolution module, the output of the (M-m+1)th convolution unit of the second convolution module, and the output of the (m-1)th deconvolution unit of the second deconvolution module;

[0101] The signal reconstruction module is used to restore the signal features to the time domain signal. The input is the output of the first deconvolution module and the second deconvolution module. After processing by the fully connected layer, the signal features are concatenated to obtain the eliminated signal features, and the output is the time domain eliminated signal.

[0102] The psychoacoustic model is used to calculate the sound masking threshold, and its input is the sound signal characteristics and the secondary path label. The output is connected to the loss calculation module, and the secondary path label is obtained from the data set; the loss calculation module is used to calculate the network loss, and its input is the output of the signal reconstruction module, the output of the psychoacoustic model, the secondary path label, the expected signal label and the nonlinear coefficient label. The output is connected to the convolutional recurrent neural network, and the secondary path label, the expected signal label and the nonlinear coefficient label are obtained from the data set.

[0103] Example 3

[0104] Based on the active noise control system based on convolutional recurrent network disclosed in Examples 1 and 2, this embodiment further discloses an active noise control method based on convolutional recurrent network. The method uses the following steps to complete training and testing. The training and testing process is as follows: Figure 4 As shown:

[0105] S1. Set noise environment parameters and calculate propagation paths, select nonlinear coefficients, signal-to-noise ratios, and sound signals to generate reference and expected signals, and construct noise control datasets.

[0106] S1.1. Set the size of the room model to 4×3×2m (length×width×height), the speaker position to (2.5,1.5,1.0)m, the reference microphone position to (1.0,1.5,1.0)m, and the error microphone position to (3.0,1.5,1.0)m. Build a noise environment model and use the image method to calculate the room impulse response: When the sound source is located at r s =(x s ,y s ,z s ), the microphone is located at r = (x, y, z), then the discrete impulse response between the sound source and the microphone is:

[0107]

[0108] LPF{δ(n-τf s )}≈δ(n-round{τf s}) (twenty three)

[0109] Among them, the set P = {(q,j,k):q,j,k∈{0,1}} indicates whether the sound source in the three directions of up and down, left and right, and front and back is included in the calculation. If it is 1, it is included in the calculation; the set M = {(m x ,m y ,m z ):-N r ≤m x ,m y ,m z ≤N r} represents the number of reflections of the sound source in the up and down, left and right, front and back directions, N r is the maximum number of reflections given; is the reflection coefficient of the six walls of the room model, τ is the arrival time of the sound from the sound source image to the microphone, f s is the sampling frequency, d is the distance between the sound source image and the microphone, and LPF{·} indicates that the cutoff frequency is f s / 2 low-pass filter. In Example 2, the sampling frequency f s 8000Hz. Set the reverberation time of the room to RT 60 ∈{0.1,0.3,0.5}s, the calculation formula is:

[0110]

[0111] Where V is the volume of the room, c is the speed of light, and S i is the wall area, β i is the wall reflection coefficient, and i is the wall number.

[0112] The discrete impulse response p(n) between the reference microphone and the loudspeaker and the discrete impulse response s(n) between the error microphone and the loudspeaker are calculated according to the above method;

[0113] S1.2. Select the signal-to-noise ratio snr∈[-5,0,5,10]dB and the nonlinear coefficient η∈[0.1,1.0,10.0], which represent strong nonlinearity, moderate nonlinearity, and weak nonlinearity, respectively. Select the sound signal a(n) from the clean speech library. Construct the reference signal x(n) and the expected signal d(n) based on the discrete impulse responses of the primary and secondary paths in S1.1. The signal generation process is as follows: Figure 5 As shown: According to the selected signal-to-noise ratio snr and nonlinear coefficient η, the x(n) obtained is,

[0114]

[0115] a s (n) = f thf (a(n))*s(n) (26)

[0116] d0(n)=v(n)*p(n) (27)

[0117]

[0118] Where T is the signal length, which is set to 5s. If it is less than 5s, it will be padded with zeros to 5s. s (n) is the sound signal that has passed through the secondary path, d0(n) is the original noise that has passed through the primary path, v(n) is the original noise, and f thf(·) represents the nonlinear effect of the secondary path, which is represented by the hyperbolic tangent function and the nonlinear coefficient η, and * represents the convolution operation;

[0119] The expected signal is calculated as follows, d(n) = x(n)*p(n) (29)

[0120] S1.3. Use the secondary path discrete impulse response s(n), nonlinear coefficient η, and desired signal d(n) as noise scene labels, and the sound signal a(n) and reference signal x(n) as noise control data to construct a noise control dataset.

[0121] S2. Input the sound signal and reference signal in the data set into the preprocessing module to obtain the sound signal feature set and the reference signal feature set respectively, and construct the training data set and the test data set;

[0122] S2.1. Perform a short-time Fourier transform on the reference signal x(n) and the sound signal a(n) to obtain frequency domain features X(i,k) and A(i,k), respectively, where i is the frame index, i∈[0,N), N is the total number of frames, and k is the frequency index. In Example 2, the frame length is 256, the frame shift is 64, and the frames are windowed and framed using a Hamming window function. A short-time Fourier transform is performed, and the discrete frequency index k∈[0,128] is obtained.

[0123] S2.2. Extract the real and imaginary parts of the signal frequency domain features and reconstruct them as follows to obtain the features X(i, k, c) and A(i, k, c):

[0124] X(i,k,c)=[Re{X(i,k)},Im{X(i,k)}],c∈[0,1] (30)

[0125] A(i,k,c)=[Re{A(i,k)},Im{A(i,k)}],c∈[0,1] (31)

[0126] Where c is the channel index, index 0 is the frequency domain real part feature of the signal, and index 1 is the frequency domain imaginary part feature of the signal;

[0127] S3, input the sound signal features and the reference signal features into the convolutional recurrent neural network to obtain the elimination signal features Y(i, k, c);

[0128] S4, inputting the cancellation signal feature into the signal reconstruction module to obtain the cancellation signal;

[0129] S4.1. The output of the convolutional recurrent neural network, i.e., the signal feature Y(i, k, c) is eliminated, and reconstructed into a frequency domain signal according to the following formula:

[0130] Y(i,k)=Re{Y(i,k)}+j·Im{Y(i,k)}=Y(i,k,0)+j·Y(i,k,1) (32)

[0131] S4.2. Calculate the cancellation signal y(n) by performing an inverse short-time Fourier transform on the frequency domain signal Y(i,k);

[0132] S5. Input the elimination signal, sound signal characteristics, nonlinear coefficient label, secondary path label, and expected signal label into the network loss module to calculate the network loss;

[0133] S5.1. Calculate the estimated error signal ε(n) based on the expected signal label, the nonlinear coefficient label, the secondary path label, the sound signal characteristics, and the cancellation signal:

[0134] ε(n)=d(n)-f thf (y(n)+a(n))*s(n)+f thf (a(n))*s(n) (33)

[0135] S5.2. Calculate the sound signal a passing through the secondary path based on the secondary path label and sound signal characteristics. s The power spectrum of (n) is obtained by the Johnston psychoacoustic model to obtain the masking threshold power spectrum P th (k);

[0136] S5.2.1, obtain a by Fourier transform s (n) the frequency domain signal A s (k), calculate the power spectrum P s (k) and the energy B in the i-th critical band i :

[0137] P s (ω)=Re 2 {A s (ω)}+Im 2 {A s (ω)} (34)

[0138]

[0139] Among them, bl i represents the lower frequency limit of the i-th critical band, bh i represents the upper frequency limit of the i-th critical band.

[0140] S5.2.2. Calculate the propagation function S ij and the diffuse power spectrum C i :

[0141]

[0142] C i =S ij ×B i (37)

[0143] S5.2.3. Calculate the masking threshold T i :

[0144]

[0145] σ i =α SFM (14.5+i)+5.5(1-α SFM ) (39)

[0146]

[0147]

[0148] Among them, σ i is the bias, α SFM is the pitch factor, SFM dB is the spectral flatness, SFM dBmax Generally take -60dB, G m is the geometric mean of the power spectrum, A m is the arithmetic mean of the power spectrum;

[0149] S5.2.4. Compare the masking threshold with the absolute hearing threshold of the human ear, T abs Compare and take the larger value:

[0150] T i =max(T i ,T abs ) (42)

[0151]

[0152] Wherein, f is the sampling rate of the signal, in kHz. In Example 2, f=8, T abs The unit is dB;

[0153] S5.2.5, set the masking threshold T i Convert to linear frequency domain and get the power spectrum P of the masking threshold th (k);

[0154] S5.3. Calculate the network loss L:

[0155] L=mean(H·P ε (k)) (44)

[0156]

[0157] Among them, H is the weight of loss, P ε (ω) is the power spectrum of the estimated error signal ε(n);

[0158] S6. Calculate the gradient of the network loss and update the weights of the convolutional recurrent neural network using backpropagation and the Adam optimizer.

[0159] S7. Repeat steps S3, S4, S5, and S6 until the loss of the convolutional recurrent neural network is lower than the preset threshold, and save the network parameters at this point. The training batch size is 20, the learning rate is 0.001, the total number of training rounds is 200, and the learning rate is adjusted every 2 rounds. The learning rate decay rate is 0.98;

[0160] S8, input the sound signal and the reference signal into the preprocessing module to obtain corresponding signal features, input them into the convolutional recurrent neural network saved in S7 to obtain the elimination signal features, and after processing by the signal reconstruction module, obtain the elimination signal to eliminate the environmental noise;

[0161] S8.1. Process the reference signal x(n) and the sound signal a(n) through a preprocessing module to obtain features X(i, k, c) and A(i, k, c);

[0162] S8.2. Input features X(i, k, c) and A(i, k, c) into the convolutional recurrent neural network to obtain the eliminated signal features Y(i, k, c);

[0163] S8.3. Input the feature Y(i,k,c) into the signal reconstruction module, calculate the frequency domain elimination signal Y(i,k) according to formula (32), and calculate the time domain signal y(n) through short-time inverse Fourier transform.

[0164] In order to verify the technical effect of the present invention, Example 1 and Example 2 are compared with the existing baseline system 1 (see Ghasemi, Sepehr, Marhaban, et al. Nonlinear Thf-Fxlms Algorithm For Active NoiseControl With Loudspeaker Nonlinearity[J]. Asian Journal of Control: Affiliatedwith ACPA, the Asian Control Professors'Association, 2016.) and baseline system 2 (see Hao ZA, Dlwa B. Deep ANC: A deep learning approach to active noise control[J]. Neural Networks, 2021, 141: 1-10.). The evaluation indicators use speech quality perception evaluation PESQ and short-time objective intelligibility STOI. The speech of the training set comes from the TIMIT training set, and the noise comes from the TUT acoustic training set. The speech of the test set comes from the TIMIT test set, and the noise is selected from the factory1 noise of NOISEX-92. The test room size and microphone and speaker positions are the same as those of the training set, and the reverberation time of the test is RT 60 ∈{0.2,0.4}s, calculate the primary path p(n) and secondary path s(n) as in step S1.1. Evaluate the error signal before and after filtering at signal-to-noise ratios of -7dB, -2dB, 2dB, and 7dB, and nonlinear coefficients of 0.15, 1.5, and 15.0, completing the test.

[0165] The error signal e0(n) before filtering is: e0(n) = d(n) - f thf (a(n))*s(n) (46)

[0166] The error signal e(n) after filtering is: e(n) = d(n) - f thf (y(n)+a(n))*s(n) (47)

[0167] The comparison results are shown in Tables 1 and 2. It can be seen that under different signal-to-noise ratios and varying degrees of nonlinear effects, the implementation scheme of this patent can improve both the PESQ and STOI indicators, indicating that ambient noise is effectively suppressed in terms of auditory perception, and voice quality is effectively improved for the user. Compared with Baseline System 1 and Baseline System 2, the implementation scheme of this patent can simultaneously suppress ambient noise and ensure voice quality in different noise scenarios under the AIANC system, resulting in positive ΔPESQ and ΔSTOI values under different conditions.

[0168] Table 1. ΔPESQ scores of the AIANC algorithm under different signal-to-noise ratios and nonlinearity levels

[0169]

[0170] Table 2. ΔSTOI scores of the AIANC algorithm under different signal-to-noise ratios and nonlinearity levels

[0171]

[0172] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. An active noise control system based on convolutional recurrent neural network, characterized in that: The active noise control system includes a preprocessing module, a convolutional recurrent neural network, a signal reconstruction module, and a network loss module, wherein: The convolutional recurrent neural network includes a first convolution module, a second convolution module, a recurrent neural network, a first deconvolution module, and a second deconvolution module. The network loss module includes a psychoacoustic model and a loss calculation module. The preprocessing module is connected to the first convolution module and the second convolution module respectively. The first convolution module is connected to the second convolution module, the recurrent neural network, and the first deconvolution module. The second convolution module is connected to the first convolution module, the recurrent neural network, the first deconvolution module, and the second deconvolution module. The first deconvolution module and the second deconvolution module are connected to the signal reconstruction module. The network loss module is connected to the convolutional recurrent neural network. The preprocessing module is used to preprocess the signal. After the reference signal passes through the preprocessing module, the reference signal feature is output and input into the first convolution module. After the sound signal passes through the preprocessing module, the sound signal feature is output and input into the second convolution module and the network loss module. The first convolution module is used as an encoder to perform convolution processing on the reference signal features, and the second convolution module is used as an encoder to perform convolution processing on the sound signal features. The first convolution module and the second convolution module each include M sequentially connected convolution units. The output of the Mth convolution unit of the first convolution module and the output of the Mth convolution unit of the second convolution module are feature-concatenated and used as the input of the recurrent neural network. The first deconvolution module and the second deconvolution module are used as decoders to perform deconvolution processing on signal features. The first deconvolution module and the second deconvolution module both include M sequentially connected deconvolution units. The inputs of the first deconvolution module and the second deconvolution module are both obtained by splicing the output of the recurrent neural network, the output of the Mth convolution unit of the first convolution module, and the output of the Mth convolution unit of the second convolution module. The input of the mth deconvolution unit of the first deconvolution module is obtained by splicing the output of the (M-m+1)th convolution unit of the first convolution module, the output of the (M-m+1)th convolution unit of the second convolution module, and the output of the (m-1)th deconvolution unit of the first deconvolution module, where m∈(1,M]; the input of the mth deconvolution unit of the second deconvolution module is obtained by splicing the output of the (M-m+1)th convolution unit of the first convolution module, the output of the (M-m+1)th convolution unit of the second convolution module, and the output of the (m-1)th deconvolution unit of the second deconvolution module; The signal reconstruction module is used to restore the signal characteristics to a time domain signal, the input is the elimination signal characteristics obtained by splicing the outputs of the first deconvolution module and the second deconvolution module, and the output is a time domain elimination signal; The psychoacoustic model is used to calculate the sound masking threshold, and its input is the sound signal characteristics and the secondary path label. The output is connected to the loss calculation module, and the secondary path label is obtained from the data set; the loss calculation module is used to calculate the network loss, and its input is the output of the signal reconstruction module, the output of the psychoacoustic model, the secondary path label, the expected signal label and the nonlinear coefficient label. The output is connected to the convolutional recurrent neural network, and the secondary path label, the expected signal label and the nonlinear coefficient label are obtained from the data set.

2. An active noise control method of an active noise control system based on a convolutional recurrent neural network according to claim 1, characterized in that: The noise control method comprises the following steps: S1. Set noise environment parameters and calculate propagation paths, select nonlinear coefficients, signal-to-noise ratios, and sound signals to generate reference and expected signals, and construct noise control datasets. S2. Input the sound signal and reference signal in the data set into the preprocessing module to obtain the sound signal feature set and the reference signal feature set respectively, and construct the training data set and the test data set; S3, inputting the sound signal features and the reference signal features into the convolutional recurrent neural network to obtain the elimination signal features; S4, inputting the cancellation signal feature into the signal reconstruction module to obtain the cancellation signal; S5. Input the elimination signal, sound signal characteristics, nonlinear coefficient label, secondary path label, and expected signal label into the network loss module to calculate the network loss; S6. Calculate the gradient of the network loss and use backpropagation to update the weights of the convolutional recurrent neural network. S7, repeat steps S3, S4, S5, and S6 until the loss of the convolutional recurrent neural network is lower than a preset threshold, and save the network parameters at this time; S8: Input the sound signal and reference signal into the preprocessing module to obtain the corresponding signal features, input them into the convolutional recurrent neural network saved in S7 to obtain the elimination signal features, and obtain the elimination signal after processing by the signal reconstruction module to eliminate the environmental noise.

3. The active noise control method according to claim 2, characterized in that: The dataset construction process in step S1 is as follows: S1.1, set the length, width, and height of the room model, the position of the speaker, the position of the reference microphone, and the position of the error microphone, build a noise environment model, and use the mirror method to calculate the discrete impulse response p(n) between the reference microphone and the speaker, and the discrete impulse response s(n) between the error microphone and the speaker, where the path between the reference microphone and the speaker is the main path, and the path between the error microphone and the speaker is the secondary path; S1.2, select the signal-to-noise ratio snr, the nonlinear coefficient η, and select the sound signal a(n) in the clean speech library, and construct the reference signal x(n) and the expected signal d(n) according to the main path and the secondary path in S1.1: the x(n) constructed according to the selected signal-to-noise ratio snr and the nonlinear coefficient η is, a s (n)=f thf (a(n))*s(n) (2) d0(n)=v(n)*p(n) (3) Where T is the signal length, a s (n) is the sound signal through the secondary path, d0(n) is the original noise through the primary path, v(n) is the original noise, f thf () represents the nonlinear effect of the secondary path, which is represented by the hyperbolic tangent function and the nonlinear coefficient η, and * represents the convolution operation; The expected signal is calculated as follows: d(n) = x(n)*p(n) (5) S1.

3. Use the secondary path discrete impulse response s(n), nonlinear coefficient η and expected signal d(n) as noise scene labels, and the sound signal a(n) and reference signal x(n) as noise control data to construct a noise control dataset.

4. The active noise control method according to claim 2, characterized in that: The process of obtaining the feature set in step S2 is as follows: S2.

1. Perform short-time Fourier transform on the reference signal x(n) and the sound signal a(n) to obtain frequency domain features X(i,k) and A(i,k), respectively, where i is the frame index, i∈[0,N), N is the total number of frames, and k is the frequency index. S2.

2. Extract the real and imaginary parts of the signal frequency domain features and reconstruct them as follows to obtain the features X(i, k, c) and A(i, k, c): X(i,k,c)=[Re{X(i,k)},Im{X(i,k)}],c∈[0,1] (6) A(i,k,c)=[Re{A(i,k)},Im{A(i,k)}],c∈[0,1] (7) Among them, c is the channel index, the channel index 0 is the frequency domain real part feature of the signal, and the channel index 1 is the frequency domain imaginary part feature of the signal.

5. The active noise control method according to claim 2, characterized in that: The process of signal reconstruction in step S4 is as follows: S4.

1. The output of the convolutional recurrent neural network, i.e., the signal feature Y(i, k, c) is eliminated, and reconstructed into a frequency domain signal according to the following formula: Y(i,k)=Re{Y(i,k)}+j·Im{Y(i,k)}=Y(i,k,0)+j·Y(i,k,1) (8) S4.

2. Calculate the cancellation signal y(n) by performing inverse short-time Fourier transform on the frequency domain signal Y(i,k).

6. The active noise control method according to claim 3, characterized in that: The network loss calculation process in step S5 is as follows: S5.

1. Calculate the estimated error signal ε(n) based on the expected signal label, the nonlinear coefficient label, the secondary path label, the sound signal characteristics, and the cancellation signal: ε(n)=d(n)-f thf (y(n)+a(n))*s(n)+f thf (a(n))*s(n) (9) S5.

2. Calculate the sound signal a passing through the secondary path based on the secondary path label and sound signal characteristics. s The power spectrum of (n) is obtained by the Johnston psychoacoustic model to obtain the masking threshold power spectrum P th (k), the process is as follows: S5.2.1, obtain a by Fourier transform s (n) the frequency domain signal A s (k), calculate the power spectrum P s (k) and the energy B in the i-th critical band i : P s (k)=Re 2 {A s (k)}+Im 2 {A s (k)} (10) Among them, bl i Indicates the lower limit of the frequency of the i-th critical band, bh i represents the upper frequency limit of the i-th critical band; S5.2.2, calculate the propagation function S ij And the diffuse power spectrum C i : C i =S ij ×B i (13) S5.2.

3. Calculate the masking threshold T i : s i =a SFM (14.5+i)+5.5(1-a SFM ) (15) Among them, σ i is the bias, α SFM is the pitch factor, SFM dB is the spectral flatness, SFM dBmax Take -60dB, G m is the geometric mean of the power spectrum, A m is the arithmetic mean of the power spectrum; S5.2.

4. Compare the masking threshold with the absolute hearing threshold of the human ear, T abs Compare and take the larger value: T i =max(T i ,T abs ) (18) Where f is the sampling rate of the signal in kHz, T abs The unit is dB; S5.2.5, set the masking threshold T i Convert to linear frequency domain and get the power spectrum P of the masking threshold th (k); S5.

3. Calculate the network loss L: L=mean(H·P ε (k)) (20) Among them, H is the weight of the loss, P ε (k) is the power spectrum of the estimated error signal ε(n).

7. The active noise control method according to claim 5, characterized in that: The calculation process of the elimination signal in step S8 is as follows: S8.

1. Process the reference signal x(n) and the sound signal a(n) through a preprocessing module to obtain features X(i, k, c) and A(i, k, c); S8.

2. Input features X(i, k, c) and A(i, k, c) into the convolutional recurrent neural network to obtain the eliminated signal features Y(i, k, c); S8.

3. Input the feature Y(i,k,c) into the signal reconstruction module, calculate the frequency domain cancellation signal Y(i,k) according to formula (8), and calculate the time domain signal y(n) through short-time inverse Fourier transform.

Images