Active noise reduction method and system based on spectrum mapping

Abstract

The invention belongs to the technical field of active noise reduction, and provides an active noise reduction method and system based on spectrum mapping. Based on spectrum mapping, a noise signal or a speech signal with noise is input into an LSTM network, and the spectrum of the noise signal or the noise signal in the noisy speech signal is mapped and output, and the noise is offset according to the principle of destructive interference, so as to achieve the purpose of reducing the noise sound pressure level at the error microphone, and finally the error signal is used to calculate the loss function and fed back to the LSTM network until the network loss function is minimized and reaches a convergence state. This method applies the knowledge of deep learning to the field of active noise reduction, which not only improves the noise reduction effect, but also reduces the dependence of traditional signal processing methods on hardware devices.

Description

Technical Field

[0001] The present invention belongs to the technical field of active noise reduction, and in particular relates to an active noise reduction method and system based on spectrum mapping. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] With the development of science and technology and industry, people's quality of life has been continuously improved, and noise pollution has also arisen, gradually becoming an important factor affecting people's quality of life and physical and mental health. Studies have shown that mild noise can make people anxious and affect sleep quality; if you are in a noise environment above 85 decibels for a long time, it will affect your hearing, and noise above 120 decibels can cause hearing loss. At the same time, noise can also increase the secretion of cortisol in the human body, leading to an increase in the incidence of diseases such as hypertension, heart disease and gastric ulcers. Therefore, effective control of noise has received more and more attention, and how to quickly and effectively control noise is a hot spot and difficulty in current research.

[0004] At present, noise control is mainly divided into two types: passive noise reduction and active noise reduction. Passive noise reduction mostly uses physical methods to control noise signals from three aspects: noise generation, propagation and reception. This method has a good control effect on high-frequency noise. Active noise reduction methods are based on the principle of destructive interference. They offset noise signals by generating anti-noise signals with the same amplitude and opposite phase as the noise signal, making up for the poor noise reduction effect of passive noise reduction methods on low-frequency noise. Traditional active noise reduction methods mostly use linear filters as controllers, and use adaptive algorithms to update filter coefficients in real time to generate anti-noise signals to minimize the sound pressure value at the observation point. However, in reality, the effect of traditional linear systems will be affected by the nonlinear distortion problems of devices such as speakers and microphones. At present, active noise reduction methods are mainly based on signal processing methods, using adaptive filtering algorithms to update filter coefficients and generate anti-noise signals. Such methods are not accurate enough to represent nonlinear factors in reality, and the hardware devices such as speakers and microphones in the system have a more obvious impact on the system effect. Summary of the invention

[0005] In order to solve at least one technical problem existing in the above-mentioned background technology, the present invention provides an active noise reduction method and system based on spectrum mapping, which estimates the imaginary spectrum diagram and real spectrum diagram of the noise signal from the noisy speech by using LSTM network for complex spectrum mapping, obtains the anti-noise signal through inverse Fourier transform, and eliminates the noise signal.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A first aspect of the present invention provides an active noise reduction method based on spectrum mapping, comprising the following steps:

[0008] Acquire a reference speech signal;

[0009] The neural network model is trained based on the reference speech signal and the impulse response of the physical structure under different sound field environments, and the trained neural network model is used for spectrum mapping to generate an estimated noise signal;

[0010] The estimated noise signal is inverted and then played to obtain an anti-noise signal;

[0011] According to the principle of destructive interference, the noise signal and the noise signal in the noisy speech signal are eliminated according to the anti-noise signal.

[0012] A second aspect of the present invention provides an active noise reduction system based on spectrum mapping, comprising:

[0013] The speech signal acquisition module is configured to: acquire a reference speech signal;

[0014] The noise signal estimation module is configured to: train a neural network model according to a reference speech signal and an impulse response of a physical structure under different sound field environments, perform spectrum mapping using the trained neural network model, and generate an estimated noise signal;

[0015] The anti-noise signal generating module is configured to: invert the estimated noise signal and then play it to obtain the anti-noise signal;

[0016] The noise elimination module is configured to eliminate the noise signal and the noise signal in the noisy speech signal according to the anti-noise signal based on the destructive interference principle.

[0017] A third aspect of the present invention provides a computer-readable storage medium.

[0018] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the active noise reduction method based on spectrum mapping as described above.

[0019] A fourth aspect of the present invention provides a computer device.

[0020] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the active noise reduction method based on spectrum mapping as described above are implemented.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] The present invention estimates the virtual spectrogram and real spectrogram of the input noise signal and the real spectrogram and virtual spectrogram of the noise signal in the noisy speech signal by using an LSTM network, and cancels the noise signal and the noise signal in the noisy speech signal according to the destructive interference principle, thereby improving the noise reduction effect.

[0023] The present invention applies deep learning knowledge to the field of active noise reduction and uses a pre-trained LSTM network to estimate the real and imaginary spectrograms of the ideal anti-noise signal instead of using a filter to generate the spectrum of the anti-noise signal, thereby reducing the dependence of traditional signal processing methods on hardware devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0025] Figure 1 This is an overall flow chart of an active noise reduction method based on spectrum mapping according to Embodiment 1 of the present invention;

[0026] Figure 2 This is a schematic diagram of active noise reduction data processing flow based on spectrum mapping according to Embodiment 1 of the present invention;

[0027] Figure 3 This is a LSTM network structure diagram of Example 1 of the present invention. DETAILED DESCRIPTION

[0028] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0029] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0030] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0031] Embodiment 1

[0032] This embodiment provides an active noise reduction method based on spectrum mapping, which estimates the virtual spectrum and real spectrum of the noise signal from the noisy speech by using LSTM network for complex spectrum mapping, obtains the anti-noise signal through inverse Fourier transform, and eliminates the noise signal. The invention applies the knowledge of deep learning to the field of active noise reduction, which not only improves the noise reduction effect, but also reduces the dependence of traditional signal processing methods on hardware devices. In order to overcome the shortcomings of traditional active noise reduction methods, the present invention provides a method of complex spectrum mapping for active noise reduction, estimates the real spectrum and virtual spectrum of the anti-noise signal, and the generated ideal anti-noise should be the same as the primary noise, so as to achieve the purpose of completely eliminating the primary noise.

[0033] like Figure 1-Figure 2 As shown, an active noise reduction method based on spectrum mapping includes the following steps:

[0034] S101: Acquire reference speech signal data x(t) at a reference microphone;

[0035] S102: Convert the reference speech signal x(t) from the time domain to the frequency domain through short-time Fourier transform STFT to obtain the real spectrum X r and the imaginary spectrum X i ;

[0036] S103: Based on the physical structure of the designed sound field environment, use the image method to simulate the reverberation that occurs in the sound field environment, including the design of the sound field shape and the sensor placement, and the estimation of the impulse response of the main path and the secondary path using the image method;

[0037] S104: Based on the real spectrum X of the reference speech signal r and the imaginary spectrum X i The neural network model is trained in the impulse response of the physical structure under different sound field environments, and the trained neural network model is used for spectrum mapping to generate the real spectrum Y of the estimated noise signal r and virtual spectrum Y i ;

[0038] The training process of the neural network model includes: using an image method to simulate the sound field spatial impulse response of the main path and the secondary path, and placing the model in different sound field spatial impulse responses for training; obtaining an error signal e(t) detected at the error microphone, calculating a loss function based on the error signal, and feeding back to the neural network model until the network loss function is minimized and reaches a convergence state;

[0039] S105: Obtain an estimated noise signal y(t) by performing an inverse short-time Fourier transform (ISTFT) on the real spectrum and the imaginary spectrum of the noise signal;

[0040] S106: The loudspeaker inverts the noise signal y(t) and then plays it to obtain an anti-noise signal a(t);

[0041] S107: Since the anti-noise signal a(t) and the noise signal y(t) have the same amplitude and opposite phase, according to the principle of destructive interference, the noise signal y(t) and the noise signal in the noisy speech signal are eliminated according to the anti-noise signal a(t);

[0042] S108: Use normalized mean square error (NMSE), short-time objective intelligibility (STOI) and perceptual evaluation of speech quality (PESQ) to evaluate the noise reduction effect of the system.

[0043] After obtaining the reference speech signal data x(t), the reference speech signal data is preprocessed, including the following steps:

[0044] (1) Classifying the reference speech signal into speech data without noise and a noise signal;

[0045] The data in the dataset is saved as an HDF5 file. The speech data without noise is classified into the sph class; the noise is classified into the noi class; the speech with noise is a mixture of the noise signal and the speech signal without noise.

[0046] There are two situations when training is required. When the reference signal is a noise signal, select a variety of noises in the sound effect library as the training set, and select a part of the noise as the verification set.

[0047] When the reference signal is a noisy speech signal, the noisy speech signal is mixed by randomly selecting a noise signal and speech data without noise from the above-mentioned sound effect library.

[0048] For example, when the reference signal is a noise signal, 10,000 noises are selected from the sound effect library as the training set, and engine noise and factory noise in NOISEX-92 are used as the verification set to test the network noise reduction effect;

[0049] When the reference signal is a noisy speech signal, the noisy signal is randomly selected from the above 10,000 noise signals and the clean speech in the TIMIT corpus and mixed.

[0050] (2) Cut and frame the signal.

[0051] For example, the speech data is cut into a signal with a duration of 6 minutes, the sampling rate of the input data set is 16KHZ, the input signal is divided into 20ms frames, a 0 frame of 5ms is added before the frame, and consecutive frames overlap by 5ms; the signal-to-noise ratio of the mixed noise is randomly selected from (-5dB, -4dB, -3dB, -2dB, -1dB, 0dB), but each data size is not limited to this and can be adjusted according to the data sampling rate; the processed signal is passed into the LSTM network after Fourier transform (STFT).

[0052] In S103, the simulated sound field space is a rectangular space, and the specific arrangement method is:

[0053] For example, the sound field space is simulated as a rectangular space of 3m×4m×2m, where the position of the reference microphone for receiving the noise signal is (1.5, 1, 1)m, the position of the speaker for generating the anti-noise signal is (1.5, 2.5, 1)m, and the position of the error microphone for detecting the error signal is (1.5, 3, 1)m.

[0054] The estimates of primary and secondary paths include:

[0055] Implementation example: The image method is used to simulate the room impulse responses (IRs) generated by different reverberation times. During the training phase, impulse responses are randomly selected from multiple reverberation times (0.15s, 0.175s, 0.225s, 0.2s, 0.25s), and the impulse response length is set to 512. Each reverberation time is not limited to this and can be adjusted according to the physical structure.

[0056] In S103, after Fourier transform, a zero frame having the same length as the prediction frame is added before the frame of the reference speech signal to achieve the purpose of predicting the future frame;

[0057] If the frame length to be predicted is m, then a zero frame of length m should be added before the frame of the reference speech signal of length n, that is, the total length of the signal frame with the added zero frame is (m+n). Since the neural network training target remains unchanged, each time a frame of length n is intercepted, it is equivalent to predicting the future m frames.

[0058] 1 / 4 of the reference speech signal window length is used as the frame shift to achieve the purpose of obtaining multiple prediction frames using one time-frequency unit.

[0059] like Figure 3As shown, in S104, the training process includes training for noise and training for noisy speech signals, the neural network model adopts a deep neural network (DNN) and a four-layer long short-term memory network (LSTM network), and fully connected layers are added to both ends of the four-layer LSTM network, retaining the advantages of the fully connected layer and LSTM;

[0060] The training samples for training the model are divided into noise signals and noisy speech signals:

[0061] If the training sample is a noise signal, the training target should be an ideal noise-resistant signal;

[0062] If the training sample is a noisy speech signal, the training target should be the noise component in the noisy speech signal.

[0063] Wherein, in S104, calculating the loss function according to the error signal includes:

[0064] When the reference signal is a noise signal, the loss function is the mean square value of the error signal, and its formula is as follows:

[0065]

[0066] Where m is the signal length and e(t) is the error signal at the error microphone.

[0067] When the reference signal is a noisy speech signal, the loss function should be the difference between the error signal and the pure speech, and its formula is as follows:

[0068]

[0069] Where p(t) is the impulse response of the main path, and v(t) is the pure speech signal.

[0070] The optimizer of the entire LSTM network is determined, and the noise signal of the HDF5 file is read out. The optimizer adopts Adam.

[0071] Embodiment 2

[0072] This embodiment provides an active noise reduction system based on spectrum mapping, including:

[0073] The speech signal acquisition module is configured to: acquire a reference speech signal;

[0074] The reference speech signal is transformed by Fourier transform to obtain a real spectrum and an imaginary spectrum;

[0075] The noise signal estimation module is configured to: train a neural network model based on the real spectrum and imaginary spectrum of the reference speech signal and the impulse response under the physical structure of different sound field environments according to the designed physical structure of the sound field environment, perform spectrum mapping using the trained neural network model, and generate an estimated noise signal;

[0076] The anti-noise signal generating module is configured to: invert the noise signal and then play it to obtain the anti-noise signal;

[0077] The noise elimination module is configured to eliminate the noise signal and the noise signal in the noisy speech signal according to the anti-noise signal based on the destructive interference principle.

[0078] Embodiment 3

[0079] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the active noise reduction method based on spectrum mapping as described above are implemented.

[0080] Embodiment 4

[0081] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the active noise reduction method based on spectrum mapping as described above are implemented.

[0082] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0083] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0084] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0085] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0086] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0087] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An active noise reduction method based on spectrum mapping, It is characterized in that The steps include: Obtain a reference speech signal; train a neural network model according to the reference speech signal and the impulse response of the physical structure under different sound field environments, use the trained neural network model to perform spectrum mapping, and generate an estimated noise signal; After obtaining the reference speech signal data, the reference speech signal is transformed from the time domain to the frequency domain through a short-time Fourier transform to obtain a real spectrum and an imaginary spectrum; The training of the neural network model in the impulse response of the physical structure under different sound field environments includes: using an image method to simulate the sound field space impulse response of the main path and the secondary path, and placing the model in different sound field space impulse responses for training; The estimated noise signal is inverted and then played to obtain an anti-noise signal; According to the principle of destructive interference, the noise signal and the noise signal in the noisy speech signal are eliminated according to the anti-noise signal.

2. The active noise reduction method based on spectrum mapping as claimed in claim 1, It is characterized in that The design of the physical structure of the sound field environment includes the design of the sound field shape and the sensor placement position. The sound field shape is a rectangular space. The sensor placement position includes the position of the reference microphone, the position of the speaker that generates the anti-noise signal, and the position of the error microphone.

3. The active noise reduction method based on spectrum mapping as claimed in claim 1, It is characterized in that The neural network model adopts a deep neural network and a four-layer long short-term memory network, and fully connected layers are added at both ends of the four-layer long short-term memory network.

4. The active noise reduction method based on spectrum mapping as claimed in claim 1, It is characterized in that The training of the neural network model also includes: obtaining an error signal detected at the error microphone, calculating a loss function based on the error signal, and providing feedback to the neural network model until the network loss function is minimized and reaches a convergence state.

5. The active noise reduction method based on spectrum mapping as claimed in claim 1, It is characterized in that After obtaining the reference speech signal data, the reference speech signal data is preprocessed, the reference speech signal is classified, the reference speech signal is divided into speech data without noise and a noise signal, and the classified reference speech signal is cut and framed.

6. An active noise reduction system based on spectrum mapping, using an active noise reduction method based on spectrum mapping as claimed in any one of claims 1 to 5, It is characterized in that include: The speech signal acquisition module is configured to: acquire a reference speech signal; The noise signal estimation module is configured to: train a neural network model according to a reference speech signal and an impulse response of a physical structure under different sound field environments, perform spectrum mapping using the trained neural network model, and generate an estimated noise signal; The anti-noise signal generating module is configured to: invert the estimated noise signal and then play it to obtain the anti-noise signal; The noise elimination module is configured to eliminate the noise signal and the noise signal in the noisy speech signal according to the anti-noise signal based on the destructive interference principle.

7. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the program is executed by a processor, the steps of the active noise reduction method based on spectrum mapping as described in any one of claims 1 to 5 are implemented.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the program, the steps in the active noise reduction method based on spectrum mapping as described in any one of claims 1 to 5 are implemented.

Images