Noise cancellation device and detection method thereof

Abstract

The present application relates to a kind of noise cancellation device and its detection method.Audio data is segmented, and according to its feature, into background noise, occasional noise and speech noise, and further extract multiple occasional noise from occasional noise;Further utilize background noise, occasional noise to further carry out the noise reduction processing of speech audio;Processing effect is more accurate, precision is better, and the noise reduction effect is better;Utilize the spectrum of background noise to carry out EQ adjustment to speech data, realize the noise reduction of overall background noise;Further utilize multiple occasional noise to carry out secondary EQ adjustment to speech data, further realize the noise reduction of occasional noise;Make full use of the method of speech recognition to further identify noise, and the recognition accuracy is higher, and the effect is better.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of audio processing, in particular to a noise cancellation device and a detection method thereof. BACKGROUND

[0002] At present, most of the speech noise processing adopts an algorithm based on deep learning. This speech noise reduction method is mainly divided into two categories: one is a method based on the TF time-frequency domain, including a mask-based and a non-mask-based method; the other is a method based on the time domain. However, the pre-learning process of the algorithm based on deep learning is very long, the required calculation amount is extremely large, and it cannot be directly applied to speech noise reduction in all scenes.

[0003] CN101903942B provides a noise cancellation system for generating a noise cancellation signal to be added to a desired signal to mitigate the effects of ambient noise. A detector is adapted to detect the magnitude of the input signal during the speech-free period; operate in a first mode when the input signal is above a threshold, and operate in a second mode when the input signal is below the threshold.

[0004] CN111095405A provides dynamic selection of noise cancellation algorithms and dynamic activation and deactivation of microphones to provide multi-mode noise cancellation for a voice detection device in situations where ambient noise prevents voice navigation from accurately interpreting voice commands. When ambient noise exceeding a threshold is detected, a specific noise cancellation algorithm best suited for the situation is selected, and one or more noise detection microphones are activated. The noise detection microphone(s) that receive the highest level of ambient noise can remain activated, while the remaining noise detection microphones can be deactivated. Then, the speech signal received by the speech microphone can be optimized by canceling the ambient noise signal received from the activated noise detection microphone(s) using the selected noise cancellation algorithm.

[0005] However, the above-mentioned noise processing method has poor processing effect and cannot specifically analyze and process different noises. SUMMARY

[0006] In view of the above, to solve the above-mentioned problems, a noise cancellation device is provided, which comprises an audio acquisition module, a memory, a central processing unit, an audio processing module, an EQ adjustment module, a spectrum analysis module and an output module.

[0007] The audio acquisition module is used to connect a microphone, a sound card or other recording equipment and acquire audio data; the audio acquisition module stores the acquired audio data in the storage module;

[0008] The central processing unit is connected to the memory and sends the audio data in the memory to the audio processing module for audio processing to realize noise cancellation.

[0009] The audio processing module stores the noise-canceled audio data in the memory, and the central processing unit outputs the noise-canceled audio data in the memory through the output module;

[0010] The audio processing module is connected with the frequency spectrum analysis module and the EQ adjustment module; the frequency spectrum analysis module is used for performing frequency spectrum analysis on the audio data, and the EQ adjustment module is used for performing EQ adjustment on the audio data.

[0011] The sampling rate of the audio acquisition module is 44100 Hz, and the bit rate is 128 bit; the sampling rate of the audio output by the output module is 44100 Hz, and the bit rate is 128 bit.

[0012] A method for performing noise detection by using the noise-cancellation device, comprising the following steps:

[0013] Step 1: The audio acquisition module acquires audio data to be detected, with a sampling rate of 44100 Hz and a bit rate of 128 bit, and stores the audio data to be detected in the memory;

[0014] Step 2: The central processing unit calls the audio data to be detected from the memory and sends it to the audio processing module;

[0015] Step 3: The audio processing module divides the audio; the dividing reference is to set an energy threshold A0, if the energy of the audio for more than 50 ms is less than A0 or greater than A0, the audio is divided from the audio data to be detected as an audio segment;

[0016] Step 4: The audio segment with energy less than A0 is analyzed separately, and each spectrum is fitted with multiple peaks to obtain the number of fitting peaks and the range of all fitting peak strengths; the fitted spectrum is analyzed for similarity, and the number of fitting peaks, the frequency of each fitting peak, the strength of each fitting peak and the range of fitting peak strengths are used as the calculation parameters of similarity; the audio segment corresponding to the spectrum with similarity greater than the threshold is divided into a group, and the total duration of each group of audio segments is calculated;

[0017] Step 5: Select the group of audio segments with the longest total duration in step 4, and mark the group of audio segments as background noise; splice the background noise, and analyze the spliced background noise for frequency spectrum to obtain the background noise spectrum;

[0018] Step 6, audio segments with energy greater than A0 are sorted by time length, and audio segments with time length less than 150 ms are marked as incidental noise; the audio segments corresponding to the incidental noise are subjected to spectral analysis, and each spectrum analyzed is subjected to multi-peak fitting to obtain the number of fitted peaks and the range of all fitted peak intensities; the fitted spectrum is subjected to similarity analysis, and the number of fitted peaks, the frequency of each fitted peak, the intensity of each fitted peak, and the range of fitted peak intensities are used as calculation parameters for similarity; audio segments corresponding to incidental noise with similarity greater than a threshold are grouped, and the total time length of each group of audio segments is calculated;

[0019] Step 7, select the group of incidental noise audio segments with the longest total time length in step 6, and mark the group of incidental noise audio segments as multi-incident noise; and splice the multi-incident noise, and perform spectral analysis on the spliced multi-incident noise to obtain the multi-incident noise spectrum;

[0020] Step 8, select audio segments with energy greater than A0 and sort them by time length, perform speech recognition on audio segments with time length greater than 150 ms, and perform semantic analysis on the results of speech recognition; if the results of speech recognition are semantically complete, mark the corresponding audio segments as ordinary speech; if the matching degree of the results of speech recognition is lower than a threshold or the semantics are unclear, mark the corresponding audio segments as speech noise.

[0021] A method for performing noise cancellation using the noise cancellation device, characterized by using the noise detection method, specifically comprising the following steps:

[0022] Step A, the audio acquisition module acquires audio data to be detected, and performs noise detection using steps 1-8 of the noise detection method;

[0023] Step B, audio segments with energy less than A0 in the audio data, audio segments corresponding to incidental noise, and audio segments corresponding to speech noise are directly muted;

[0024] Step C, the audio segments corresponding to the ordinary speech are subjected to spectral analysis, and then subjected to EQ adjustment, and the frequency of the fitted peak corresponding to the background noise spectrum is weakened on the spectrum of the audio segment corresponding to the ordinary speech, and the weakening amplitude is the same as the intensity of the fitted peak corresponding to the background noise spectrum;

[0025] Step D, the audio segments corresponding to the ordinary speech are further segmented, each segment has the same time length as the average time length of the audio segments corresponding to the multi-incident noise, and the segmented audio segments corresponding to the ordinary speech are subjected to spectral analysis and similarity analysis with the multi-incident noise spectrum;

[0026] Step E, screening out the common voice audio segment with similarity greater than the threshold in step D, and then weakening the frequency of the fitting peak corresponding to the multiple accidental noise spectrum on its spectrum, and the weakening amplitude is the same as the strength of the fitting peak corresponding to the multiple accidental noise spectrum;

[0027] Step F, splicing all the audio segments in their original order to obtain the noise-removed audio data.

[0028] The beneficial effects of the present application are:

[0029] The present application divides the audio data, and divides it into background noise, accidental noise and voice noise according to its characteristics, and further extracts multiple accidental noise from accidental noise; further, the background noise and accidental noise are used for further noise reduction processing of voice audio; the processing effect is more accurate, the precision is better, and the noise reduction effect is better;

[0030] The spectrum of the background noise is used for once EQ adjustment of the voice data, realizing the noise reduction of the overall background noise; further, the multiple accidental noise is used for twice EQ adjustment of the voice data, further realizing the noise reduction of the accidental noise; the voice recognition method is fully used for further identification of the noise, and the identification accuracy is higher and the effect is better. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 It is a schematic diagram of the overall architecture of the present application. DETAILED DESCRIPTION

[0032] The advantages, features and methods of achieving the purposes of the present application will be clear through the drawings and the following detailed description.

[0033] Example 1:

[0034] The EQ adjustment referred to in the present application is a graphic equalizer, which is a kind of audio processing effecter frequently used in audio processing, and the range of EQ adjustment is 5-20000Hz.

[0035] A noise removal device, comprising an audio acquisition module, a memory, a central processing unit, an audio processing module, an EQ adjustment module, a spectrum analysis module and an output module;

[0036] The audio acquisition module is used for connecting a microphone, a sound card or other recording equipment, and acquiring audio data; the audio acquisition module stores the collected audio data in the storage module;

[0037] The central processing unit connects the memory and sends the audio data in the memory to the audio processing module for audio processing, realizing noise removal;

[0038] The audio processing module stores the audio data after noise elimination in the memory, and the central processing unit outputs the audio data after noise elimination in the memory through the output module;

[0039] The audio processing module is connected with the frequency spectrum analysis module and the EQ adjustment module; the frequency spectrum analysis module is used for frequency spectrum analysis of the audio data, and the EQ adjustment module is used for EQ adjustment of the audio data.

[0040] The sampling rate of the audio acquisition module is 44100 Hz, and the bit rate is 128 bit; the audio sampling rate output by the output module is 44100 Hz, and the bit rate is 128 bit.

[0041] Embodiment 2

[0042] A method for performing noise detection by using the noise elimination device, comprising the following steps:

[0043] Step 1: the audio acquisition module acquires audio data to be detected, the sampling rate is 44100 Hz, and the bit rate is 128 bit; and stores the audio data to be detected in the memory;

[0044] Step 2: the central processing unit calls the audio data to be detected from the memory and sends it to the audio processing module;

[0045] Step 3: the audio processing module divides the audio; the dividing reference is to set an energy threshold A0, if the energy of the audio for more than 50 ms is less than A0 or greater than A0, the audio is divided from the audio data to be detected as an audio segment;

[0046] Step 4: the audio segment with energy less than A0 is analyzed respectively, and each spectrum analyzed is fitted with multiple peaks to obtain the number of fitting peaks and the range of all fitting peak strengths; the spectrum after fitting is analyzed for similarity, the number of fitting peaks, the frequency of each fitting peak, the strength of each fitting peak and the range of fitting peak strengths are used as the calculation parameters of similarity; the audio segment corresponding to the spectrum with similarity greater than the threshold is divided into a group, and the total duration of each group of audio segments is calculated;

[0047] The specific method of similarity analysis can adopt Euclidean distance, Mahalanobis distance or Pearson correlation coefficient; the fitting function used in multiple peak fitting is Gaussian function.

[0048] Step 5: select the group of audio segments with the longest total duration in step 4, and mark the group of audio segments as background noise; splice the background noise, and analyze the spliced background noise for frequency spectrum to obtain the background noise spectrum;

[0049] Step 6, audio segments with energy greater than A0 are sorted by time length, and audio segments with time length less than 150 ms are marked as incidental noise; the audio segments corresponding to the incidental noise are subjected to spectral analysis, and each spectrum analyzed is subjected to multi-peak fitting to obtain the number of fitted peaks and the range of all fitted peak intensities; the fitted spectrum is subjected to similarity analysis, and the number of fitted peaks, the frequency of each fitted peak, the intensity of each fitted peak, and the range of fitted peak intensities are used as calculation parameters for similarity; audio segments corresponding to incidental noise with similarity greater than a threshold are grouped, and the total time length of each group of audio segments is calculated;

[0050] Step 7, select the group of incidental noise audio segments with the longest total time length in step 6, and mark the group of incidental noise audio segments as multi-incident noise; and splice the multi-incident noise, and perform spectral analysis on the spliced multi-incident noise to obtain the multi-incident noise spectrum;

[0051] Step 8, select audio segments with energy greater than A0 and sort them by time length, perform speech recognition on audio segments with time length greater than 150 ms, and perform semantic analysis on the results of speech recognition; if the results of speech recognition are semantically complete, mark the corresponding audio segments as ordinary speech; if the matching degree of the results of speech recognition is lower than a threshold or the semantics are unclear, mark the corresponding audio segments as speech noise.

[0052] Embodiment 3:

[0053] A method for performing noise cancellation using the noise cancellation device, characterized by using the noise detection method, specifically comprising the following steps:

[0054] Step A, the audio acquisition module acquires audio data to be detected, and performs noise detection using steps 1-8 of the noise detection method;

[0055] Step B, audio segments with energy less than A0 in the audio data, audio segments corresponding to incidental noise, and audio segments corresponding to speech noise are directly muted;

[0056] Step C, the audio segments corresponding to the ordinary speech are subjected to spectral analysis, and then subjected to EQ adjustment, and the frequency of the fitted peak corresponding to the background noise spectrum is weakened on the spectrum of the audio segment corresponding to the ordinary speech, and the weakening amplitude is the same as the intensity of the fitted peak corresponding to the background noise spectrum;

[0057] Step D, the audio segments corresponding to the ordinary speech are further segmented, each segment has the same time length as the average time length of the audio segments corresponding to the multi-incident noise, and the segmented audio segments corresponding to the ordinary speech are subjected to spectral analysis and similarity analysis with the multi-incident noise spectrum.

[0058] Step E, screening out the common voice audio segments with similarity greater than the threshold in step D, and then weakening the frequency of the fitting peak corresponding to the multi-occurrence accidental noise spectrum on the spectrum thereof, the weakening amplitude being the same as the strength of the fitting peak corresponding to the multi-occurrence accidental noise spectrum;

[0059] Step F, splicing all the audio segments according to the original order to obtain the noise-eliminated audio data.

[0060] The above merely provides the preferred embodiments of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of the changes or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A noise detection method is performed using a noise elimination device to segment audio data and divide it into background and background according to its characteristics. Noise, accidental noise and speech noise, and further extract multiple accidental noises from the accidental noise; further use background noise and accidental noise to further reduce the noise of speech audio; It is characterized by comprising the following steps: Step 1: The audio acquisition module acquires the audio data to be detected and stores the audio data to be detected in a memory; Step 2: The central processing unit calls the audio data to be detected from the memory and sends it to the audio processing module; Step 3: The audio processing module segments the audio data. The segmentation benchmark is to set an energy threshold A0. If the energy of the audio for more than 50ms is less than A0 or greater than A0, it is segmented from the audio data to be detected as an audio segment. Step 4: Perform spectrum analysis on each audio segment with energy less than A0, and perform multi-peak fitting on each analyzed spectrum. Perform similarity analysis on the fitted spectrum, and group the audio segments corresponding to the spectrum with similarity greater than a threshold. Calculate the total duration of each group of audio segments. Step 5: Select the group of audio clips with the longest total length in step 4 and mark them as background noise; The background noise is spliced, and the spectrum of the spliced ​​background noise is analyzed to obtain the background noise spectrum; Step 6: Sort the audio clips with energy greater than A0 by duration, and mark the audio clips with a duration less than 150ms as accidental noise; perform spectrum analysis on the audio clips corresponding to the accidental noise, and perform multi-peak fitting on each analyzed spectrum; perform similarity analysis on the fitted spectrum, and group the accidental noise audio clips corresponding to the spectrums with similarity greater than a threshold, and calculate the total duration of each group of audio clips; Step 7: Select the group of accidental noise audio clips with the longest total duration in step 6, and mark the group of accidental noise audio clips as multiple accidental noise; splice the multiple accidental noises, and perform spectrum analysis on the spliced ​​multiple accidental noises to obtain a spectrum of the multiple accidental noise; Step 8. Select audio clips with energy greater than A0 and sort them by duration. Perform speech recognition on audio clips with a duration greater than 150ms, and perform semantic analysis on the speech recognition results. If the speech recognition result is semantically complete, the corresponding audio clip is marked as normal speech; if the speech recognition result has a matching degree lower than the threshold or the semantics are unclear, the corresponding audio clip is marked as speech noise.

2. The noise detection method according to claim 1, wherein: The noise elimination device includes an audio acquisition module, a memory, a central processing unit, an audio processing module, an EQ adjustment module, a spectrum analysis module and an output module; The audio acquisition module is used to connect to a microphone, sound card or other recording equipment and obtain audio data; The audio acquisition module stores the collected audio data in the storage module; The central processing unit is connected to the memory and sends the audio data in the memory to the audio processing module for audio processing to achieve noise elimination; The audio processing module stores the audio data after noise elimination in the memory, and the central processing unit outputs the audio data after noise elimination in the memory through the output module; The audio processing module is connected to the spectrum analysis module and the EQ adjustment module; the spectrum analysis module is used to perform spectrum analysis on the audio data, and the EQ adjustment module is used to perform EQ adjustment on the audio data; The sampling rate of the audio acquisition module is 44100Hz and the bit rate is 128bit. The audio sampling rate of the output module is 44100Hz and the bit rate is 128bit.

3. The noise detection method according to claim 1, wherein: In steps 4 and 6, the multi-peak fitting obtains the number of fitted peaks and the range of all fitted peak intensities. In the similarity analysis, the number of fitted peaks, the frequency of each fitted peak, the intensity of each fitted peak, and the range of fitted peak intensities are used as similarity calculation parameters.

4. A noise elimination method, characterized in that: The specific steps include: Step A: The audio acquisition module acquires audio data to be detected and performs noise detection using the noise detection method described in claim 1; Step B: directly muting the audio segments with energy less than A0, the audio segments corresponding to occasional noise, and the audio segments corresponding to speech noise in the audio data; Step C: Spectral analysis is performed on the audio segment corresponding to the normal speech, and then EQ adjustment is performed to attenuate the frequency of the fitting peak corresponding to the background noise spectrum in the spectrum of the audio segment corresponding to the normal speech, with the attenuation amplitude being the same as the intensity of the fitting peak corresponding to the background noise spectrum; Step D: further segmenting the audio segments corresponding to normal speech, with the duration of each segment being the same as the average duration of the audio segments corresponding to multiple occasional noises, and performing spectrum analysis on the segmented normal speech audio segments and similarity analysis on the spectrum with the multiple occasional noises; Step E: Screen out the ordinary speech audio segments whose similarity in step D is greater than the threshold, and then weaken the frequency of the fitting peak corresponding to the multiple occasional noise spectrum in the frequency spectrum of the ordinary speech audio segments, with the weakening amplitude being the same as the intensity of the fitting peak corresponding to the multiple occasional noise spectrum; Step F: splice all the audio clips in their original order to obtain audio data after noise elimination.

5. The noise elimination method according to claim 4, characterized in that: The sampling rate of the audio acquisition module is 44100Hz and the bit rate is 128bit. The audio sampling rate of the output module is 44100Hz and the bit rate is 128bit.

Images