A single channel noise detection method and device

Abstract

The present invention discloses a single-channel noise detection method and device, including: inputting a mixed sound signal into a pre-trained noise separation model to obtain a plurality of single-source noise signals; the noise separation model is constructed according to a time domain encoder and a dual-path recurrent neural network; the separated single-source noise signals are input into a pre-trained noise enhancement model, and each single-source noise signal is enhanced respectively; the noise enhancement model is constructed according to a time-frequency domain generative adversarial network; each enhanced single-source noise signal is obtained, and each single-source noise signal is evaluated according to preset indicators to complete noise detection. The present invention separates mixed sound signals by constructing a noise separation model through a time domain encoder and a dual-path recurrent neural network, which can quickly and effectively separate the noise source in the mixed sound signal and improve the accuracy of noise monitoring.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a single-channel noise detection method and device. Background Art

[0002] Noise pollution prevention and control is related to the physical and mental health of the people. It is the most universal livelihood project and an important part of ecological civilization construction and ecological environment protection. At present, there are many types of noise in complex acoustic scenes, and there is a lack of effective intelligent noise quantification and analysis technology, which makes it impossible to clearly determine which type of noise exceeds the standard in the scene. Existing noise detection methods usually identify various common noises in the scene through sound separation.

[0003] Existing sound separation methods generally use neural networks to perform local feature modeling and global feature modeling, and use multi-scale time-delay sampling modeling to make up for the information missing in piecewise modeling. Although this method improves the accuracy of sound separation to a certain extent, it is impossible to achieve real-time detection due to the large number of model parameters and long processing time in the piecewise modeling. Summary of the invention

[0004] The present invention provides a single-channel noise detection method and device to solve the technical problem that the existing sound detection method cannot take into account both the detection accuracy and the real-time performance of sound separation processing.

[0005] In order to solve the above technical problems, an embodiment of the present invention provides a single-channel noise detection method, comprising:

[0006] Inputting the mixed sound signal into a pre-trained noise separation model to obtain a number of single-source noise signals; the noise separation model is constructed based on a time domain encoder and a dual-path recurrent neural network;

[0007] The separated single-source noise signals are input into a pre-trained noise enhancement model to perform enhancement operations on each single-source noise signal respectively; the noise enhancement model is constructed according to a time-frequency domain generative adversarial network;

[0008] The enhanced single-source noise signals are obtained, and the single-source noise signals are evaluated according to preset indicators to complete the noise detection.

[0009] The present invention separates mixed sound signals by constructing a noise separation model through a time domain encoder and a dual-path recurrent neural network. By capturing the characteristic information of the sound signal through the dual paths, the noise source in the mixed sound signal can be quickly and effectively separated. At the same time, the time-frequency domain generative adversarial network enhancement module is used to enhance the separated single-source noise, making the single-source noise clearer, facilitating the subsequent evaluation of the single-source noise according to indicators, thereby completing the quantitative analysis of the noise and improving the accuracy of noise monitoring.

[0010] Furthermore, the mixed sound signal is input into a pre-trained noise separation model to obtain a number of single-source noise signals, specifically:

[0011] Extracting features of the mixed sound signal according to a time domain encoder to obtain a time-frequency feature representation of the mixed sound signal;

[0012] Inputting the time-frequency feature representation into the dual-path neural recurrent network, extracting the time domain features and frequency features of the time domain feature representation, and acquiring time series information according to the time domain features and frequency features; wherein the forward path of the dual-path neural recurrent network extracts the time domain information of the audio signal, and the backward path of the dual-path neural recurrent network extracts the frequency domain information of the audio signal;

[0013] Inputting the time series information into the first modeling layer and the second modeling layer to obtain local features and context information of the time series information;

[0014] A plurality of single-source masks are acquired according to the local features and the context information, and the single-source masks are converted into single-source noise signals through a decoder.

[0015] Furthermore, the step of inputting the time series information into the first modeling layer and the second modeling layer to obtain the local features and context information of the time series information is specifically as follows:

[0016] Extracting features of the time series information within a local time range according to the first modeling layer to obtain local features of the time series information;

[0017] The local features and time series information are modeled and integrated according to the second modeling layer to obtain context information of the mixed sound signal.

[0018] Furthermore, before the separated single-source noise signal is input into the pre-trained noise enhancement model, the method further includes:

[0019] The weight parameters of the noise enhancement model are adjusted according to the discriminator network, and the noise enhancement model includes an encoder and a time-frequency domain generative adversarial network.

[0020] Furthermore, the weight parameters of the noise enhancement model are adjusted according to the discriminator network, specifically:

[0021] Inputting the first audio data into the encoder and the time-frequency domain generative adversarial network in sequence to obtain the second audio data; the first audio data is each single-source noise signal after the original noise data is processed by the noise separation model;

[0022] Calculate a similarity index value between the second audio data and the original audio data according to the discriminator network;

[0023] An identification operation is performed on the second audio data according to a preset threshold and the similarity index value, and a weight parameter of the noise enhancement model is adjusted according to the identification result.

[0024] Furthermore, the separated single-source noise signals are input into a pre-trained noise enhancement model, and enhancement operations are performed on each single-source noise signal, specifically:

[0025] Performing conversion operations on each single-source noise signal according to short-time Fourier transform to obtain a time-frequency spectrum diagram of each single-source noise signal;

[0026] The time-frequency spectrograms are sequentially input into an encoder and a time-frequency domain generative adversarial network for enhancement operations to obtain time-frequency features of each time-frequency spectrogram; the time-frequency domain generative adversarial network includes several TS-Conformer layers;

[0027] The time-frequency features are processed according to the mask decoder and the complex decoder to generate second audio data of each single-source noise signal, where the second audio data is an audio segment after each single-source noise signal is enhanced.

[0028] Furthermore, the acquisition of each enhanced single-source noise signal, and evaluation of each single-source noise signal according to a preset index to complete noise detection are specifically as follows:

[0029] Obtain each enhanced single-source noise signal, and calculate the A-weighted sound pressure level of each single-source noise signal;

[0030] Each single-source noise signal is graded and classified according to the A-weighted sound pressure level, and a detection report is output to complete noise monitoring.

[0031] Furthermore, before inputting the mixed sound signal into the pre-trained noise separation model, the method further includes:

[0032] Adjusting the sampling rate of each sound signal in the mixed sound signal according to a preset sampling rate to obtain a mixed sound signal with a consistent sampling rate;

[0033] and calculating a normalization ratio according to the maximum dBFS value of the mixed sound signal, and adjusting the waveform of each audio signal in the mixed sound signal according to the normalization ratio;

[0034] The mixing ratio of the mixed sound signal is dynamically adjusted according to the noise detection scenario, and the mixed sound signal is data enhanced to construct a mixed sound signal set; the data enhancement includes signal-to-noise ratio adjustment, time shift operation and multi-sound source mixing operation.

[0035] In a second aspect, the present invention provides a single-channel noise detection device, comprising: a noise separation module, a noise enhancement module and an evaluation module;

[0036] The noise separation module is used to input the mixed sound signal into a pre-trained noise separation model to obtain a plurality of single-source noise signals; the noise separation model is constructed according to a time domain encoder and a dual-path recurrent neural network;

[0037] The noise enhancement module is used to input the separated single-source noise signals into a pre-trained noise enhancement model to perform enhancement operations on each single-source noise signal; the noise enhancement model is constructed according to a time-frequency domain generative adversarial network;

[0038] The evaluation module is used to obtain each enhanced single-source noise signal, evaluate each single-source noise signal according to a preset index, and complete noise detection.

[0039] In a third aspect, the present invention provides a computer device, comprising: a processor, a communication interface and a memory, wherein the processor, the communication interface and the memory are interconnected, wherein the memory stores an executable program code, and the processor is used to call the executable program code to execute the single-channel noise detection method. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic diagram of a flow chart of a single-channel noise detection method provided by an embodiment of the present invention;

[0041] Figure 2 A schematic diagram of a training process of a noise separation model provided in an embodiment of the present invention;

[0042] Figure 3 A schematic diagram of the structure of a time domain encoder provided by an embodiment of the present invention;

[0043] Figure 4 A schematic diagram of a dual-path recurrent neural network structure provided by an embodiment of the present invention;

[0044] Figure 5 A schematic diagram of a decoder structure of a single-channel noise detection method provided by an embodiment of the present invention;

[0045] Figure 6 A schematic diagram of an encoder structure of a noise enhancement model provided in an embodiment of the present invention;

[0046] Figure 7 A schematic diagram of the structure of a time-frequency domain generative adversarial network provided by an embodiment of the present invention;

[0047] Figure 8A schematic diagram of a mask decoder of a noise enhancement model provided in an embodiment of the present invention;

[0048] Fig. 9 A schematic diagram of a complex decoder of a noise enhancement model provided in an embodiment of the present invention;

[0049] Fig.10 A schematic diagram of a discriminator network structure of a single-channel noise detection method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] Embodiment 1

[0052] Please refer to Figure 1 , Figure 1 A schematic flow chart of a single-channel noise detection method provided by an embodiment of the present invention includes steps 101 to 103, which are specifically as follows:

[0053] Step 101: inputting a mixed sound signal into a pre-trained noise separation model to obtain a plurality of single-source noise signals; the noise separation model is constructed according to a time domain encoder and a dual-path recurrent neural network;

[0054] In this embodiment, before inputting the mixed sound signal into the pre-trained noise separation model, the method further includes:

[0055] Adjusting the sampling rate of each sound signal in the mixed sound signal according to a preset sampling rate to obtain a mixed sound signal with a consistent sampling rate;

[0056] and calculating a normalization ratio according to the maximum dBFS value of the mixed sound signal, and adjusting the waveform of each audio signal in the mixed sound signal according to the normalization ratio;

[0057] The mixing ratio of the mixed sound signal is dynamically adjusted according to the noise detection scenario, and the mixed sound signal is data enhanced to construct a mixed sound signal set; the data enhancement includes signal-to-noise ratio adjustment, time shift operation and multi-sound source mixing operation.

[0058] In this embodiment, a clean noise database is established by collecting pure noise sources and preprocessing the pure noise sources. The specific preprocessing operations include standardization processing and data enhancement expansion, so that the mixed noise signals in the noise database can achieve sampling rate matching, energy normalization, duration alignment, volume balance, mixing ratio control and data enhancement.

[0059] In this embodiment, if the mixed speech signal set does not match the preset sampling rate, it can be adjusted according to the preset sampling rate to ensure that all speech signals have the same sampling rate, and the sampling rate is preset according to the model or algorithm requirements.

[0060] In this embodiment, the volume of the mixed sound signal is normalized according to dBFS (decibels relative to full scale). Specifically, the normalization ratio calculation formula is:

[0061] dBFS new =20×log 10 (max|waveform old |) (1)

[0062] Among them, waveform old is the input waveform before normalization, dBFS input is the dBFS value of the input audio waveform signal.

[0063] Normalizing each sound signal in the mixed sound signal according to the normalization ratio, specifically:

[0064]

[0065] Among them, waveform new is the normalized waveform, waveform old is the input waveform before normalization.

[0066] In this embodiment, the energy balance and consistency of the mixed speech are ensured by volume normalization to avoid excessive volume of certain source signals, thereby affecting the performance and results of the noise separation model.

[0067] In this embodiment, the mixing ratio is dynamically adjusted during the training process so that the noise separation model and the noise enhancement model can adapt to speech separation tasks with different mixing ratios. By introducing samples with different mixing ratios in the training data according to the detection scenario or detection task, the contribution of different audio sources in the mixed speech is balanced, ensuring that the signal of the target audio source can be accurately extracted and separated, thereby improving the robustness and generalization ability of the model.

[0068] In this embodiment, data enhancement is performed on the mixed sound signal to generate new samples. The data enhancement includes signal-to-noise ratio adjustment, time shift operation and multi-sound source mixing operation.

[0069] In this embodiment, by adjusting the SNR value of the mixed sound signal, different noise levels and noise interference degrees can be simulated. By randomly applying different SNR values, different noise levels can be simulated, data samples can be increased, and noise separation scenarios can be increased. Thus, by simulating the training of the model under different noise levels and signal-to-noise ratio conditions, the robustness and generalization ability of the model can be improved.

[0070] In this embodiment, the mixed speech signal is shifted in time to increase the diversity of training data, so as to improve the model's separation effect on sound signals with different time changes and time offsets. And by superimposing multiple clean noise signals together, the situation where there are multiple noise sources in a real scene is simulated. The sample diversity in the mixed sound signal set is increased, and the noise detection accuracy of the model is improved.

[0071] The present invention adopts data preprocessing and data enhancement methods to improve the richness and diversity of data by processing and expanding existing pure noise sources, so as to train noise separation models and noise enhancement models according to diverse data samples to improve the accuracy and stability of noise separation and quantitative analysis.

[0072] In this embodiment, a noise separation model is constructed based on a time-frequency domain generative adversarial network, and the noise separation model is trained based on audio data in a mixed sound signal set.

[0073] Please refer to Figure 2 , Figure 2 A schematic diagram of a training process of a noise separation model provided in an embodiment of the present invention.

[0074] In this embodiment, a mixed sound signal is obtained from a mixed sound signal set, and a feature representation of the mixed sound signal is extracted by an encoder. Then, a separation mask of the mixed sound signal is obtained according to a dual-path recurrent neural network, and then the separation mask is decoded by a decoder to generate a separation noise source.

[0075] In this embodiment, the encoder is specifically a time domain encoder, and the input multi-channel mixed noise signal is converted into a time domain feature representation through the time domain encoder.

[0076] Please refer to Figure 3 , Figure 3 A schematic diagram of the structure of a time domain encoder provided by an embodiment of the present invention.

[0077] In this embodiment, the time domain encoder includes a 1D convolution layer, an activation function and a fully connected layer. The 1D convolution layer is used to filter and extract features from the input speech signal in the time domain. The activation function is usually applied to the output of the convolution layer to introduce nonlinear transformation.

[0078] Please refer to Figure 4 , Figure 4 A schematic diagram of the dual-path recurrent neural network structure provided for an embodiment of the present invention.

[0079] In this embodiment, a dual-path recurrent neural network is formed according to a series of bidirectional recurrent neural network units, and each unit has two paths, a forward path and a backward path. The forward path and the backward path in the dual-path recurrent neural network extract time domain features and frequency features from the output of the time domain encoder respectively, and enhance the expressiveness of the features through cross-path connection. In each bidirectional recurrent neural network unit, the input features are processed by the forward and backward paths to obtain the output features of the unit. By forward and backward propagating information in the time dimension, the bidirectional recurrent neural network can capture the contextual information of the input sequence.

[0080] In this embodiment, the dual-path recurrent neural network includes a local feature modeling layer and a global feature modeling layer. During training, the mixed sound signal is encoded and feature extracted, and then the slices are spliced ​​into 3D tensors and input into the local feature modeling layer. The local feature modeling layer mainly models and enhances local features, including a one-dimensional convolutional layer, a nonlinear activation function, and a normalization layer. Then it is input into the global feature modeling layer, and the global feature modeling layer mainly models and integrates global features. Finally, the output results are processed and overlap-added to obtain a separation mask.

[0081] Please refer to Figure 5 , Figure 5 A schematic diagram of a decoder structure of a single-channel noise detection method provided by an embodiment of the present invention.

[0082] In this embodiment, the extracted features are further processed according to the decoder to restore the time domain and frequency domain details of the source audio. Specifically, the decoder captures the time-frequency feature differences between the source audios according to operations such as convolution and pooling. The separated features are processed by operations such as inverse convolution to restore the frequency domain details of the source audio. The size of the feature map is expanded by the inverse convolution operation so that it has the same time and frequency resolution as the original audio. The activation function introduces nonlinear properties to increase the expressiveness and adaptability of the model.

[0083] In this embodiment, during the model training stage, the noise separation model needs to be jointly optimized according to several loss functions, wherein the loss function includes a scale-invariant signal-to-noise ratio, a waveform loss function, a mask and a loss function.

[0084] In this embodiment, the scale-invariant signal-to-noise ratio is specifically:

[0085]

[0086] Among them, s i is the speech component of the ith noise source in the original mixed signal, e i It is the residual between the estimated signal of the ith noise source obtained after separation and its original speech signal (i.e., separation error).

[0087] In this embodiment, the waveform loss function is specifically:

[0088] Loss2 = MSE (waveform estimated ,waveform origin ) (4)

[0089] Among them, waveform estimated Waveform estimated for the separation network, waveform origin is the corresponding original single source waveform.

[0090] In this embodiment, the mask and loss function are specifically:

[0091] Loss3=MSE(∑mask i ,1) (5)

[0092] Among them, mask i is the estimated mask of the i-th source. ∑mask i Add the corresponding positions of the masks of all sources.

[0093] In this embodiment, the noise separation model is jointly trained according to three loss functions, and the total loss function formula of the noise separation model is as follows:

[0094] Loss tatal =Loss1+10*Loss2+100*Loss3 (6)

[0095] In this embodiment, the single-source noise signal separated by the noise separation model and the original pure single-source signal are used to iteratively update the network parameters with the total loss function as the improvement benchmark. The iteration is stopped until convergence or reaching the target performance, and the noise separation model training is completed.

[0096] In this embodiment, the robustness of the noise separation model to the signal amplitude is improved according to the scale-invariant signal-to-noise ratio, waveform loss function, mask and loss function, and the noise separation model is adapted to the data distribution and changes of different sound categories, so that the model can handle diverse sound scenes. And by minimizing the gap between "the sum of each single source mask" and "1" as the loss function, the physical feasibility of the separation result can be ensured, the separation performance can be improved, over-separation or under-separation can be avoided, and numerical stability can be improved.

[0097] In this embodiment, the mixed sound signal is input into a pre-trained noise separation model to obtain a number of single-source noise signals, specifically:

[0098] Extracting features of the mixed sound signal according to a time domain encoder to obtain a time-frequency feature representation of the mixed sound signal;

[0099] Inputting the time-frequency feature representation into the dual-path neural recurrent network, extracting the time domain features and frequency features of the time domain feature representation, and acquiring time series information according to the time domain features and frequency features; wherein the forward path of the dual-path neural recurrent network extracts the time domain information of the audio signal, and the backward path of the dual-path neural recurrent network extracts the frequency domain information of the audio signal;

[0100] Inputting the time series information into the first modeling layer and the second modeling layer to obtain local features and context information of the time series information;

[0101] A plurality of single-source masks are acquired according to the local features and the context information, and the single-source masks are converted into single-source noise signals through a decoder.

[0102] In this embodiment, the time-frequency feature representation of the mixed sound signal is extracted according to the time domain encoder, and the time domain feature is input into the dual-path recurrent neural network. After the time domain feature is processed by the forward and backward paths, the output feature of the unit is obtained, and the time series information is obtained by forward and backward propagation of information in the time dimension, and the time series information is input into the first modeling layer and the second modeling layer to obtain local features and context information, and finally the local feature information and context information are modeled and integrated, and the output results are processed and overlap-added to obtain the separation mask.

[0103] In this embodiment, the time series information is input into the first modeling layer and the second modeling layer to obtain the local features and context information of the time series information, specifically:

[0104] Extracting features of the time series information within a local time range according to the first modeling layer to obtain local features of the time series information;

[0105] The local features and time series information are modeled and integrated according to the second modeling layer to obtain context information of the mixed sound signal.

[0106] In this embodiment, the first modeling layer is a local feature modeling layer, and the second modeling layer is a global feature modeling layer. In the local feature modeling layer, local features are modeled and enhanced according to a one-dimensional convolution layer, a nonlinear activation function, and a normalization layer, and a sliding window process is performed on the time series information through a 1D convolution operation, and features are extracted within a local time range using a convolution kernel to capture the local structure and related information of the time series information. The local structure and related information are input into the global feature modeling layer, and the entire time series information is modeled according to a recurrent neural network in the global feature modeling layer to obtain global context information.

[0107] In this embodiment, the estimated single source mask is obtained by slicing, intra-block modeling, inter-block modeling, and finally overlapping and adding the feature representation output by the encoder, and then the single source mask is restored to the separated single source noise signal through the decoder.

[0108] Step 102: Input the separated single-source noise signals into a pre-trained noise enhancement model, and perform enhancement operations on each single-source noise signal respectively; the noise enhancement model is constructed according to a time-frequency domain generative adversarial network;

[0109] In this embodiment, the separated single-source noise signals are input into a pre-trained noise enhancement model, and enhancement operations are performed on each single-source noise signal, specifically:

[0110] Performing conversion operations on each single-source noise signal according to short-time Fourier transform to obtain a time-frequency spectrum diagram of each single-source noise signal;

[0111] The time-frequency spectrograms are sequentially input into an encoder and a time-frequency domain generative adversarial network for enhancement operations to obtain time-frequency features of each time-frequency spectrogram; the time-frequency domain generative adversarial network includes several TS-Conformer layers;

[0112] The time-frequency features are processed according to the mask decoder and the complex decoder to generate second audio data of each single-source noise signal, where the second audio data is an audio segment after each single-source noise signal is enhanced.

[0113] In this embodiment, the noise enhancement model is constructed by a time-frequency domain generative adversarial network. During training, the single-source noise signal output by the noise separation model is obtained, the speech waveform is converted into a complex time-frequency spectrum, and then the compressed spectrum is obtained by power-law compression to obtain the amplitude, phase, real part and imaginary part of the compressed spectrum. The amplitude, real part and imaginary part of the compressed spectrum are then spliced ​​together as the input of the encoder, and the feature representation of the time-frequency spectrum is extracted by the encoder, which is then input into the generator network to generate the predicted clean audio according to the decoder. Figure 6 As shown, Figure 6 A schematic diagram of an encoder structure of a noise enhancement model provided in an embodiment of the present invention.

[0114] Please refer to Figure 7 , Figure 7 A schematic diagram of the structure of a time-frequency domain generative adversarial network provided in an embodiment of the present invention.

[0115] In this embodiment, the time-frequency domain generative adversarial network is a generator network, and the generator network includes four TS-Conformer layers. The TS-Conformer layer TS-Conformer is a model that combines Transformer and Conformer, which includes two Conformers. The Conformer can capture long-distance dependencies, so that the model can better understand contextual information and improve the accuracy of sequence modeling.

[0116] Please refer to Fig. 9 and Fig.10 , Fig. 9 A schematic diagram of a mask decoder of a noise enhancement model provided in an embodiment of the present invention; Fig.10 A schematic diagram of a complex decoder of the noise enhancement model provided in an embodiment of the present invention.

[0117] In this embodiment, the decoder extracts output from the 4-layer TS-Conformer module in a decoupled manner as input to the decoder. The decoder includes a mask decoder and a complex decoder.

[0118] In this embodiment, the mask decoder includes an expanded Dense Net module, a sub-pixel convolution layer and two convolution layers. The expanded Dense Net module is the same as the encoder. Through mechanisms such as expanded convolution, dense connection and feature reuse, the decoder's receptive field and context understanding ability are enhanced, the decoder's performance is improved, and a more accurate and high-quality speech reconstruction result is produced. The sub-pixel convolution layer is used to convert the frequency dimension back to its initial size.

[0119] In this embodiment, the complex decoder combines the output amplitude of the mask decoder with the phase of the compressed spectrogram to obtain an amplitude-enhanced complex spectrogram, and then adds the real and imaginary outputs of the complex decoder element by element to obtain the final complex spectrogram. The final complex spectrogram is subjected to inverse power law compression and inverse short-time Fourier transform (ISTFT) to obtain the final enhanced audio clip.

[0120] In this embodiment, before the separated single-source noise signal is input into the pre-trained noise enhancement model, the method further includes:

[0121] The weight parameters of the noise enhancement model are adjusted according to the discriminator network, and the noise enhancement model includes an encoder and a time-frequency domain generative adversarial network.

[0122] In this embodiment, the weight parameters of the noise enhancement model are adjusted according to the discriminator network, specifically:

[0123] Inputting the first audio data into the encoder and the time-frequency domain generative adversarial network in sequence to obtain the second audio data; the first audio data is each single-source noise signal after the original noise data is processed by the noise separation model;

[0124] Calculate a similarity index value between the second audio data and the original audio data according to the discriminator network;

[0125] An identification operation is performed on the second audio data according to a preset threshold and the similarity index value, and a weight parameter of the noise enhancement model is adjusted according to the identification result.

[0126] Please refer to Fig.10 , Fig.10 A schematic diagram of a discriminator network structure of a single-channel noise detection method provided by an embodiment of the present invention.

[0127] In this embodiment, the quality of the speech clips generated by the generator network is judged by calculating the similarity evaluation score between the speech generated by the generator and the original audio. The discriminator consists of four convolution blocks, after which global average pooling is performed, and then two feedforward layers are passed, and finally a Sigmoid activation function is used to obtain the final result.

[0128] In this embodiment, specifically, the PESQ score is used as the evaluation index of the discriminator, and the original audio data and the second audio data are used as input to estimate the maximum normalized PESQ score. The discriminator presets a threshold, and when the PESQ score between the original audio data and the second audio data reaches the threshold, it is considered that the generated speech is good enough, and the generated speech is finally output; when the PESQ score between the original audio data and the second audio data is lower than the threshold, the speech is re-input into the generator network to continue training until the evaluation result meets the preset threshold and outputs this part of the speech.

[0129] In this embodiment, the sound signal that has not passed the identification of the discriminator is re-transmitted to the generator, and the generator readjusts the parameters of the network model according to the joint Loss function to make the generator output closer to the real noise distribution, that is, the sound signal that has not passed the identification of the discriminator before is passed through the generator again until the generated noise signal can pass the identification of the discriminator. At the same time, the generator will also continuously update the discriminator accordingly, adjust the discriminator parameters, and improve the discriminator's ability to identify noise signals.

[0130] In this embodiment, the joint loss function includes the joint error in time and frequency, the error between waveform signals and the error between amplitude information obtained by the discriminator. Specifically:

[0131] The joint error in time and frequency includes a linear combination of amplitude error and real and imaginary part error, and the expression is as follows:

[0132] L TF =αL Mag .+(1―α)L RI (7)

[0133] Here, α is a hyperparameter, and we set the value of α to 0.7. Mag is the amplitude error between the predicted result and the original result, L RI is the error between the real and imaginary parts of the predicted result and the original result. Mag , L RI Using mean square error to calculate, the respective expressions are as follows:

[0134]

[0135] Among them, X m Represents the amplitude information of the original speech, Represents the amplitude information of the predicted speech, and E represents the mean square error operation.

[0136]

[0137] Among them, X r Represents the real part of the original speech, Represents the real part information of the predicted speech; X i Represents the imaginary part of the original speech, represents the imaginary part of the predicted speech, and E represents the mean square error operation.

[0138] The error between the waveform signals is the error between the predicted time domain waveform signal and the original waveform signal. The specific formula is as follows:

[0139]

[0140] Among them, x represents the time domain signal of the original speech, The error between the time domain waveform signals is obtained by calculating the mean absolute difference between the two.

[0141] The discriminator obtains the error between the amplitude information for the discriminator partial error between the predicted speech signal and the original speech signal:

[0142]

[0143] Among them, X m Represents the amplitude information of the original speech, Represents the amplitude information of the predicted speech, Q PESQ Represents the maximum normalized PESQ score, with a value between (0, 1), and E represents the mean square error operation.

[0144] The error of the generator part is as follows:

[0145]

[0146] Where X m Represents the amplitude information of the original speech, Represents the amplitude information of the predicted speech, and E represents the mean square error operation.

[0147] The joint loss function of the generator is composed of the linear combination of these three parts, as follows:

[0148] L G =γ1L TF +γ2L GAN +γ3L Time (13)

[0149] Among them, γ1, γ2, and γ3 are also hyperparameters. According to experience, γ1, γ2, and γ3 can be set to 0.9, 0.2, and 0.05, respectively.

[0150] In this embodiment, a Conformer-based time-frequency domain generative adversarial network enhancement module is used, which combines the advantages of convolutional layers and Transformer layers to capture both local relationships and long-term dependencies. This can effectively extract and enhance the time-frequency features of single-source noise, making it clearer and easier to analyze.

[0151] Step 103: Acquire each enhanced single-source noise signal, evaluate each single-source noise signal according to a preset index, and complete noise detection.

[0152] In this embodiment, the acquisition of each enhanced single-source noise signal, and evaluation of each single-source noise signal according to a preset index to complete noise detection are specifically as follows:

[0153] Obtain each enhanced single-source noise signal, and calculate the A-weighted sound pressure level of each single-source noise signal;

[0154] Each single-source noise signal is graded and classified according to the A-weighted sound pressure level, and a detection report is output to complete noise monitoring.

[0155] In this embodiment, the A-weighted sound pressure level is used as a quantitative index for analysis. By quantitatively analyzing the single-source noise after separation and enhancement, the level and impact of the noise can be more accurately evaluated, providing a basis for subsequent processing and decision-making.

[0156] The present invention also provides a single-channel noise detection device, comprising: a noise separation module, a noise enhancement module and an evaluation module;

[0157] The noise separation module is used to input the mixed sound signal into a pre-trained noise separation model to obtain a plurality of single-source noise signals; the noise separation model is constructed according to a time domain encoder and a dual-path recurrent neural network;

[0158] The noise enhancement module is used to input the separated single-source noise signals into a pre-trained noise enhancement model to perform enhancement operations on each single-source noise signal; the noise enhancement model is constructed according to a time-frequency domain generative adversarial network;

[0159] The evaluation module is used to obtain each enhanced single-source noise signal, evaluate each single-source noise signal according to a preset index, and complete noise detection.

[0160] The present invention also provides a computer device, comprising: a processor, a communication interface and a memory, wherein the processor, the communication interface and the memory are connected to each other, wherein the memory stores an executable program code, and the processor is used to call the executable program code to execute the single-channel noise detection method.

[0161] In this embodiment, a noise separation model is constructed by a time domain encoder and a dual-path recurrent neural network to separate the mixed sound signal. By capturing the characteristic information of the sound signal through the dual paths, the noise source in the mixed sound signal can be quickly and effectively separated. At the same time, the time-frequency domain generative adversarial network enhancement module is used to enhance the separated single-source noise, making the single-source noise clearer, facilitating the subsequent evaluation of the single-source noise according to indicators, and improving the accuracy of noise monitoring.

[0162] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A single-channel noise detection method, characterized in that: include: The sampling rate of each sound signal in the mixed sound signal is adjusted according to a preset sampling rate to obtain a mixed sound signal with a consistent sampling rate; and a normalization ratio is calculated according to a maximum dBFS value and a minimum dBFS value of the mixed sound signal, and the waveform of each audio signal in the mixed sound signal is adjusted according to the normalization ratio; the mixing ratio of the mixed sound signal is dynamically adjusted according to a noise detection scenario, and the mixed sound signal is data enhanced to construct a mixed sound signal set; the data enhancement includes signal-to-noise ratio adjustment, time shift operation and multi-sound source mixing operation; Input the mixed sound signal into a pre-trained noise separation model to obtain several single-source noise signals; the noise separation model is constructed according to a time domain encoder and a dual-path recurrent neural network, specifically: extract features from the mixed sound signal according to the time domain encoder to obtain a time-frequency feature representation of the mixed sound signal; input the time-frequency feature representation into the dual-path recurrent neural network, extract the time domain features and frequency features of the time domain feature representation, and obtain time series information according to the time domain features and frequency features; wherein the forward path of the dual-path recurrent neural network extracts the time domain information of the audio signal, and the backward path of the dual-path recurrent neural network extracts the frequency domain information of the audio signal; input the time series information into the first modeling layer and the second modeling layer, extract features from the time series information within a local time range according to the first modeling layer, and obtain local features of the time series information; model and integrate the local features and the time series information according to the second modeling layer to obtain context information of the mixed sound signal; obtain several single-source masks according to the local features and the context information, and convert the single-source masks into single-source noise signals through a decoder; The weight parameters of the noise enhancement model are adjusted according to the discriminator network, and the noise enhancement model includes an encoder and a time-frequency domain generative adversarial network. Specifically, the first audio data is sequentially input into the encoder and the time-frequency domain generative adversarial network to obtain the second audio data; the first audio data is each single-source noise signal after the original noise data is processed by the noise separation model; the similarity index value of the second audio data and the original audio data is calculated according to the discriminator network; the second audio data is discriminated according to a preset threshold and the similarity index value, and the weight parameters of the noise enhancement model are adjusted according to the discrimination result; The separated single-source noise signal is input into a pre-trained noise enhancement model, and each single-source noise signal is enhanced respectively. Specifically, the time-frequency spectrum is sequentially input into an encoder and a time-frequency domain generative adversarial network for enhancement, and the time-frequency features of the time-frequency spectrum of each single-source noise signal are obtained; the time-frequency domain generative adversarial network includes a plurality of TS-Conformer layers; the time-frequency features are processed according to a mask decoder and a complex decoder to generate second audio data of each single-source noise signal, and the second audio data is an audio segment after each single-source noise signal is enhanced; the noise enhancement model is constructed according to the time-frequency domain generative adversarial network; The enhanced single-source noise signals are obtained, and the single-source noise signals are evaluated according to preset indicators to complete the noise detection. Specifically, the enhanced single-source noise signals are obtained, and the A-weighted sound pressure level of each single-source noise signal is calculated; each single-source noise signal is graded and classified according to the A-weighted sound pressure level, and a detection report is output to complete the noise monitoring.

2. A single-channel noise detection device, characterized in that: Used to implement a single-channel noise detection method as claimed in claim 1, comprising: a noise separation module, a noise enhancement module and an evaluation module; The noise separation module is used to input the mixed sound signal into a pre-trained noise separation model to obtain a plurality of single-source noise signals; the noise separation model is constructed according to a time domain encoder and a dual-path recurrent neural network; The noise enhancement module is used to input the separated single-source noise signals into a pre-trained noise enhancement model to perform enhancement operations on each single-source noise signal; the noise enhancement model is constructed according to a time-frequency domain generative adversarial network; The evaluation module is used to obtain each enhanced single-source noise signal, evaluate each single-source noise signal according to preset indicators, and complete noise detection. Specifically: obtain each enhanced single-source noise signal, calculate the A-weighted sound pressure level of each single-source noise signal; grade and classify each single-source noise signal according to the A-weighted sound pressure level, and output a detection report to complete noise monitoring.

3. A computer device, characterized in that: include: A processor, a communication interface and a memory, wherein the processor, the communication interface and the memory are connected to each other, wherein the memory stores an executable program code, and the processor is used to call the executable program code to execute the single-channel noise detection method as claimed in claim 1.

Images