Speech noise reduction method, device and storage medium

Abstract

The present application discloses a speech denoising method, device and storage medium, and the present application relates to the field of speech recognition technology. The method acquires a collected speech signal, decomposes the speech signal into spectrum blocks based on short-time Fourier transform, and calculates the absolute value of the spectrum blocks to generate a power spectrum. Then, the noise spectrum of the noise signal is acquired, a gain value is determined according to the noise spectrum and the power spectrum, and the gain value is applied to the spectrum block to generate a target spectrum block. Then, the target spectrum block is converted into a target signal based on an inverse short-time Fourier transform, and the eigenvalue in the target signal is extracted according to the Bark band scale. Finally, the eigenvalue and the target signal are input into a denoising model, and a target language signal is output. The above method pre-processes the speech signal through short-time Fourier transform, and then combines the Bark band scale and the denoising model to eliminate non-stationary noise in the speech signal.

Description

Technical Field

[0001] The present application relates to the field of speech recognition technology, and in particular to a speech noise reduction method, device and storage medium. Background Art

[0002] As remote conferencing becomes more and more popular, noise reduction in voice communications has become a key factor in the conference experience. At present, voice communication noise is generally divided into two types: stationary noise and non-stationary noise. Related voice communication noise reduction technologies are mainly aimed at stationary noise, such as constant air conditioning noise, etc. Generally, they are designed by first detecting the frequency band where the noise occurs, and then designing a linear filter or filter group to eliminate the noise. In actual processes, many noises are non-stationary noises, such as sudden impact sounds, keyboard tapping, and table tapping sounds. Since the frequency band of such noises is unstable and difficult to detect and record, the related technologies cannot eliminate non-stationary noises in voice communications.

[0003] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The present application provides a speech noise reduction method, device and storage medium, aiming to solve the problem that related technologies cannot eliminate non-stationary noise in voice communication.

[0005] To achieve the above object, the present application provides a method for speech noise reduction, which comprises the following steps:

[0006] Acquire the collected voice signal;

[0007] Decomposing the speech signal into spectrum blocks based on short-time Fourier transform, and calculating the absolute value of the spectrum blocks to generate a power spectrum;

[0008] Acquire a noise spectrum of a noise signal, determine a gain value according to the noise spectrum and the power spectrum, and apply the gain value to the spectrum block to generate a target spectrum block;

[0009] Converting the target spectrum block into a target signal based on an inverse short-time Fourier transform, and extracting a characteristic value in the target signal according to a Bark band scale;

[0010] The characteristic value and the target signal are input into a noise reduction model, and a target language signal is output.

[0011] In one embodiment, the step of extracting the characteristic value and the signal to be processed in the target signal according to the Bark frequency band scale comprises:

[0012] acquiring the target signal;

[0013] Convert the frequency axis of the target signal into Bark units based on a frequency band scale conversion formula, and map it to the Bark domain;

[0014] Dividing the Bark domain into at least one frequency domain band, and calculating an energy value of each frequency domain band;

[0015] The characteristic value is extracted from the energy value.

[0016] In one embodiment, after the step of converting the target spectrum block into a target signal based on an inverse short-time Fourier transform, the method further includes:

[0017] Acquire the target signal, and determine the determined pitch period of the target signal by using a pitch detection algorithm;

[0018] Outputting a processed first signal based on the pitch period and a comb filter;

[0019] The coherence energy of the processed first signal is calculated, and when the coherence energy is higher than a preset threshold, the first signal is determined as the target signal.

[0020] In one embodiment, the method comprises:

[0021] Establishing the denoising model, wherein the denoising model consists of an input layer, a hidden layer and an output layer;

[0022] Obtaining a training set, wherein the training set includes input data and expected results;

[0023] The denoising model is trained using the training set, and parameters in the denoising model are updated according to a stochastic gradient descent algorithm.

[0024] In one embodiment, before the step of establishing the noise reduction model, the method further includes:

[0025] Obtain the collected voice data and convert the sound signal into a feature vector;

[0026] The feature vectors are labeled and associated with corresponding labels, and the feature vectors associated with the labels are determined as the training set.

[0027] In one embodiment, after the step of training the denoising model using the training set and updating the parameters in the denoising model according to the stochastic gradient descent algorithm, the method further includes:

[0028] A loss function is defined, and a difference between an output value of the denoising model and a true label is calculated according to the loss function;

[0029] Updating the model parameters of the denoising model based on the optimization algorithm, and determining the target loss function when the loss value of the loss function is a preset value;

[0030] Output the denoising model corresponding to the target loss function.

[0031] In one embodiment, after the step of outputting the denoising model corresponding to the target loss function, the method further includes:

[0032] Determine a test set, evaluate the trained denoising model according to the test set, and output the evaluation result;

[0033] An adjustment operation is performed on the denoising model according to the evaluation result, wherein the adjustment operation includes adjusting a network architecture, a learning rate, and a regularization parameter of the denoising model.

[0034] In one embodiment, before the step of determining the target loss function when the loss value of the loss function is a preset value, the step further includes:

[0035] According to the chain rule, the gradient of the network parameters corresponding to the loss function is calculated layer by layer;

[0036] The weight values ​​and bias values ​​in the network parameters are updated according to the gradient, and the loss value of the loss function is calculated according to the updated network parameters, so that the loss value output by the loss function is gradually reduced.

[0037] In addition, to achieve the above-mentioned purpose, the present application also provides a speech noise reduction device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the speech noise reduction method as described above.

[0038] In addition, to achieve the above-mentioned purpose, the present application also provides a storage medium, which is a computer-readable storage medium, and a computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, the steps of the speech noise reduction method as described above are implemented.

[0039] The present application provides a speech noise reduction method, a speech noise reduction device and a storage medium, which obtains a collected speech signal, decomposes the speech signal into spectrum blocks based on short-time Fourier transform, calculates the absolute value of the spectrum block, generates a power spectrum, thereby obtaining the noise spectrum of the noise, determines the gain value according to the noise spectrum and the power spectrum, and applies the gain value to the spectrum block to generate a target spectrum block. Then, the target spectrum block is converted into a target signal based on an inverse short-time Fourier transform, and the eigenvalue in the target signal is extracted according to the Bark band scale, and finally the eigenvalue and the target signal are input into a noise reduction model to output a target language signal. Thus, the speech signal is preprocessed by short-time Fourier transform, and then combined with the Bark band scale and the noise reduction model to eliminate non-stationary noise in the speech signal. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0042] Figure 1 This is a flow chart of the first embodiment of the speech noise reduction method of the present application;

[0043] Figure 2 This is a flow chart of a second embodiment of the speech noise reduction method of the present application;

[0044] Figure 3 This is a flow chart of a third embodiment of the speech noise reduction method of the present application;

[0045] Figure 4 This is a flow chart of a fourth embodiment of the speech noise reduction method of the present application;

[0046] Figure 5 This is a schematic diagram of the architecture of the hardware operating environment of the speech noise reduction device involved in the embodiment of the present application.

[0047] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0048] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] In order to better understand the above technical solution, exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0050] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0051] The main solution of the present application is: acquiring a collected speech signal; decomposing the speech signal into spectrum blocks based on short-time Fourier transform, and calculating the absolute value of the spectrum blocks to generate a power spectrum; acquiring a noise spectrum of a noise signal, determining a gain value according to the noise spectrum and the power spectrum, and applying the gain value to the spectrum block to generate a target spectrum block; converting the target spectrum block into a target signal based on an inverse short-time Fourier transform, and extracting eigenvalues ​​in the target signal according to the Bark band scale; inputting the eigenvalues ​​and the target signal into a denoising model, and outputting a target language signal.

[0052] At present, the noise in voice communication is generally divided into two types, namely stationary noise and non-stationary noise. The relevant voice communication noise reduction technology mainly targets stationary noise, such as constant air conditioning noise, etc. Generally, it is to first detect the frequency band where the noise occurs, and then design a linear filter or filter group to eliminate the noise. In actual processes, many noises are non-stationary noises, such as sudden impact sounds, keyboard tapping and table tapping sounds, etc. Since the frequency band of such noise is unstable, it is not easy to be detected and recorded, which makes the relevant technology unable to eliminate non-stationary noise in voice communication.

[0053] The present application obtains the collected speech signal, decomposes the speech signal into spectrum blocks based on short-time Fourier transform, calculates the absolute value of the spectrum blocks, generates a power spectrum, thereby obtaining the noise spectrum of the noise signal, determines the gain value according to the noise spectrum and the power spectrum, and applies the gain value to the spectrum block to generate a target spectrum block. Then, the target spectrum block is converted into a target signal based on an inverse short-time Fourier transform, and the eigenvalues ​​in the target signal are extracted according to the Bark band scale, and finally the eigenvalues ​​and the target signal are input into a noise reduction model to output a target language signal. Thus, the speech signal is preprocessed by short-time Fourier transform, and then combined with the Bark band scale and the noise reduction model to eliminate non-stationary noise in the speech signal.

[0054] Embodiment 1

[0055] Please refer to Figure 1In a first embodiment, the speech noise reduction method comprises the following steps:

[0056] Step S10: Acquire the collected voice signal.

[0057] In this embodiment, the voice noise reduction server in the conference system performs the processing action. When the conference system starts a conference room, each conference participant of the conference system will start voice communication. At this time, the noise reduction server will obtain the voice signal uploaded by each conference participant. The voice signal is pre-processed by the noise reduction server, and then the processed voice signal is used as the voice signal to be denoised.

[0058] Optionally, after the noise reduction server collects the initial language signal, the collected language signal, that is, the portion of the initial language signal with a frequency band of 125HZ-16000HZ, is used as the voice signal to be denoised. In the actual interception process, the sampling rate of the acquisition device for collecting the voice signal is first adjusted so that the acquisition device captures the highest frequency component of 16000Hz. According to the Nyquist sampling theorem, the sampling rate is twice the highest frequency that needs to be captured, so a sampling rate of 32kHz is required. Then, through the fast Fourier transform, each frame signal of the collected voice signal is converted from the time domain to the frequency domain to obtain a spectrum representation. A digital bandpass filter is designed with a passband range of 125Hz to 16000Hz. The spectrum is passed through the designed bandpass filter to retain the frequency components within the target frequency band and filter out other frequency components. For the spectrum filtered by the digital bandpass filter, an inverse fast Fourier transform is applied to convert the spectrum signal back to the time domain, thereby determining the voice signal.

[0059] Step S20: Decomposing the speech signal into spectrum blocks based on short-time Fourier transform, and calculating the absolute value of the spectrum blocks to generate a power spectrum.

[0060] In this embodiment, short-time Fourier transform is a technique used in the field of signal processing to analyze signals whose spectrum varies with time. Through short-time Fourier transform, the collected speech signal can be divided into multiple spectrum blocks, and then the absolute value of each spectrum block is calculated to obtain the power spectrum.

[0061] Specifically, the method of decomposing the speech signal into spectrum blocks based on short-time Fourier transform is to first select a suitable window function, such as a Hamming window, a Heining window or a rectangular window. The length of the window function determines the time width of each spectrum block. The speech signal is divided into multiple overlapping segments, and the length of each segment is equal to the length of the window function. There is a certain overlap between adjacent segments, and the overlapping length is usually half of the window length. Each signal segment is multiplied by the window function to reduce the edge effect. Fourier transform is performed on the signal segment after each window to obtain the spectrum block of the segment signal. The spectrum blocks of all signal segments are combined into a two-dimensional matrix, where the horizontal axis represents time, the vertical axis represents frequency, and the values ​​in the matrix represent the signal amplitude at different times and frequencies. Finally, the spectrum block is represented as Y(i,k), where i represents the time frame index and k represents the frequency index.

[0062] Step S30: Acquire a noise spectrum of the noise signal, determine a gain value according to the noise spectrum and the power spectrum, and apply the gain value to the spectrum block to generate a target spectrum block.

[0063] In this embodiment, the noise signal is pre-stored, and the noise signal is recorded by using the corresponding sensor and selecting a suitable sampling rate to ensure that all information within the frequency range can be captured. The noise signal is converted into a noise spectrum according to the short-time Fourier transform in step S20, and the gain value is calculated according to the noise spectrum and the power spectrum, wherein the gain value refers to the degree of amplification or attenuation of the signal. That is, the proportional factor of the amplification or attenuation of the signal at a specific time frame and frequency. Then each spectrum block is multiplied by the gain value to obtain multiple target spectrum blocks, and each frequency point in the spectrum block is weakened according to the gain value, thereby achieving the effect of noise reduction.

[0064] Step S40: converting the target spectrum block into a target signal based on an inverse short-time Fourier transform, and extracting characteristic values ​​in the target signal according to a Bark frequency band scale.

[0065] In this embodiment, after applying the gain value to each spectrum block to obtain the target spectrum block, the target spectrum block is converted into a time domain signal based on the inverse short-time Fourier transform to generate a target signal. Then, a processing action is performed on the target signal based on the Bark band scale, and a feature value in the target signal is extracted, wherein the Bark band scale is a psychoacoustic scale based on the way the human auditory system perceives sound.

[0066] Optionally, based on the inverse short-time Fourier transform method of converting the target spectrum block into a target signal, an inverse Fourier transform is performed on the spectrum data of each time window to obtain an estimated value of the time domain signal. Since a sliding window is used in the short-time Fourier transform, adjacent time domain signal segments will have overlapping parts. These overlapping parts need to be combined by overlapping and adding to ensure the continuity and integrity of the signal. In the short-time Fourier transform, the original signal is multiplied by the window function, and in the inverse short-time Fourier transform, the reconstructed signal also needs to consider the influence of the window function. Usually the same window function is used, and the signal is appropriately adjusted before overlapping and adding to eliminate the distortion caused by the window function. The reconstructed time domain signal is obtained, that is, the target signal.

[0067] Optionally, in this embodiment, the step of extracting the characteristic value and the signal to be processed in the target signal according to the Bark frequency band scale includes:

[0068] Acquire the target signal; convert the frequency axis of the target signal into Bark units based on a frequency band scale conversion formula and map it to the Bark domain; divide the Bark domain into at least one frequency domain band, and calculate the energy value of each frequency domain band; extract the eigenvalue from the energy value.

[0069] Specifically, the power spectrum is calculated from the results obtained in the above steps. The power spectrum is mapped to the Bark scale, that is, the frequency axis is converted from Hertz (Hz) to Bark units. The conversion formula of the Bark scale can be expressed as:

[0070]

[0071] Where f is the frequency (Hz) and Bark is the corresponding Bark value.

[0072] Then, the spectrum is divided into 24 bands according to the Bark scale. Each band corresponds to a specific range of Bark values, where these bands are usually not of equal width. The Bark scale reflects the difference in sensitivity of the human auditory system to different frequencies. The energy of each Bark band is calculated, that is, it is obtained by multiplying the spectrum with the filter corresponding to the Bark band and then summing them. Eigenvalues ​​are extracted from the energy of each Bark band, which include energy spectrum, energy ratio and energy center frequency, where the energy spectrum refers to the energy of each Bark band. The energy ratio is the energy ratio between Bark bands. The energy center frequency is the frequency position of the weighted average of the energy of all Bark bands. In addition, a set of filters needs to be designed, each filter corresponding to a Bark band.

[0073] Optionally, in this embodiment, after the step of converting the target spectrum block into a target signal based on inverse short-time Fourier transform, the step further includes:

[0074] Acquire the target signal, and determine the fundamental pitch period of the target signal through a fundamental pitch detection algorithm; output a processed first signal based on the fundamental pitch period and a comb filter; calculate the coherence energy of the processed first signal, and when the coherence energy is higher than a preset threshold, determine the first signal as the target signal.

[0075] Specifically, the pitch period of the target signal is detected by a pitch detection algorithm, and the pitch period can be determined by an autocorrelation method and a frequency domain method. Then, the pitch period is processed based on a comb filter and a first signal is output, and the formula is as follows:

[0076] (z)= ;

[0077] in, is the filter coefficient, M is the number of periods on both sides of the center tap, T is the fundamental frequency period, and k represents the frequency point. After the comb filter outputs the first signal, the coherence function is obtained, and the coherence energy of the first signal is calculated based on the coherence function. The coherence energy reflects the intensity distribution of the signal at different frequencies. Finally, it is determined whether the coherence energy is higher than the preset threshold. The preset threshold is the coherence energy of clean speech. If it is higher, the first signal is determined as the target signal. Otherwise, the M value is adjusted, and then the target gain is adjusted, and the coherence energy is continued to be calculated until it is greater than the preset threshold. It can effectively improve the harmonic characteristics of the speech, reduce the inter-harmonic noise, and thus improve the noise reduction effect.

[0078] In addition, in this embodiment, the signal is first processed using the MMSE noise reduction method, and the specific process may be that the derivation process of the gain value G is as follows:

[0079] y(n)=x(n)+d(n);

[0080] In the above formula, x(n) represents the clean speech signal, d(n) represents the noise signal, and y(n) represents the signal collected by the microphone. After performing short-time Fourier transform on the above formula, the spectrum is generated and the following formula is obtained:

[0081] Y(i,k)=X(i,k)+D(i,k);

[0082] Among them, Y(i,k), X(i,k) and D(i,k) represent the frequency domain signals of y(n), x(n), d(n) respectively, i represents the i-th time domain frame, k represents the frequency point, Y(i,k) represents the spectrum block, and the power spectrum |Y(i,k)| is generated by calculating the absolute value of the spectrum block, thereby obtaining the following formula:

[0083] G(i,k)=|S(i,k)| / |Y(i,k)|;

[0084] G(i,k) represents the gain value, which is determined by the noise spectrum and the power spectrum. Therefore, as long as G(i,k) is obtained, the target spectrum block S(i,k) can be determined by the noisy signal Y(i,k), and then the target spectrum block can be converted into the target signal by inverse short-time Fourier transform.

[0085] Step S50: input the feature value and the target signal into a noise reduction model, and output a target language signal.

[0086] In this embodiment, the preprocessed target signal is input into the noise reduction model, and the noise reduction model divides the long speech signal into short segments and extracts features from each speech signal. The method of extracting speech features can be Mel frequency cepstral coefficients (MFCC), spectrograms and Mel spectrograms, and the speech features can capture the main spectral information of the speech signal. Among them, the Mel frequency cepstral coefficient is an audio feature based on the auditory characteristics of the human ear. The Mel frequency cepstral coefficient can simulate the different sensitivities of the human ear to sounds of different frequencies, especially the sensitivity to low-frequency sounds, and the spectrum of the audio signal is nonlinearly transformed through a group of Mel filter groups to obtain a Mel spectrum. Then the Mel spectrum is logarithmized and discrete cosine transformed to obtain a group of cepstral coefficients, which are used to characterize the sound quality, pitch and volume of the audio signal.

[0087] In addition, the spectrogram is a two-dimensional image representation used to show the distribution of audio signals in the time domain and frequency domain. It obtains a spectrum sequence by performing a short-time Fourier transform (STFT) on the audio signal, and then arranges these spectrum sequences into an image in time order. The spectrogram can intuitively show how the frequency of the audio signal changes over time. The Mel Spectrogram is an improved form of the spectrogram, which combines the advantages of the human hearing characteristics and the spectrogram. Similar to the spectrogram, the Mel Spectrogram also obtains a spectrum sequence through a short-time Fourier transform, but then uses a Mel filter bank to perform a nonlinear transformation on the spectrum to obtain a Mel spectrum. Then, the Mel spectrum is arranged into an image in time order to obtain a Mel Spectrogram. Since the Mel filter bank simulates the hearing characteristics of the human ear, the Mel Spectrogram can better describe the characteristics of the audio signal such as sound quality, pitch, and volume.

[0088] Specifically, the denoising model processes the extracted speech features through a series of hidden layers, and learns how to separate the representation of pure speech from noisy features. Then the output layer of the denoising model generates a first spectrum feature close to a clean speech signal and a second spectrum feature of a noisy speech signal. Then the first spectrum feature of the clean speech signal output by the denoising model is inversely transformed back to a time domain signal, and the inversely transformed signal fragments are spliced ​​together to form a complete clean speech signal output, namely the target speech signal.

[0089] In the technical solution provided in this embodiment, the collected speech signal is obtained, and then the speech signal is decomposed into spectrum blocks based on short-time Fourier transform, and the absolute value of the spectrum block is calculated to generate a power spectrum, thereby obtaining the noise spectrum of the noise signal, determining the gain value according to the noise spectrum and the power spectrum, and applying the gain value to the spectrum block to generate a target spectrum block. Then, the target spectrum block is converted into a target signal based on an inverse short-time Fourier transform, and the eigenvalue in the target signal is extracted according to the Bark band scale, and finally the eigenvalue and the target signal are input into a noise reduction model to output a target language signal. Thus, the speech signal is preprocessed by short-time Fourier transform, and then combined with the Bark band scale and the noise reduction model to eliminate non-stationary noise in the speech signal.

[0090] Embodiment 2

[0091] Please refer to Figure 2 In a second embodiment, the method comprises:

[0092] Step S60: Establishing the denoising model, wherein the denoising model consists of an input layer, a hidden layer and an output layer.

[0093] In this embodiment, a deep neural network architecture is selected as the architecture of the denoising model, and the denoising model is constructed. The denoising model consists of an input layer, a hidden layer, and an output layer. The layers are fully connected, with a total of 5 layers, including 1 input layer, 3 hidden layers, and 1 output layer. The number of nodes in the input layer is 512, the number of nodes in the middle 3 hidden layers is 1024, and the number of nodes in the output layer is also 512.

[0094] Specifically, the DNN network establishment process and mathematical relationship are as follows:

[0095] Layer1: ;

[0096]

[0097] Layer2: ;

[0098]

[0099] because ;so:

[0100] Layer i: ;

[0101]

[0102] in is the Tanh function, is the response of layer i-1, is the neuron weight from the i-1th layer to the ith layer, It is the bias of neurons from the i-1th layer to the i-th layer. Layer is used to represent the layer in the model.

[0103] Step S70: Obtain a training set, wherein the training set includes input data and expected results.

[0104] In this embodiment, the training set is used to train the established denoising model, which includes an input data set and an expected result set. The input data set is used to train the denoising model, the denoising model will output a result, and the expected result set is used to compare with the output result of the denoising model.

[0105] Optionally, in this embodiment, before the step of establishing the noise reduction model, the step further includes:

[0106] The collected voice data is obtained, and the sound signal is converted into a feature vector; the feature vector is labeled and associated with a corresponding label, and the feature vector associated with the label is determined as the training set.

[0107] Specifically, obtain a large amount of labeled speech data, which can be sentences, words or keywords required for speech recognition, or personal speech samples for voiceprint recognition. Convert the original audio signal into a form suitable for model input, such as extracting Mel frequency cepstral coefficients and other acoustic features. Segment the audio into frames and align them with corresponding labels (such as phoneme labels, sentence labels, etc.), often achieved through methods such as Viterbi alignment, and label the feature vectors for supervised learning.

[0108] Step S80: training the denoising model using the training set, and updating the parameters in the denoising model according to a stochastic gradient descent algorithm.

[0109] In this embodiment, the input data set in the training set is input into the denoising model, and the output result of the denoising model is recorded. The parameters are updated using the stochastic gradient descent algorithm. There is a certain difference between the model output and the real output. We set this difference as , the model output plus the error is equal to the true label, recorded as:

[0110] ;

[0111] Among them, Z represents the output result of the model, X represents the input data input into the model, and W and b are the parameters in the model.

[0112] The stochastic gradient algorithm is:

[0113] ;

[0114] in is the step size (learning rate) of stochastic gradient descent.

[0115] In the technical solution provided in this embodiment, the denoising model is established, wherein the denoising model is composed of an input layer, a hidden layer and an output layer, and then a training set is obtained, wherein the training set includes input data and expected results. Finally, the denoising model is trained by the training set, and the parameters in the denoising model are updated according to the stochastic gradient descent algorithm. By establishing the model and updating the parameters using the stochastic gradient descent algorithm, the accuracy of the denoising model training is improved.

[0116] Embodiment 3

[0117] Please refer to Figure 3 In the fourth embodiment, after step S60, the following steps are further included:

[0118] Step S90: define a loss function, and calculate the difference between the output value of the denoising model and the true label according to the loss function.

[0119] Step S100: updating the model parameters of the denoising model based on the optimization algorithm, and determining the target loss function when the loss value of the loss function is a preset value.

[0120] Step S110: output the denoising model corresponding to the target loss function.

[0121] In this embodiment, the loss function is first defined, that is, a suitable loss function is selected, and then the training data is sent to each layer of the denoising model in batches, and forward calculation is performed layer by layer until the output layer, and then the current network output is compared with the true label, and the loss is calculated using the loss function. Common loss functions include mean square error loss (MSE), cross entropy loss, etc. Finally, the errors are added up each time. In order to prevent the positive and negative errors from canceling out, the values ​​are taken, so that a loss function can be obtained, as follows:

[0122] ;

[0123] The difference between the network output and the true label, i.e., the loss value, is calculated using the loss function, and the model parameters are updated using Adam (Adaptive Moment Estimation) or other optimization algorithms to minimize the loss function. Finally, when the calculated loss value is a preset value, the target loss function corresponding to the loss value is obtained, the corresponding model parameters are determined, and the denoising model corresponding to the model parameters is output, wherein the preset value is a relatively small loss value, which is used to characterize that the deviation of the denoising model is getting smaller and smaller.

[0124] Optionally, in this embodiment, before the step of determining the target loss function when the loss value of the loss function is a preset value, the step further includes:

[0125] According to the chain rule, the gradient of the network parameters corresponding to the loss function is calculated layer by layer; the weight values ​​and bias values ​​in the network parameters are updated according to the gradient, and the loss value of the loss function is calculated according to the updated network parameters, so that the loss value output by the loss function is gradually reduced.

[0126] Specifically, first calculate the gradient of the loss function with respect to the output, and then use the chain rule to propagate the gradient of the output layer backward to the hidden layer. Specifically, multiply the gradient of the output layer by the weight from the hidden layer to the output layer. According to the gradient of the hidden layer, calculate the gradient of all parameters in the network, including weights and bias terms. For weights, the gradient is calculated by multiplying the gradient of the hidden layer by the output of the previous layer. Then use the calculated parameter gradient to update the network parameters using optimization algorithms such as gradient descent. The calculation formula for the new parameters is: new parameters = old parameters - learning rate × parameter gradient. Among them, the learning rate is a hyperparameter used to control the step size of each update.

[0127] Furthermore, in this embodiment, after step S90, the following steps are further included:

[0128] The trained denoising model is obtained, and the denoising model is deployed in an embedded system.

[0129] Specifically, after the noise reduction model training is completed, the trained model is exported from the training platform to a format suitable for deployment. Common formats include TensorFlow's .pb file, TFLite, ONNX or PyTorch's .pt / .pth file. In order to run efficiently on embedded devices with limited resources, the model is quantized, the floating point weights are converted to integers, and unimportant weights are removed. Other optimization techniques can also be used to reduce the model size and improve operating efficiency. Then install the necessary operating environment on the embedded system, including a cross-compilation tool chain, an inference engine library, etc. The exported model file is then deployed to the embedded device, and the model file path is ensured to be correctly configured. Finally, use the API or library supported on the embedded device to write code to load the model, and implement preprocessing, model reasoning, and post-processing of audio data, that is, to achieve noise reduction processing of speech signals.

[0130] In addition, after deployment, the performance of the model should be tested on the embedded device, including inference speed, memory usage, and power consumption. To verify the noise reduction effect of the model in the actual application scenario, it may be necessary to compare it with the original unprocessed audio. Based on the test results, it may be necessary to return to the model optimization step and adjust the model or deployment strategy to achieve better performance or effect. After the product is released, continue to monitor the model operation status and user experience, and prepare to make necessary updates and maintenance to the model or system.

[0131] In the technical solution provided in this embodiment, a loss function is defined, the difference between the output value of the denoising model and the true label is calculated according to the loss function, and then the model parameters of the denoising model are updated based on the optimization algorithm, and the target loss function is determined when the loss value of the loss function is a preset value, and finally the denoising model corresponding to the target loss function is output, thereby completing the training of the denoising model.

[0132] Embodiment 4

[0133] Please refer to Figure 4 In the fourth embodiment, after step S90, the following steps are further included:

[0134] Step S120: determine a test set, evaluate the trained denoising model according to the test set, and output the evaluation result.

[0135] Step S130: performing an adjustment operation on the denoising model according to the evaluation result, wherein the adjustment operation includes adjusting a network architecture, a learning rate, and a regularization parameter of the denoising model.

[0136] In this embodiment, a training set in a data set is obtained, and the trained noise reduction model is deployed in a test environment. For each sample in the test set, the noise reduction model is used for noise reduction processing, and then evaluation indicators such as signal-to-noise ratio (SNR) improvement, peak signal-to-noise ratio (PSNR), mean square error (MSE), and possible subjective listening scores are calculated. The evaluation indicators of all samples are counted to obtain the overall performance of the model on the test set, wherein the evaluation result can be a score for each evaluation indicator. Then, the defects of the model are identified according to the evaluation results, such as whether it performs poorly in certain types of noise or certain frequency ranges, and the direction and degree of adjustment of the model are determined. The network architecture of the noise reduction model is adjusted again, including increasing / decreasing the number of layers, changing the number of neurons in each layer, and introducing a new network structure. And adjust the learning rate and regularization parameters, use learning rate decay and adaptive learning rate algorithms to accelerate convergence to improve model performance, or use weight decay (L2 regularization) and dropout ratios to reduce overfitting.

[0137] In addition, after the noise reduction model is tested, the performance of the adjusted model is verified using the validation set, and the model is further adjusted based on the feedback from the test set until the performance requirements are met or the performance improvement bottleneck is reached.

[0138] In the technical solution provided in this embodiment, a test set is determined, the trained denoising model is evaluated according to the test set, and the evaluation result is output, and then the denoising model is adjusted according to the evaluation result, wherein the adjustment operation includes adjusting the network architecture, learning rate and regularization parameter of the denoising model, so that the trained denoising model can be tested and optimized to improve the performance of the denoising model.

[0139] Since the system introduced in the embodiment of the present application is a system used to implement the method of the embodiment of the present application, based on the method introduced in the embodiment of the present application, the person skilled in the art can understand the specific structure and deformation of the system, so it is not repeated here. All systems used in the method of the embodiment of the present application belong to the scope of protection of this application.

[0140] The present application provides a speech noise reduction device, which includes: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the speech noise reduction method in the above-mentioned embodiment 1.

[0141] Reference below Figure 5, which shows a schematic diagram of the structure of a speech noise reduction device suitable for implementing the embodiment of the present application. The speech noise reduction device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The speech noise reduction device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0142] like Figure 5 As shown, the speech noise reduction device may include a processing device 1001 (e.g., a core processor, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the speech noise reduction device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the speech noise reduction device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a speech noise reduction device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0143] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0144] The speech noise reduction device provided by the present application adopts the speech noise reduction method in the above embodiment, which can solve the technical problem that the related technology cannot eliminate non-stationary noise in voice communication. Compared with the prior art, the beneficial effects of the speech noise reduction device provided by the present application are the same as the beneficial effects of the speech noise reduction method provided by the above embodiment, and the other technical features in the speech noise reduction device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0145] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0146] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0147] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the speech noise reduction method in the above-mentioned embodiment.

[0148] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0149] The computer-readable storage medium may be included in the speech noise reduction device; or may exist independently without being assembled into the speech noise reduction device.

[0150] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the speech noise reduction device, the speech noise reduction device: obtains a collected speech signal; decomposes the speech signal into spectrum blocks based on short-time Fourier transform, and calculates the absolute value of the spectrum blocks to generate a power spectrum; obtains a noise spectrum of the noise signal, determines a gain value according to the noise spectrum and the power spectrum, and applies the gain value to the spectrum block to generate a target spectrum block; converts the target spectrum block into a target signal based on an inverse short-time Fourier transform, and extracts a characteristic value in the target signal according to the Bark band scale; inputs the characteristic value and the target signal into a noise reduction model, and outputs a target language signal.

[0151] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0152] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0153] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0154] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned speech noise reduction method, and can solve the technical problem that the related art cannot eliminate non-stationary noise in speech communication. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the speech noise reduction method provided in the above-mentioned embodiment, and will not be repeated here.

[0155] An embodiment of the present application provides a computer program product, including a computer program, which implements the steps of the above-mentioned speech noise reduction method when executed by a processor.

[0156] The computer program product provided by the present application can solve the technical problem that the related art cannot eliminate non-stationary noise in voice communication. Compared with the prior art, the beneficial effects of the computer program product provided by the embodiment of the present application are the same as the beneficial effects of the voice noise reduction method provided by the above embodiment, which will not be repeated here.

[0157] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.

Claims

1. A speech noise reduction method, characterized in that: The method comprises the following steps: Acquire the collected voice signal; Decomposing the speech signal into spectrum blocks based on short-time Fourier transform, and calculating the absolute value of the spectrum blocks to generate a power spectrum; Acquire a noise spectrum of a noise signal, determine a gain value according to the noise spectrum and the power spectrum, and apply the gain value to the spectrum block to generate a target spectrum block, wherein the noise signal is pre-stored, and the noise signal is recorded by using a corresponding sensor and selecting a suitable sampling rate to ensure that all information within a frequency range can be captured; Convert the target spectrum block into a target signal based on inverse short-time Fourier transform, and obtain the target signal; Convert the frequency axis of the target signal into Bark units based on a frequency band scale conversion formula, and map it to the Bark domain; Dividing the Bark domain into at least one frequency domain band, and calculating an energy value of each frequency domain band; extracting a characteristic value from the energy value; The eigenvalues ​​and the target signal are input into a denoising model, and a target speech signal is output. The denoising model processes the extracted eigenvalues ​​through a series of hidden layers, and learns how to separate the representation of pure speech from noisy features. Then, the output layer of the denoising model generates a first spectral feature close to a clean speech signal and a second spectral feature of a noisy speech signal. The first spectral feature of the clean speech signal output by the denoising model is then inversely transformed back into a time domain signal, and the inversely transformed signal fragments are spliced ​​together to form a complete clean speech signal output, namely, the target speech signal.

2. The method according to claim 1, characterized in that After the step of converting the target spectrum block into a target signal based on inverse short-time Fourier transform, the method further includes: Acquire the target signal, and determine the determined pitch period of the target signal by using a pitch detection algorithm; Outputting a processed first signal based on the pitch period and a comb filter; The coherence energy of the processed first signal is calculated, and when the coherence energy is higher than a preset threshold, the first signal is determined as the target signal.

3. The method according to claim 1, characterized in that The method comprises: Establishing the denoising model, wherein the denoising model consists of an input layer, a hidden layer and an output layer; Obtaining a training set, wherein the training set includes input data and expected results; The denoising model is trained using the training set, and parameters in the denoising model are updated according to a stochastic gradient descent algorithm.

4. The method according to claim 3, characterized in that Before the step of establishing the noise reduction model, the method further includes: Obtain the collected voice data and convert the sound signal into a feature vector; The feature vectors are labeled and associated with corresponding labels, and the feature vectors associated with the labels are determined as the training set.

5. The method according to claim 3, characterized in that After the step of training the denoising model by using the training set and updating the parameters in the denoising model according to the stochastic gradient descent algorithm, the method further includes: A loss function is defined, and a difference between an output value of the denoising model and a true label is calculated according to the loss function; Updating the model parameters of the denoising model based on the optimization algorithm, and determining the target loss function when the loss value of the loss function is a preset value; Output the denoising model corresponding to the target loss function.

6. The method according to claim 5, characterized in that After the step of outputting the denoising model corresponding to the target loss function, the method further includes: Determine a test set, evaluate the trained denoising model according to the test set, and output the evaluation result; An adjustment operation is performed on the denoising model according to the evaluation result, wherein the adjustment operation includes adjusting a network architecture, a learning rate, and a regularization parameter of the denoising model.

7. The method according to claim 5, characterized in that Before the step of determining the target loss function when the loss value of the loss function is a preset value, the method further includes: According to the chain rule, the gradient of the network parameters corresponding to the loss function is calculated layer by layer; The weight values ​​and bias values ​​in the network parameters are updated according to the gradient, and the loss value of the loss function is calculated according to the updated network parameters, so that the loss value output by the loss function is gradually reduced.

8. A speech noise reduction device, characterized in that: The speech noise reduction device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the speech noise reduction method according to any one of claims 1 to 7.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the speech noise reduction method according to any one of claims 1 to 7 are implemented.

Images