Denoising model training method, device and storage medium

Abstract

The present application provides a noise reduction model training method, device and storage medium, which relates to the field of speech signal processing technology. The method includes: generating at least one random number, determining noise information based on the random number, performing noise addition operation on quiet data according to the noise addition information, and obtaining noise addition data, wherein the noise addition information includes: target noise data, a combined time period of the target noise data and the quiet data, and a signal-to-noise ratio synthesized from the target noise data and the quiet data; extracting acoustic features from the noise addition data, and training a noise reduction model based on the acoustic features to obtain a target noise reduction model. The method can solve the problem of low accuracy in speech recognition of noisy speech.

Description

Technical Field

[0001] The present application relates to the field of speech signal processing, and more specifically, to a noise reduction model training method, device and storage medium. Background Art

[0002] In the process of receiving voice signals, there are various noises such as the surrounding environment, transmission media, electromagnetic interference, etc., which make the received voice no longer pure original voice, and will cause a significant decline in performance when applied to voice signals.

[0003] The main purpose of speech noise reduction is to extract the purest original speech from noisy speech, perform noise reduction on noisy speech, and thus improve the accuracy of speech recognition. However, most of the current noise reduction technologies directly process speech signals, resulting in low accuracy in speech recognition of noisy speech. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a noise reduction model training method, device and storage medium to solve the current problem of low accuracy in speech recognition of noisy speech.

[0005] In a first aspect, an embodiment of the present application provides a denoising model training method, comprising:

[0006] Generate at least one random number, determine noise information based on the random number, perform noise addition operation on quiet data according to the noise addition information to obtain noisy data, wherein the noise addition information includes: target noise data, a combined time period of the target noise data and the quiet data, and a signal-to-noise ratio of the target noise data and the quiet data;

[0007] Acoustic features are extracted from the noisy data, and a noise reduction model is trained based on the acoustic features to obtain a target noise reduction model.

[0008] Wherein, determining the noise information based on the random number includes one or more of the following:

[0009] Determine, based on a first random number among the random numbers, to select the noise data corresponding to the first random number from the preset noise library, and use the noise data as target noise data;

[0010] determining a merging time period of the target noise data and the quiet data based on a second random number among the random numbers;

[0011] A signal-to-noise ratio of a combination of the target noise data and the quiet data is determined based on a third random number among the random numbers.

[0012] In the above implementation process, noise data can be randomly selected from the noise library, the merging time period of noise data and quiet data and the signal-to-noise ratio can be determined, so that the generated noisy data can more closely simulate the noise in reality. This can enable the trained target noise reduction model to have higher generalization ability and higher accuracy in denoising and speech recognition of real data to be processed.

[0013] Optionally, the training of the noise reduction model based on the acoustic features includes:

[0014] For a piece of the noisy data, input the noisy data into the denoising model and the trained speech recognition model respectively, obtain a first loss value of the noisy data in the denoising model according to a first loss function, and obtain a second loss value in the speech recognition model according to a second loss function;

[0015] Determine whether the combined loss value of the first loss value and the second loss value exceeds a preset loss threshold based on an interpolation and merging method;

[0016] When the combined loss value exceeds a preset loss threshold, removing the noisy data;

[0017] When the combined loss value does not exceed the preset loss threshold, it is determined to use the first loss value to train the denoising model.

[0018] In the above implementation process, the noisy data is input into the denoising model and the trained speech recognition model respectively, and a pre-trained speech recognition model is used to guide the training of the denoising model based on the interpolation merging method, which can prevent the interference of long-tail data with too low signal-to-noise ratio generated by the noise, thereby improving the speech recognition accuracy of the denoising model.

[0019] Optionally, the determining, based on a first random number among the random numbers, selecting the noise data corresponding to the first random number from a preset noise library, and taking the noise data as target noise data may include:

[0020] Based on a mixed congruential algorithm, the first random number is generated, where the first random number is one of all random numbers uniformly distributed in a first preset interval, and each number in the first preset interval corresponds to a piece of the noise data in the noise library;

[0021] The noise data corresponding to the first random number is selected from the noise library as target noise data.

[0022] Optionally, determining the merging time period of the target noise data and the quiet data based on the second random number among the random numbers may include:

[0023] Generate the second random number based on a mixed congruential algorithm, where the second random number is a random number among all random numbers uniformly distributed in a second preset interval, where the second preset interval is (0, In-Is), where In is the duration of the target noise data, and Is is the duration of the quiet data;

[0024] The value of the second random number is used as the duration of the merging time period of the target noise data and the quiet data, thereby determining the merging time period of the target noise data and the quiet data.

[0025] Optionally, determining a signal-to-noise ratio of a combination of the target noise data and the quiet data based on a third random number among the random numbers may include:

[0026] Based on the formula Generate the third random number, the third random number is in accordance with the normal distribution A random number among all random numbers, μ is the mean, σ is the standard deviation;

[0027] The third random number is used as a ratio of a signal-to-noise ratio of the target noise data and the quiet data.

[0028] In the above implementation process, noise data can be randomly selected from the noise library, the merging time period of noise data and quiet data and the signal-to-noise ratio can be determined, so that the generated noise-added data can more closely simulate the noise in reality, thereby enabling the trained target noise reduction model to have a higher generalization ability and a higher accuracy in noise reduction and speech recognition of the real data to be processed. Optionally, the extraction of acoustic features from the noise-added data may include:

[0029] adjusting the frequency spectrum of the noisy data based on a filter;

[0030] Dividing the noise-added data in frames;

[0031] Adding a window to each frame of the noise-added data based on a window function;

[0032] Performing a fast Fourier transform on the windowed noisy data to obtain a frequency domain signal of the noisy data;

[0033] The frequency domain signal is filtered based on a Mel filter to obtain the acoustic feature.

[0034] In the above implementation process, adjusting the spectrum of the noisy data can make the spectrum of the noisy data signal flat and not easily affected by the finite word length effect. Using a Hamming window to window a frame of noisy signal can reduce the influence of the Gibbs effect. The frequency domain signal is filtered based on the Mel filter to simulate the masking effect of the human ear. Based on the above characteristics, the accuracy of acoustic feature extraction is improved.

[0035] Optionally, after training the noise reduction model based on the acoustic features to obtain a target noise reduction model, the method may further include:

[0036] extracting actual acoustic features from the data to be processed;

[0037] Inputting the actual acoustic features into the target noise reduction model to obtain the acoustic features after noise reduction;

[0038] Speech recognition is performed on the data to be processed based on the acoustic features after noise reduction.

[0039] In the above implementation process, the target noise reduction model trained by the embodiment of the present application is used to perform noise reduction and recognition on the actual noise data, which can achieve higher accuracy when performing noise reduction and speech recognition on the actual data to be processed. By directly using the filterbank features as the input and output of the model, it is possible to avoid the need for re-conversion in the speech recognition application, thereby improving the efficiency of model recognition.

[0040] In a second aspect, an embodiment of the present application provides a denoising model training device, comprising:

[0041] a noise adding module, configured to generate at least one random number, determine noise adding information based on the random number, perform noise adding operation on quiet data according to the noise adding information, and obtain noisy data, wherein the noise adding information includes: target noise data, a merging time period of the target noise data and the quiet data, and a signal-to-noise ratio of the target noise data and the quiet data;

[0042] The training module is used to extract acoustic features from the noisy data and train the noise reduction model based on the acoustic features to obtain a target noise reduction model.

[0043] In the above implementation process, the noise reduction model training method provided in the embodiment of the present application can randomly select noise data from the noise library, determine the merging time period of noise data and quiet data, and the signal-to-noise ratio, so that the generated noisy data can more closely simulate the noise in reality, thereby enabling the trained target noise reduction model to have higher generalization ability and higher accuracy when performing noise reduction and speech recognition on real data to be processed.

[0044] Optionally, the training module may be specifically configured to perform one or more of the following steps:

[0045] Determine, based on a first random number among the random numbers, to select the noise data corresponding to the first random number from the preset noise library, and use the noise data as target noise data;

[0046] determining a merging time period of the target noise data and the quiet data based on a second random number among the random numbers;

[0047] A signal-to-noise ratio of a combination of the target noise data and the quiet data is determined based on a third random number among the random numbers.

[0048] Optionally, the training module may be specifically used for:

[0049] The method comprises the following steps: adjusting the frequency spectrum of the noisy data based on a filter; dividing the noisy data in frames; windowing each frame of the noisy data based on a window function; performing a fast Fourier transform on the windowed noisy data to obtain a frequency domain signal of the noisy data; and filtering the frequency domain signal based on a Mel filter to obtain the acoustic feature.

[0050] In the above implementation process, adjusting the spectrum of the noisy data can make the spectrum of the noisy data signal flat and not easily affected by the finite word length effect. Using a Hamming window to window a frame of noisy signal can reduce the influence of the Gibbs effect. The frequency domain signal is filtered based on the Mel filter to simulate the masking effect of the human ear. Based on the above characteristics, the accuracy of acoustic feature extraction is improved.

[0051] Optionally, the training module can also be used to:

[0052] For a piece of the noisy data, the noisy data is respectively input into the denoising model and the trained speech recognition model, a first loss value of the noisy data is obtained in the denoising model according to a first loss function, and a second loss value is obtained in the speech recognition model according to a second loss function; based on the interpolation merging method, it is determined whether the combined loss value of the first loss value and the second loss value exceeds a preset loss threshold; when the combined loss value exceeds the preset loss threshold, the noisy data is discarded; when the combined loss value does not exceed the preset loss threshold, it is determined to use the first loss value to train the denoising model.

[0053] In the above implementation process, the noisy data is input into the denoising model and the trained speech recognition model respectively, and a pre-trained speech recognition model is used to guide the training of the denoising model based on the interpolation merging method, which can prevent the interference of long-tail data with too low signal-to-noise ratio generated by the noise, thereby improving the speech recognition accuracy of the denoising model.

[0054] Optionally, the noise adding module may be specifically used for:

[0055] Based on a mixed congruential algorithm, the first random number is generated, the first random number is a random number among all random numbers uniformly distributed in a first preset interval, each number in the first preset interval corresponds to a piece of noise data in the noise library; and the noise data corresponding to the first random number is selected from the noise library as the target noise data.

[0056] Optionally, the noise adding module can also be used to:

[0057] Generate the second random number based on a mixed congruential algorithm, where the second random number is a random number among all random numbers uniformly distributed in a second preset interval, where the second preset interval is (0, In-Is), where In is the duration of the target noise data, and Is is the duration of the quiet data;

[0058] The value of the second random number is used as the duration of the merging time period of the target noise data and the quiet data, thereby determining the merging time period of the target noise data and the quiet data.

[0059] Optionally, the noise adding module can also be used to:

[0060] Based on the formula Generate the third random number, the third random number is in accordance with the normal distribution A random number among all random numbers, μ is the mean, σ is the standard deviation;

[0061] The third random number is used as a ratio of a signal-to-noise ratio of the target noise data and the quiet data.

[0062] In the above implementation process, the quiet data is denoised by randomly selecting noise data, merging time periods and signal-to-noise ratios, so that the generated noisy data has the continuity of the noise data in reality and is more in line with the noise in reality, thereby more effectively training the denoising model and improving the accuracy of the denoising model.

[0063] Optionally, the denoising model training device may further include:

[0064] The speech recognition module is used to extract actual acoustic features from the data to be processed; input the actual acoustic features into the target noise reduction model to obtain the acoustic features after noise reduction; and perform speech recognition on the data to be processed based on the acoustic features after noise reduction.

[0065] In the above implementation process, the target noise reduction model trained by the embodiment of the present application is used to perform noise reduction and recognition on the actual noise data, which can achieve higher accuracy when performing noise reduction and speech recognition on the actual data to be processed, and can avoid the need for re-conversion in the speech recognition application, thereby improving the efficiency of model recognition.

[0066] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores program instructions, and when the processor reads and runs the program instructions, it executes the steps in any of the above implementation methods.

[0067] In a fourth aspect, an embodiment of the present application further provides a storage medium, wherein the readable storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the steps in any of the above implementation methods are executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0069] Figure 1 A schematic diagram of the steps of the denoising model training method provided in an embodiment of the present application;

[0070] Figure 2 A schematic diagram of a step of extracting acoustic features provided in an embodiment of the present application;

[0071] Figure 3 A schematic diagram of the steps for training a noise reduction model provided in an embodiment of the present application;

[0072] Figure 4 A schematic diagram of the steps of processing real noise data using a target noise reduction model provided in an embodiment of the present application;

[0073] Figure 5 A schematic diagram of a noise reduction model training device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0074] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they may sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of boxes in the block diagram and / or the flowchart, can be implemented by a dedicated hardware-based system that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part.

[0075] During the research, the applicant found that the methods of directly reducing the noise of noisy signals are mainly divided into methods of directly processing the signal and reducing the noise of noisy signals through neural networks. Among the technical means of reducing the noise of noisy signals based on neural networks, there are single-channel neural network speech noise reduction models and deep convolutional neural networks for speech enhancement of noisy signals. However, using a fixed signal-to-noise ratio in a single-channel neural network speech noise reduction model will reduce the generalization ability of the neural network model. When using a deep convolutional neural network, the network structure is complex and the training is difficult. When used as a subsequent application, the network measurement is too large. At the same time, the input and output of both neural networks are 257 dimensions, which cannot be applied to data with different sampling rates, and there is a problem of low practicality.

[0076] Based on this, the embodiment of the present application provides a noise reduction model training method, which uses the trained noise reduction model to perform noise reduction and speech recognition on noisy signals, thereby improving the generalization ability and recognition accuracy of the model. Figure 1 , Figure 1 A schematic diagram of the steps of the noise reduction model training method provided in an embodiment of the present application, the method may include the following steps:

[0077] In step S11, at least one random number is generated, noise information is determined based on the random number, and noise is added to the quiet data according to the noise information to obtain the noisy data, wherein the noise information includes: target noise data, a combined time period of the target noise data and the quiet data, and a signal-to-noise ratio of the target noise data and the quiet data.

[0078] Among them, the preset noise library can directly use an open source noise library, or can specifically collect various types of noise to establish a noise library for training a noise reduction model. The embodiment of the present application does not limit this.

[0079] Specifically, a random number interval can be set, and a random number generation function can be used to generate random numbers in the interval, and each random number corresponds to a noise data in the noise library. The random number generation function can be implemented by programming languages ​​or software such as C language, C++, C#, Java, Matlab, Hypertext Preprocessor (PHP), C51, etc.

[0080] In addition, physical random number generators can also be used, such as the noise of electronic components, Intel810RNG chip, or particle decay and air noise to generate random numbers.

[0081] The implementation means of randomly selecting the merging time period and randomly generating the signal-to-noise ratio can be implemented by using the above-mentioned method of generating random numbers through a random number generation function and using a physical random number generator, which will not be repeated here.

[0082] It should be understood that in a specific implementation process, any one or a combination of selecting noise data from a preset noise library, determining a merging time period of the noise data and quiet data, and determining a signal-to-noise ratio of the noise data and quiet data can be implemented based on a random number to achieve the technical effect of randomly adding noise to quiet data. When implementing the above process, the same random number can be used in multiple sub-steps of adding noise to quiet data, or a generated random number can be used in each sub-step.

[0083] In step S12, acoustic features are extracted from the noisy data, and a noise reduction model is trained based on the acoustic features to obtain a target noise reduction model.

[0084] It can be seen that the noise reduction model training method provided in the embodiment of the present application can randomly select noise data from the noise library, determine the merging time period of noise data and quiet data and the signal-to-noise ratio, so that the generated noisy data can more closely simulate the noise in reality, thereby enabling the trained target noise reduction model to have higher generalization ability and higher accuracy when performing noise reduction and speech recognition on real data to be processed.

[0085] In an optional embodiment, for step S12, the present application embodiment provides an implementation method for extracting acoustic features, please refer to Figure 2 , Figure 2 A schematic diagram of a step of extracting acoustic features provided in an embodiment of the present application, the step may include the following:

[0086] In step S121 , the frequency spectrum of the noisy data is adjusted based on a filter.

[0087] In step S122, the noise-added data is divided into frames.

[0088] In step S123, each frame of the noise-added data is windowed based on a window function.

[0089] In step S124, a fast Fourier transform is performed on the windowed noisy data to obtain a frequency domain signal of the noisy data.

[0090] In step S125, the frequency domain signal is filtered based on a Mel filter to obtain the acoustic feature.

[0091] Exemplarily, the acoustic feature extraction process of the embodiment of the present application is a filterbank feature extraction process. In step S121, the spectrum of the noise data can be adjusted by a first-order finite excitation response high-pass filter to make the spectrum of the signal flat. In step S122, according to the short-term stable characteristics of the speech signal, the noise data is processed in frames, and the noise data is divided into frames of data. In step S123, a Hamming window can be added to a frame of noise signal, and the signal of the noise data is truncated to facilitate the fast Fourier transform of the noise data in step S124, so as to transform the time domain signal of the noise data into the power spectrum of the signal, that is, the frequency domain signal. Finally, in step S125, the frequency domain signal is filtered to obtain a multidimensional vector of the noise data, which can be a 40-dimensional vector or an 80-dimensional vector.

[0092] Specifically, in step S125, a group of triangular window filters linearly distributed on the Mel frequency scale (a total of 24 triangular window filters) can be used to filter the frequency domain signal so that the range covered by each triangular window filter is close to a critical bandwidth of the human ear, thereby simulating the masking effect of the human ear.

[0093] It can be seen that in the step of extracting acoustic features provided in the embodiment of the present application, adjusting the spectrum of the noisy data can make the spectrum of the noisy data signal flat and not easily affected by the finite word length effect. Using a Hamming window to window a frame of noisy signal can reduce the impact of the Gibbs effect. The frequency domain signal is filtered based on a Mel filter to simulate the masking effect of the human ear. Based on the above characteristics, the accuracy of acoustic feature extraction is improved. On the other hand, the embodiment of the present application uses filterbank features as the input and output of the noise reduction model, which can avoid the problem of converting speech data again in speech recognition applications, thereby improving the efficiency of model recognition.

[0094] Optionally, with respect to the step S12 of training the noise reduction model based on the acoustic features, the present application embodiment further provides an implementation step of training the noise reduction model, see Figure 3 , Figure 3 A schematic diagram of a step of training a noise reduction model provided in an embodiment of the present application may include the following steps:

[0095] In step S31, for a piece of the noisy data, the noisy data is input into the denoising model and the trained speech recognition model respectively, a first loss value of the noisy data is obtained in the denoising model according to a first loss function, and a second loss value is obtained in the speech recognition model according to a second loss function.

[0096] Among them, the first loss function can be the MSE loss function, with the formula Calculation, where MSE is the first loss value, y i is the acoustic characteristic of quiet data, The second loss function in the speech recognition model can be a cross entropy loss function, with the formula H(p,q)=-∑ x p(x)ln(q(x)) is calculated, where H(p,q) is the value of the second loss function, p(x) is the probability of label classification of the speech recognition network, and q(x) is the probability of classification of the noisy data after passing through the speech recognition network.

[0097] In step S32, it is determined whether the combined loss value of the first loss value and the second loss value exceeds a preset loss threshold based on the interpolation combination method.

[0098] Exemplarily, an interpolation merging method can be used, based on the formula Loss = a × Loss1 + (1-a) × Loss2, where Loss is the combined loss value of the first loss value and the second loss value, Loss1 is the first loss value, Loss2 is the second loss value, and a is an empirical value parameter with a value range of (0,1). By judging whether the Loss value exceeds a set threshold, it is determined whether to remove this noisy data or use this noisy data to train the denoising model.

[0099] In step S33, when the combined loss value exceeds a preset loss threshold, the noisy data is removed.

[0100] In step S34, when the combined loss value does not exceed the preset loss threshold, it is determined to use the first loss value to train the denoising model.

[0101] It can be seen that the embodiment of the present application inputs the noisy data into the denoising model and the trained speech recognition model respectively, and uses a pre-trained speech recognition model to guide the training of the denoising model based on the interpolation and merging method, which can prevent the interference of long-tail data with too low signal-to-noise ratio generated by the noise addition, thereby improving the speech recognition accuracy of the denoising model.

[0102] Exemplarily, with respect to step S11, determining the noise adding information based on the random number may include one or more of the following:

[0103] Determine, based on a first random number among the random numbers, to select the noise data corresponding to the first random number from the preset noise library, and use the noise data as target noise data;

[0104] determining a merging time period of the target noise data and the quiet data based on a second random number among the random numbers;

[0105] A signal-to-noise ratio of a combination of the target noise data and the quiet data is determined based on a third random number among the random numbers.

[0106] Exemplarily, for the above-mentioned sub-step of determining, based on the first random number among the random numbers, to select the noise data corresponding to the first random number from the preset noise library and taking the noise data as the target noise data, the first random number can be generated based on a mixed congruential algorithm, the first random number being one of all random numbers uniformly distributed in a first preset interval, each number in the first preset interval corresponding to a piece of the noise data in the noise library; the noise data corresponding to the first random number is selected from the noise library as the target noise data

[0107] Among them, the first preset interval can be (1, N), N is the total number of noise audios in the noise library, in the interval, each number corresponds to one noise audio, and when the first random number is n, the nth noise data is selected from the noise library as the target noise data.

[0108] Exemplarily, for the sub-step of determining the merging time period of the target noise data and the quiet data based on the second random number in the random numbers in the above steps, the first random number can be used to determine the selected merging time period, or a second random number uniformly distributed on a second preset interval can be generated based on a mixed congruential algorithm. The second preset interval can be (0, In-Is), In is the duration of the noise data, Is is the duration of the quiet data, and the value of the second random number is used as the duration of the merging time period of the target noise data and the quiet data, thereby determining the merging time period of the target noise data and the quiet data.

[0109] Exemplarily, for the sub-step of determining the signal-to-noise ratio of the target noise data and the quiet data based on the third random number in the random number in the above step, it can be based on the formula Generate the third random number, the third random number is in accordance with the normal distribution A random number is selected from all random numbers of , μ is the mean, σ is the standard deviation; and the third random number is used as the ratio of the signal-to-noise ratio of the target noise data and the quiet data.

[0110] The signal-to-noise ratio of the noisy audio is controlled by the mean and variance of the normal distribution. The mean can be set to 10 and the standard deviation to 5, thereby ensuring that the signal-to-noise ratio has a 95% chance of being between 5 and 10.

[0111] It can be seen that the embodiments of the present application can randomly select noise data from the noise library, determine the merging time period of noise data and quiet data, and the signal-to-noise ratio, so that the generated noisy data can more closely simulate the noise in reality, thereby enabling the trained target noise reduction model to have a higher generalization ability and a higher accuracy when performing noise reduction and speech recognition on real data to be processed.

[0112] In addition, after completing the training of the denoising model through steps S11-S12 to obtain the target denoising model, the present embodiment provides an implementation step of using the target denoising model to process the real noise data, please refer to Figure 4 , Figure 4 A schematic diagram of a step of processing real noise data using a target noise reduction model provided in an embodiment of the present application, the steps may include the following:

[0113] In step S41, actual acoustic features are extracted from the data to be processed.

[0114] In step S42, the actual acoustic features are input into the target noise reduction model to obtain the acoustic features after noise reduction.

[0115] In step S43, speech recognition is performed on the data to be processed based on the acoustic features after noise reduction.

[0116] The steps of extracting actual acoustic features from the data to be processed can refer to the above steps S121-S125, which will not be repeated here.

[0117] It can be seen that the target denoising model trained by the embodiment of the present application is used to perform denoising and recognition on actual noise data, which can achieve higher accuracy when performing denoising and speech recognition on actual data to be processed. By directly using the filterbank features as the input and output of the model, it is possible to avoid the need for re-conversion in speech recognition applications, thereby improving the efficiency of model recognition.

[0118] Based on the same inventive concept, the present application embodiment also provides a noise reduction model training device 50, please refer to Figure 5 , Figure 5 A schematic diagram of a noise reduction model training device provided in an embodiment of the present application, the device 50 may include:

[0119] The noise adding module 51 is used to generate at least one random number, determine the noise adding information based on the random number, perform the noise adding operation on the quiet data according to the noise adding information, and obtain the noisy data, wherein the noise adding information includes: the target noise data, the merging time period of the target noise data and the quiet data, and the signal-to-noise ratio of the target noise data and the quiet data.

[0120] The training module 52 is used to extract acoustic features from the noisy data, and train the noise reduction model based on the acoustic features to obtain a target noise reduction model.

[0121] Optionally, the training module 52 may be specifically configured to perform one or more of the following steps:

[0122] Determine, based on a first random number among the random numbers, to select the noise data corresponding to the first random number from the preset noise library, and use the noise data as target noise data;

[0123] determining a merging time period of the target noise data and the quiet data based on a second random number among the random numbers;

[0124] A signal-to-noise ratio of a combination of the target noise data and the quiet data is determined based on a third random number among the random numbers.

[0125] Optionally, the training module 52 may be specifically used for:

[0126] The method comprises the following steps: adjusting the frequency spectrum of the noisy data based on a filter; dividing the noisy data in frames; windowing each frame of the noisy data based on a window function; performing a fast Fourier transform on the windowed noisy data to obtain a frequency domain signal of the noisy data; and filtering the frequency domain signal based on a Mel filter to obtain the acoustic feature.

[0127] Optionally, the training module 52 may also be used to:

[0128] For a piece of the noisy data, the noisy data is respectively input into the denoising model and the trained speech recognition model, a first loss value of the noisy data is obtained in the denoising model according to a first loss function, and a second loss value is obtained in the speech recognition model according to a second loss function; based on the interpolation merging method, it is determined whether the combined loss value of the first loss value and the second loss value exceeds a preset loss threshold; when the combined loss value exceeds the preset loss threshold, the noisy data is discarded; when the combined loss value does not exceed the preset loss threshold, it is determined to use the first loss value to train the denoising model.

[0129] Optionally, the noise adding module 51 may be specifically used for:

[0130] Based on a mixed congruential algorithm, the first random number is generated, the first random number is a random number among all random numbers uniformly distributed in a first preset interval, each number in the first preset interval corresponds to a piece of noise data in the noise library; and the noise data corresponding to the first random number is selected from the noise library as the target noise data.

[0131] Optionally, the noise adding module 51 may also be used for:

[0132] Generate the second random number based on a mixed congruential algorithm, where the second random number is a random number among all random numbers uniformly distributed in a second preset interval, where the second preset interval is (0, In-Is), where In is the duration of the target noise data, and Is is the duration of the quiet data;

[0133] The value of the second random number is used as the duration of the merging time period of the target noise data and the quiet data, thereby determining the merging time period of the target noise data and the quiet data.

[0134] Optionally, the noise adding module 51 may also be used for:

[0135] Based on the formula Generate the third random number, the third random number is in accordance with the normal distribution A random number among all random numbers, μ is the mean, σ is the standard deviation;

[0136] The third random number is used as a ratio of a signal-to-noise ratio of the target noise data and the quiet data.

[0137] Optionally, the denoising model training device 50 may further include:

[0138] The speech recognition module is used to extract actual acoustic features from the data to be processed; input the actual acoustic features into the target noise reduction model to obtain the acoustic features after noise reduction; and perform speech recognition on the data to be processed based on the acoustic features after noise reduction.

[0139] Based on the same inventive concept, an embodiment of the present application also provides an electronic device, which includes a memory and a processor, wherein program instructions are stored in the memory, and when the processor reads and runs the program instructions, the steps in any of the above implementation methods are executed.

[0140] Based on the same inventive concept, an embodiment of the present application further provides a storage medium, wherein the readable storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the steps in any of the above implementation methods are executed.

[0141] The storage medium may be a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or any other medium that can store program codes. The storage medium is used to store the program, and the processor executes the program after receiving the execution instruction. The method executed by the electronic terminal of the process definition disclosed in any embodiment of the present invention can be applied to the processor or implemented by the processor.

[0142] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0143] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0144] Furthermore, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0145] Alternatively, the present invention may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, the present invention may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part.

[0146] The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0147] In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0148] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A denoising model training method, characterized in that: include: Generate at least one random number, determine noise information based on the random number, perform noise addition operation on quiet data according to the noise addition information to obtain noisy data, wherein the noise addition information includes: target noise data, a combined time period of the target noise data and the quiet data, and a signal-to-noise ratio of the target noise data and the quiet data; Extracting acoustic features from the noisy data, and training a noise reduction model based on the acoustic features to obtain a target noise reduction model; Among them, the training of the noise reduction model based on the acoustic features includes: for a piece of the noisy data, inputting the noisy data into the noise reduction model and the trained speech recognition model respectively, obtaining a first loss value of the noisy data in the noise reduction model according to a first loss function, and obtaining a second loss value in the speech recognition model according to a second loss function; judging whether the combined loss value of the first loss value and the second loss value exceeds a preset loss threshold based on interpolation and merging; when the combined loss value exceeds the preset loss threshold, eliminating the noisy data; when the combined loss value does not exceed the preset loss threshold, judging to use the first loss value to train the noise reduction model.

2. The method according to claim 1, characterized in that The determining of the noise adding information based on the random number includes one or more of the following: Determine, based on a first random number among the random numbers, to select the noise data corresponding to the first random number from a preset noise library, and use the noise data as target noise data; determining a merging time period of the target noise data and the quiet data based on a second random number among the random numbers; A signal-to-noise ratio of a combination of the target noise data and the quiet data is determined based on a third random number among the random numbers.

3. The method according to claim 2, characterized in that The step of determining, based on a first random number among the random numbers, to select the noise data corresponding to the first random number from the preset noise library, and using the noise data as target noise data comprises: Based on the mixed congruential algorithm, the first random number is generated, and the first random number is a random number among all random numbers uniformly distributed in the first preset interval, and each number in the first preset interval corresponds to a piece of the noise data in the noise library. The noise data corresponding to the first random number is selected from the noise library as target noise data.

4. The method according to claim 2, characterized in that: The determining the merging time period of the target noise data and the quiet data based on the second random number among the random numbers comprises: Generate the second random number based on a mixed congruential algorithm, where the second random number is a random number among all random numbers uniformly distributed in a second preset interval, where the second preset interval is (0, In-Is), where In is the duration of the target noise data, and Is is the duration of the quiet data; The value of the second random number is used as the duration of the merging time period of the target noise data and the quiet data, thereby determining the merging time period of the target noise data and the quiet data.

5. The method according to claim 2, characterized in that: The step of determining the signal-to-noise ratio of the target noise data and the quiet data based on the third random number in the random numbers comprises: Probability density function formula based on normal distribution Generate the third random number, the third random number is in accordance with the normal distribution A random number among all random numbers in , μ is the mean, σ is the standard deviation, and x is a random number that conforms to the normal distribution; The third random number is used as a ratio of a signal-to-noise ratio of the target noise data and the quiet data.

6. The method according to claim 1, characterized in that The extracting acoustic features from the noisy data comprises: adjusting the frequency spectrum of the noisy data based on a filter; Dividing the noise-added data in frames; Adding a window to each frame of the noise-added data based on a window function; Performing a fast Fourier transform on the windowed noisy data to obtain a frequency domain signal of the noisy data; The frequency domain signal is filtered based on a Mel filter to obtain the acoustic feature.

7. The method according to any one of claims 1 to 6, characterized in that: After the noise reduction model is trained based on the acoustic features to obtain a target noise reduction model, the method further includes: extracting actual acoustic features from the data to be processed; Inputting the actual acoustic features into the target noise reduction model to obtain the acoustic features after noise reduction; Speech recognition is performed on the data to be processed based on the acoustic features after noise reduction.

8. A noise reduction model training device, characterized in that: include: a noise adding module, configured to generate at least one random number, determine noise adding information based on the random number, perform noise adding operation on quiet data according to the noise adding information, and obtain noisy data, wherein the noise adding information includes: target noise data, a merging time period of the target noise data and the quiet data, and a signal-to-noise ratio of the target noise data and the quiet data; A training module is used to extract acoustic features from the noisy data, and train a noise reduction model based on the acoustic features to obtain a target noise reduction model; wherein, the training of the noise reduction model based on the acoustic features includes: for one piece of the noisy data, inputting the noisy data into the noise reduction model and the trained speech recognition model respectively, obtaining a first loss value of the noisy data in the noise reduction model according to a first loss function, and obtaining a second loss value in the speech recognition model according to a second loss function; judging whether the combined loss value of the first loss value and the second loss value exceeds a preset loss threshold based on an interpolation and merging method; when the combined loss value exceeds the preset loss threshold, eliminating the noisy data; when the combined loss value does not exceed the preset loss threshold, judging to use the first loss value to train the noise reduction model.

9. A storage medium, characterized in that: The storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the steps in the method according to any one of claims 1 to 7 are executed.

Images