Noise reduction method, device, electronic device and computer-readable storage medium

Abstract

An embodiment of the present application provides a noise reduction method, device, electronic device, and computer-readable storage medium. The method includes: obtaining a clean speech training set according to the speech energy distribution characteristics; performing convolutional room impulse response on the clean speech training set to obtain simulated reverberant speech; adding noise to the simulated reverberant speech to obtain simulated noisy speech; calculating the normalized magnitude spectrum of the simulated noisy speech through a noise reduction model to obtain mask data, and performing compression processing on the mask data to obtain a compressed mask; decompressing the compressed mask according to a preset compression parameter to obtain a decompressed mask; and performing noise reduction processing on the real noisy speech according to the decompressed mask to obtain predicted clean speech. In this way, the speech noise reduction effect of a video conference can be improved.

Description

Technical Field

[0001] This application relates to the field of video technology, and in particular, to a noise reduction method, apparatus, electronic device, and computer-readable storage medium. Background Art

[0002] With the continuous development of video technology, people have higher and higher requirements for the call sound quality of conference video devices. The noise that appears during a video conference, especially sudden noise, will seriously affect the user experience of the video conference. At present, using a neural network model to eliminate sudden noise has become mainstream in the academic community. The existing solutions for eliminating sudden noise mainly use a section of clean speech as a label, add noise to it as the input data of the neural network model, and then directly train the neural network model. The speech environment of this solution is quite different from the real video conference environment, resulting in a relatively poor noise reduction effect when applied to real video conferences for noise reduction. Summary of the Invention

[0003] To solve the above technical problems, embodiments of this application provide a noise reduction method, apparatus, electronic device, and computer-readable storage medium.

[0004] In a first aspect, an embodiment of this application provides a noise reduction method, the method including:

[0005] Obtaining a clean speech training set according to the speech energy distribution feature;

[0006] Performing a convolutional room impulse response on the clean speech training set to obtain simulated reverberant speech;

[0007] Adding noise to the simulated reverberant speech to obtain simulated noisy speech;

[0008] Calculating the normalized amplitude spectrum of the simulated noisy speech through a noise reduction model to obtain mask data, and performing compression processing on the mask data to obtain a compressed mask;

[0009] Performing decompression processing on the compressed mask according to a preset compression parameter to obtain a decompressed mask;

[0010] Performing noise reduction processing on the real noisy speech according to the decompressed mask to obtain predicted clean speech.

[0011] In a second aspect, an embodiment of this application provides a noise reduction apparatus, the apparatus including:

[0012] An obtaining module, configured to obtain a clean speech training set according to the speech energy distribution feature;

[0013] An impulse module, configured to perform a convolutional room impulse response on the clean speech training set to obtain simulated reverberant speech;

[0014] A noise adding module, configured to add noise to the simulated reverberant speech to obtain simulated noisy speech;

[0015] A calculation module, configured to calculate the normalized magnitude spectrum of the simulated noisy speech through a noise reduction model to obtain mask data, and perform compression processing on the mask data to obtain a compressed mask;

[0016] A decompression module, configured to decompress the compressed mask according to a preset compression parameter to obtain a decompressed mask;

[0017] A noise reduction module, configured to perform noise reduction processing on the real noisy speech according to the decompressed mask to obtain predicted clean speech.

[0018] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor. The memory is used to store a computer program, and the computer program executes the noise reduction method provided in the first aspect when running on the processor.

[0019] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program executes the noise reduction method provided in the first aspect when running on a processor.

[0020] For the noise reduction method, device, electronic device, and computer-readable storage medium provided in the present application above, a clean speech training set is obtained according to the speech energy distribution characteristics; the clean speech training set is subjected to convolutional room impulse response to obtain simulated reverberant speech; the simulated reverberant speech is subjected to noise addition processing to obtain simulated noisy speech; the normalized magnitude spectrum of the simulated noisy speech is calculated through a noise reduction model to obtain mask data, and the mask data is subjected to compression processing to obtain a compressed mask; the compressed mask is decompressed according to a preset compression parameter to obtain a decompressed mask; the real noisy speech is subjected to noise reduction processing according to the decompressed mask to obtain predicted clean speech. By simulating the actual video conference scenario, simulated noisy speech is obtained, and then for the far-field environment, the normalization processing of the simulated noisy speech is performed to obtain the normalized magnitude spectrum, preventing extreme values input from affecting the performance of the noise reduction model, and compressing the output of the mask data output by the noise reduction model to ensure that the mask data can obtain a very large dynamic range at any signal-to-noise ratio, improving the real video conference noise reduction effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application and should not be regarded as limiting the protection scope of the present application. In each drawing, similar components are numbered similarly.

[0022] Figure 1 shows a schematic flowchart of the noise reduction method provided by an embodiment of the present application;

[0023] Figure 2 shows a schematic structural diagram of the noise reduction model provided by an embodiment of the present application;

[0024] Figure 3 shows another schematic flowchart of the noise reduction method provided by an embodiment of the present application;

[0025] Figure 4 shows a schematic structural diagram of the noise reduction device provided by an embodiment of the present application. Detailed implementation manners

[0026] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments.

[0027] Generally, the components of the embodiments of the present application described and illustrated herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the present application claimed, but merely represents selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative efforts fall within the scope of protection of the present application.

[0028] Hereinafter, the terms "including", "having" and their cognates that can be used in various embodiments of the present application are only intended to represent specific features, numbers, steps, operations, elements, components or combinations of the foregoing items, and should not be construed as first excluding the existence or adding the possibility of one or more other features, numbers, steps, operations, elements, components or combinations of the foregoing items.

[0029] In addition, the terms "first", "second", "third", etc. are only used for descriptive distinction and cannot be construed as indicating or implying relative importance.

[0030] Unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art to which various embodiments of the present application belong. The terms (such as those defined in a general-use dictionary) will be interpreted as having the same meaning as the contextual meaning in the relevant technical field and will not be interpreted as having an idealized meaning or an overly formal meaning unless clearly defined in various embodiments of the present application.

[0031] Embodiment 1

[0032] An embodiment of the present disclosure provides a noise reduction method.

[0033] Specifically, referring to Figure 1 , the noise reduction method includes:

[0034] Step S101, obtain a clean speech training set according to the speech energy distribution characteristics.

[0035] In this embodiment, based on the actual situation of video conferencing, the speech situations of users in the video conferencing can be divided into the following three categories: The first category: after user A finishes speaking, user B then speaks; The second category, user A and user B speak simultaneously; The third category, user C outputs speech alone. The speech energy distribution characteristics in each category can be determined according to the actual video conferencing speech situation, and then the corresponding clean speech training set can be obtained according to the speech energy distribution characteristics in each category.

[0036] Step S102, perform a convolutional room impulse response on the clean speech training set to obtain simulated reverberant speech.

[0037] In this embodiment, the convolutional room impulse response (RIR) can simulate the actual room reverberation for the clean speech training set to obtain simulated reverberant speech.

[0038] Step S103, add noise to the simulated reverberant speech to obtain simulated noisy speech.

[0039] In one implementation manner, noise data can be superimposed on the simulated reverberant speech according to a preset signal-to-noise ratio to obtain simulated noisy speech, where the preset signal-to-noise ratio can be randomly selected from [-5, 20] dB.

[0040] Step S104, calculate the normalized magnitude spectrum of the simulated noisy speech through a noise reduction model to obtain mask data, and perform compression processing on the mask data to obtain compressed masks.

[0041] In one implementation manner, the steps for obtaining the normalized magnitude spectrum of the simulated noisy speech include:

[0042] Calculate the normalized magnitude spectrum of the simulated noisy speech according to Formula 1:

[0043] Formula 1:

[0044] where |Y(t,f)| is the magnitude spectrum of the simulated noisy speech, and μ represents the average magnitude spectrum from the first frame magnitude spectrum to the current frame magnitude spectrum.

[0045] In one embodiment, the noise reduction model includes an input layer, a hidden layer, and an output layer. The hidden layer includes multiple gated recurrent units; the masked data includes a masked real part and a masked imaginary part.

[0046] Calculating the normalized magnitude spectrum of the simulated noisy speech through the noise reduction model to obtain masked data may include the following steps:

[0047] Input the normalized magnitude spectrum through the input layer. The normalized magnitude spectrum includes a first preset number of magnitude values, and the first preset number is determined according to the number of output units of each gated recurrent unit;

[0048] Calculate the normalized magnitude spectrum through each of the multiple gated recurrent units to obtain a calculation result sequence. The calculation result sequence includes a second preset number of result values, and the second preset number is determined according to the first preset number and the number of layers of the gated recurrent unit;

[0049] Output the calculation result sequence through the output layer, and split the calculation result sequence into the masked real part and the masked imaginary part.

[0050] Please refer to Figure 2 , the noise reduction model includes an input layer 201, a first gated recurrent unit 202, a second gated recurrent unit 203, and an output layer 204. The gated recurrent unit (GRU) is a gating mechanism in the recurrent neural network (RNN). The normalized magnitude spectrum input by the input layer 201 has 161 magnitude values. The first gated recurrent unit 202 and the second gated recurrent unit 203 each have 320 output units. The output layer 204 outputs 322 points. The first 161 points among the 322 points are used as the masked real part, and the last 161 points among the 322 points are used as the masked imaginary part. Supplementary note that the first preset number of magnitude values of the normalized magnitude spectrum can be calculated according to the following formula 2;

[0051] Formula 2: N = P / 2 + 1;

[0052] Wherein, N represents the first preset number of magnitude values of the normalized magnitude spectrum, and P represents the number of output units of the gated recurrent unit.

[0053] Specifically, for Figure 2 the shown noise reduction model, the first preset number of magnitude values of the normalized magnitude spectrum is 161, and its calculation process is as follows: 161 = 320 / 2 + 1.

[0054] In the noise reduction model of this embodiment, since only gated recurrent units are used, the unit data of the noise reduction model is relatively small, which can improve the real-time efficiency of the noise reduction model calculation, facilitate the embedding of the noise reduction model into the corresponding electronic device, improve the convenience of implementation, and enable various electronic devices to run the noise reduction model.

[0055] The reason for compressing the mask is explained below. In the time domain, the collected noisy speech can be expressed by the following formula 3:

[0056] Formula 3: y(t) = s(t) + n(t);

[0057] Among them, y(t), s(t), and n(t) represent the noisy speech, clean speech, and noise signal in the time domain respectively. Formula 3 can be converted into the time-frequency domain through a short-time Fourier transform (STFT) with a window length of 20 ms, a frame shift of 10 ms, and 320 points, that is, expressed by the following formula 4:

[0058] Formula 4: y(t, f) = s(t, f) + n(t, f).

[0059] Among them, y(t, f), s(t, f), and n(t, f) represent the noisy speech, clean speech, and noise signal in the time-frequency domain respectively.

[0060] The mask can be expressed as M(t, f) = M r (t, f) + iM r (t, f), where M r (t, f) and M r (t, f) represent the real part and the imaginary part of the mask. It can be deduced that the noise reduction process can be expressed as the complex product of the mask and the noisy speech, that is, it can be determined according to the following formula 5:

[0061] Formula 5: S r + iS i = (M r + iM i ) × (Y r + iY i ) = (M r Y r - M i Y i ) + i(M r Y i + M i Y r );

[0062] The following formula 6 can be deduced from formula 5:

[0063] Formula 6:

[0064] It can be seen that the output value ranges of the real part and the imaginary part of the mask obtained at this time are in (-∞, +∞). In a far-field noise environment, the mask is likely to reach some relatively extreme values, making it difficult for the noise reduction model to obtain a fitting result, which may cause the noise reduction model to fail to converge and may lead to divergence.

[0065] The real part and the imaginary part of the mask can be compressed according to the following formula 7;

[0066] Formula 7:

[0067] where cIRM represents the compressed real part of the mask or the compressed imaginary part of the mask, C represents the compression curve control coefficient, K represents the compression coefficient, and Mx represents the real part M r or the imaginary part M i of the mask. When Mx represents the real part of the mask, cIRM represents the compressed real part of the mask. When Mx represents the imaginary part of the mask, cIRM represents the compressed imaginary part of the mask.

[0068] Step S105, decompress the compressed mask according to the preset compression parameters to obtain a decompressed mask.

[0069] In an embodiment, the preset compression parameters include a compression coefficient and a compression curve control coefficient, and step S105 includes the following steps:

[0070] Calculate the difference and the sum of the compression curve control coefficient and the compressed mask respectively, and divide the difference by the sum to obtain the corresponding quotient value;

[0071] Calculate the logarithm of the quotient value, and calculate the decompressed mask according to the logarithm and the compression coefficient.

[0072] Specifically, the compressed mask can be decompressed according to the following formula 8 to obtain a decompressed mask.

[0073] Formula 8:

[0074] where represents the decompressed mask, C represents the compression curve control coefficient, and K represents the compression coefficient.

[0075] It should be noted that the compression curve control coefficient is used to control the steepness of the compression curve, and the compression coefficient is used to control the degree of compression. Specifically, the decompressed mask includes a decompressed real part and a decompressed imaginary part. The decompressed real part of the mask can be obtained by substituting the compressed real part of the mask into formula 8, and the decompressed imaginary part of the mask can be obtained by substituting the compressed imaginary part of the mask into formula 8. When O x is the compressed real part of the mask, represents the decompressed real part of the mask, and when O x is the compressed imaginary part of the mask, Represents the imaginary part of the decompression mask.

[0076] Step S106, perform noise reduction processing on the real noisy speech according to the decompression mask to obtain the predicted clean speech.

[0077] In one embodiment, step S106 includes the following steps:

[0078] Perform Fourier transform on the real noisy speech to obtain the Fourier transform result of the noisy speech;

[0079] Multiply the decompression mask by the Fourier transform result of the noisy speech to obtain the predicted clean speech.

[0080] Specifically, the decompression mask includes a real part of the decompression mask and an imaginary part of the decompression mask, and the decompression mask can be expressed as M = M r +iM i , where M represents the decompression mask, M r represents the real part of the decompression mask, and M i represents the imaginary part of the decompression mask. The Fourier transform result of the real noisy speech can be expressed as R = R r +iR i , where R represents the Fourier transform result of the real noisy speech, R r represents the real part of the real noisy speech, and R i represents the imaginary part of the real noisy speech.

[0081] Specifically, the predicted clean speech can be calculated according to the following formula 4;

[0082] Formula 9: S r +iS i =(M r +iM i )×(R r +iR i );

[0083] where S r +iS i represents the predicted clean speech, M r +iM i represents the decompression mask, and R r +iR i represents the real noisy speech.

[0084] In one embodiment, step S101 may include the following steps:

[0085] Obtain the first estimated probability of the splicing type, the second estimated probability of the superposition type, and the third estimated probability of the single type according to the speech energy distribution characteristics respectively;

[0086] Generate spliced clean speech, superimposed clean speech, and single clean speech respectively according to the first estimated probability, the second estimated probability, the third estimated probability, and the user speech data;

[0087] Determine the clean speech training set according to the spliced clean speech, the superimposed clean speech, and the single clean speech.

[0088] Specifically, three types of video speech scene types of spliced speech, superimposed speech, and single speech can be summarized and analyzed according to the speech energy distribution characteristics of an actual video conference, and the generation methods of each video speech type can be determined. Spliced speech is the speech with different energy levels before and after splicing, representing the video conference scene where speakers at different distances do not speak simultaneously. Superimposed speech is the speech obtained by superimposing two speeches with different energy levels, representing the video conference scene where speakers at different distances speak simultaneously. Single speech is the scene where a single speaker speaks. Actual video conference speech can be collected in advance, the speech energy distribution characteristics corresponding to the actual video conference speech can be detected, and the respective occurrence probabilities of the three types of video speech scene types of spliced speech, superimposed speech, and single speech can be determined according to the speech energy distribution characteristics, so as to determine the first estimated probability of the spliced type, the second estimated probability of the superimposed type, and the third estimated probability of the single type.

[0089] In one embodiment, the step of generating spliced clean speech, superimposed clean speech, and single clean speech respectively according to the first estimated probability, the second estimated probability, the third estimated probability, and the user speech data includes:

[0090] Obtain a first speech and a second speech with different energies from the user speech data according to the first estimated probability, and splice the first speech and the second speech to obtain spliced clean speech;

[0091] Obtain a third speech and a fourth speech with different energies from the user speech data according to the second estimated probability, and superimpose the third speech and the fourth speech to obtain superimposed clean speech;

[0092] Obtain a fifth speech of a single user from the user speech data according to the third estimated probability, and use the fifth speech as single clean speech.

[0093] In one embodiment, the step of determining the clean speech training set according to the spliced clean speech, the superimposed clean speech, and the single clean speech includes:

[0094] Select a preset proportion of speech data from the spliced clean speech, the superimposed clean speech, and the single clean speech as the clean speech training set.

[0095] It is understandable that the preset ratio can be customized. For example, the preset ratios can be different ratios such as 80%, 85%, etc., and there is no limitation here.

[0096] The following Figure 3 is an example of the acquisition process of the noisy speech provided in this embodiment.

[0097] Please refer to Figure 3 , including the following steps:

[0098] Step S301, generating probabilities corresponding to various clean speeches.

[0099] Step S302, determining that the probability P of the spliced speech is < 0.75.

[0100] Step S303, determining that the probability of the superimposed speech is 0.75 ≤ P ≤ 0.85.

[0101] Step S304, determining that the probability P of the single speech is > 0.85.

[0102] Step S305, obtaining two 2.5 - second clean speech data of different speakers.

[0103] Step S306, obtaining two 5 - second clean speech data of different speakers.

[0104] Step S307, obtaining one 5 - second clean speech data.

[0105] Step S308, performing random energy scaling on two speeches.

[0106] Step S309, performing random energy scaling on two speeches.

[0107] Step S310, splicing two speeches.

[0108] Step S311, superimposing two speeches.

[0109] Step S312, selecting part of the clean speech to perform convolutional room impulse response to obtain simulated reverberant speech.

[0110] Step S313, obtaining one 5 - second noise data, and superimposing the noise data according to the signal - to - noise ratio to obtain the noisy speech.

[0111] The noise reduction method provided in this embodiment obtains a clean speech training set according to the speech energy distribution characteristics; performs a convolutional room impulse response on the clean speech training set to obtain simulated reverberant speech; adds noise to the simulated reverberant speech to obtain simulated noisy speech; calculates the normalized magnitude spectrum of the simulated noisy speech through a noise reduction model to obtain mask data, performs compression processing on the mask data to obtain compressed masks; decompresses the compressed masks according to preset compression parameters to obtain decompressed masks; and performs noise reduction processing on the real noisy speech according to the decompressed masks to obtain predicted clean speech. By simulating the actual video conference scenario, simulated noisy speech is obtained, and then for the far-field environment, the normalized processing of the simulated noisy speech is performed to obtain the normalized magnitude spectrum, preventing extreme values in the input from affecting the performance of the noise reduction model, and compressing the output of the mask data output by the noise reduction model to ensure that the mask data can achieve a very large dynamic range at any signal-to-noise ratio, improving the real video conference noise reduction effect.

[0112] Embodiment 2

[0113] In addition, an embodiment of the present disclosure provides a noise reduction device.

[0114] Specifically, as Figure 4 shown, the noise reduction device 400 includes:

[0115] An acquisition module 401, configured to obtain a clean speech training set according to the speech energy distribution characteristics;

[0116] An impulse module 402, configured to perform a convolutional room impulse response on the clean speech training set to obtain simulated reverberant speech;

[0117] A noise addition module 403, configured to add noise to the simulated reverberant speech to obtain simulated noisy speech;

[0118] A calculation module 404, configured to calculate the normalized magnitude spectrum of the simulated noisy speech through a noise reduction model to obtain mask data, perform compression processing on the mask data to obtain compressed masks;

[0119] A decompression module 405, configured to decompress the compressed masks according to preset compression parameters to obtain decompressed masks;

[0120] A noise reduction module 406, configured to perform noise reduction processing on the real noisy speech according to the decompressed masks to obtain predicted clean speech.

[0121] In one embodiment, the noise reduction model includes an input layer, a hidden layer, and an output layer, the hidden layer includes multiple gated recurrent units; the mask data includes a mask real part and a mask imaginary part;

[0122] The calculation module 404 is further configured to input the normalized amplitude spectrum through the input layer, where the normalized amplitude spectrum includes a first preset number of amplitude values, and the first preset number is determined according to the number of output units of each of the gated recurrent units;

[0123] Calculate the normalized amplitude spectrum through each of the multi-layer gated recurrent units to obtain a calculation result sequence, where the calculation result sequence includes a second preset number of result values, and the second preset number is determined according to the first preset number and the number of layers of the gated recurrent unit;

[0124] Output the calculation result sequence through the output layer, and split the calculation result sequence into the real part of the mask and the imaginary part of the mask.

[0125] In one embodiment, the preset compression parameter includes a compression coefficient and a compression curve control coefficient. The decompression module 405 is further configured to calculate the difference and the sum of the compression curve control coefficient and the compression mask respectively, and divide the difference by the sum to obtain a corresponding quotient value;

[0126] Calculate the logarithm of the quotient value, and calculate the decompression mask according to the logarithm and the compression coefficient.

[0127] In one embodiment, the noise reduction module 406 is further configured to perform Fourier transform on the real noisy speech to obtain a Fourier transform result of the noisy speech;

[0128] Multiply the decompression mask by the Fourier transform result of the noisy speech to obtain the predicted clean speech.

[0129] In one embodiment, the acquisition module 401 is further configured to respectively obtain a first estimated probability of the splicing type, a second estimated probability of the superposition type, and a third estimated probability of the single type according to the speech energy distribution characteristics;

[0130] Generate a spliced clean speech, a superimposed clean speech, and a single clean speech according to the first estimated probability, the second estimated probability, the third estimated probability, and the user speech data respectively;

[0131] Determine the clean speech training set according to the spliced clean speech, the superimposed clean speech, and the single clean speech.

[0132] In one embodiment, the acquisition module 401 is further configured to obtain a first speech and a second speech with different energies from the user speech data according to the first estimated probability, and splice the first speech and the second speech to obtain a spliced clean speech;

[0133] Obtain a third voice and a fourth voice with different energies from the user voice data according to the second estimated probability, and superimpose the third voice and the fourth voice to obtain a superimposed clean voice;

[0134] Obtain a fifth voice of a single user from the user voice data according to the third estimated probability, and use the fifth voice as a single clean voice.

[0135] In one embodiment, the obtaining module 401 is further configured to select a preset proportion of voice data from the spliced clean voice, the superimposed clean voice, and the single clean voice as the clean voice training set.

[0136] The noise reduction device 500 provided in this embodiment can implement the noise reduction method shown in Embodiment 1. To avoid repetition, it will not be elaborated here.

[0137] The noise reduction device provided in this embodiment obtains a clean voice training set according to the voice energy distribution characteristics; performs a convolutional room impulse response on the clean voice training set to obtain a simulated reverberant voice; performs a noise addition process on the simulated reverberant voice to obtain a simulated noisy voice; calculates a mask data by a noise reduction model for the normalized magnitude spectrum of the simulated noisy voice, performs a compression process on the mask data to obtain a compressed mask; performs a decompression process on the compressed mask according to a preset compression parameter to obtain a decompressed mask; performs a noise reduction process on a real noisy voice according to the decompressed mask to obtain a predicted clean voice. By simulating an actual video conferencing scenario, a simulated noisy voice is obtained, and then for a far-field environment, the normalized processing of the simulated noisy voice is performed to obtain a normalized magnitude spectrum, preventing extreme values input from affecting the performance of the noise reduction model, and compressing the output of the mask data output by the noise reduction model, ensuring that the mask data can achieve a very large dynamic range at any signal-to-noise ratio, and improving the real video conferencing noise reduction effect.

[0138] Embodiment 3

[0139] In addition, an embodiment of the present disclosure provides an electronic device, including a memory and a processor, where the memory stores a computer program, and the computer program, when running on the processor, executes the noise reduction method provided in Method Embodiment 1 above.

[0140] The noise reduction device 500 provided in this embodiment can execute the noise reduction method shown in Embodiment 1. To avoid repetition, it will not be elaborated here.

[0141] Embodiment 4

[0142] The present application further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and the computer program, when running on a processor, executes the noise reduction method provided in Embodiment 1.

[0143] The computer-readable storage medium provided in this embodiment can execute the noise reduction method shown in Embodiment 1. To avoid repetition, it will not be elaborated here.

[0144] It should be noted that in this article, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such a process, method, article or terminal. Without more limitations, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, article or terminal including that element.

[0145] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described embodiment method can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of this application, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions for causing a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in various embodiments of this application.

[0146] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of this application, those of ordinary skill in the art can also make many forms without departing from the purpose of this application and the scope protected by the claims, and all of them belong to the protection scope of this application.

Claims

1. A noise reduction method, characterized in that, The method includes: Obtaining a clean speech training set according to the speech energy distribution characteristics; Performing convolutional room impulse response on the clean speech training set to obtain simulated reverberant speech; Adding noise to the simulated reverberant speech to obtain simulated noisy speech; Calculating the normalized magnitude spectrum of the simulated noisy speech through a noise reduction model to obtain mask data, and performing compression processing on the mask data to obtain a compressed mask; Performing decompression processing on the compressed mask according to a preset compression parameter to obtain a decompressed mask; Performing noise reduction processing on the real noisy speech according to the decompressed mask to obtain predicted clean speech; The step of obtaining clean speech data according to the speech energy distribution characteristics includes: Respectively obtaining a first estimated probability of the splicing type, a second estimated probability of the superposition type, and a third estimated probability of the single type according to the speech energy distribution characteristics; Generating spliced clean speech, superposed clean speech, and single clean speech according to the first estimated probability, the second estimated probability, the third estimated probability, and the user speech data respectively; Determining the clean speech training set according to the spliced clean speech, the superposed clean speech, and the single clean speech; The step of generating spliced clean speech, superposed clean speech, and single clean speech according to the first estimated probability, the second estimated probability, the third estimated probability, and the user speech data respectively includes: Obtaining a first speech and a second speech with different energies from the user speech data according to the first estimated probability, and splicing the first speech and the second speech to obtain spliced clean speech; Obtaining a third speech and a fourth speech with different energies from the user speech data according to the second estimated probability, and superposing the third speech and the fourth speech to obtain superposed clean speech; Obtaining a fifth speech of a single user from the user speech data according to the third estimated probability, and using the fifth speech as single clean speech.

2. The method according to claim 1, characterized in that, The noise reduction model includes an input layer, a hidden layer, and an output layer. The hidden layer includes multiple gated recurrent units; the mask data includes a mask real part and a mask imaginary part; The step of calculating the normalized magnitude spectrum of the simulated noisy speech through a noise reduction model to obtain mask data includes: Inputting the normalized magnitude spectrum through the input layer. The normalized magnitude spectrum includes a first preset number of magnitude values, and the first preset number is determined according to the output unit number of each gated recurrent unit; Calculating the normalized magnitude spectrum through each of the multiple gated recurrent units to obtain a calculation result sequence. The calculation result sequence includes a second preset number of result values, and the second preset number is determined according to the first preset number and the number of layers of the gated recurrent unit; Outputting the calculation result sequence through the output layer, and splitting the calculation result sequence into the mask real part and the mask imaginary part.

3. The method according to claim 1, characterized in that The preset compression parameter includes a compression coefficient and a compression curve control coefficient. The step of performing decompression processing on the compressed mask according to the preset compression parameter to obtain a decompressed mask includes: Calculate the difference and sum of the compression curve control coefficient and the compression mask respectively, and divide the difference by the sum to obtain the corresponding quotient value; Calculate the logarithm of the quotient value, and calculate the decompression mask according to the logarithm value and the compression coefficient.

4. The method according to claim 1, wherein The step of performing noise reduction processing on the real noisy speech according to the decompression mask to obtain the predicted clean speech includes: Perform Fourier transform on the real noisy speech to obtain the Fourier transform result of the noisy speech; Multiply the decompression mask by the Fourier transform result of the noisy speech to obtain the predicted clean speech.

5. The noise reduction method according to claim 1, wherein The step of determining the clean speech training set according to the spliced clean speech, the superimposed clean speech and the single clean speech includes: Select a preset proportion of speech data from the spliced clean speech, the superimposed clean speech and the single clean speech as the clean speech training set.

6. A noise reduction device, characterized in that, The device includes: An acquisition module, configured to acquire a clean speech training set according to the speech energy distribution feature; An impulse module, configured to perform convolutional room impulse response on the clean speech training set to obtain simulated reverberant speech; A noise addition module, configured to perform noise addition processing on the simulated reverberant speech to obtain simulated noisy speech; A calculation module, configured to calculate the normalized amplitude spectrum of the simulated noisy speech through a noise reduction model to obtain mask data, and perform compression processing on the mask data to obtain a compression mask; A decompression module, configured to perform decompression processing on the compression mask according to preset compression parameters to obtain a decompression mask; A noise reduction module, configured to perform noise reduction processing on the real noisy speech according to the decompression mask to obtain the predicted clean speech; Obtaining clean speech data according to the speech energy distribution feature includes: Respectively obtain a first estimated probability of the splicing type, a second estimated probability of the superimposing type, and a third estimated probability of the single type according to the speech energy distribution feature; Respectively generate a spliced clean speech, a superimposed clean speech, and a single clean speech according to the first estimated probability, the second estimated probability, the third estimated probability, and the user speech data; Determine the clean speech training set according to the spliced clean speech, the superimposed clean speech, and the single clean speech; The step of respectively generating a spliced clean speech, a superimposed clean speech, and a single clean speech according to the first estimated probability, the second estimated probability, the third estimated probability, and the user speech data includes: Obtain a first speech and a second speech with different energies from the user speech data according to the first estimated probability, and splice the first speech and the second speech to obtain a spliced clean speech; Obtain a third speech and a fourth speech with different energies from the user speech data according to the second estimated probability, and superimpose the third speech and the fourth speech to obtain a superimposed clean speech; Obtain a fifth speech of a single user from the user speech data according to the third estimated probability, and use the fifth speech as the single clean speech.

7. An electronic device, characterized in that, It includes a memory and a processor, the memory stores a computer program, and the computer program executes the noise reduction method according to any one of claims 1 to 5 when running on the processor.

8. A computer-readable storage medium, characterized in that, It stores a computer program which, when running on a processor, executes the noise reduction method according to any one of claims 1 to 5.

Images