Voice noise reduction method and device, electronic equipment and storage medium

Abstract

The invention discloses a voice noise reduction method and device, electronic equipment and a storage medium, and the method comprises the steps: carrying out the frequency band segmentation of a first time domain signal containing a target sampling point, so as to obtain a plurality of sub-band time domain signals; distinguishing a first sub-band time-domain signal which is human voice and a second sub-band time-domain signal which is noise according to the intensity parameter corresponding to each sub-band time-domain signal, carrying out noise reduction processing on the second sub-band time-domain signal, and determining the second time-domain signal after noise reduction by combining the first sub-band time-domain signal, according to the invention, the method can be suitable for a single-channel structure without reference noise, reduces the calculation complexity so as to reduce the resource consumption, and carries out the targeted processing of different frequency band noises, thereby solving a technical problem that the real-time performance and the low-resource demands of a microprocessor are difficult to give consideration to in the related technology.

Description

Technical Field

[0001] The present application relates to the field of speech signal processing technology, and in particular to a speech noise reduction method, device, electronic device and storage medium. Background Art

[0002] Noise reduction methods in related technologies include: time-domain noise reduction, such as adaptive filters, subspace decomposition, Kalman filtering, time-domain Wiener filtering, etc.; frequency-domain noise reduction, such as spectral subtraction, frequency-domain Wiener filtering, optimal improved logarithmic spectrum amplitude estimation + minimum tracking method, etc. Adaptive filter noise reduction requires reference noise and is not suitable for single-channel structures. The subspace decomposition algorithm is relatively complex to calculate and consumes a lot of resources. Time-domain Wiener filtering can only reduce high-frequency noise and has little effect on suppressing in-band noise. Frequency-domain filtering requires the introduction of Fourier transform, which is affected by the window length and has too high a delay. There is a technical problem that it is difficult to balance the requirements of real-time performance and low resource consumption on the microprocessor. Summary of the Invention

[0003] The embodiments of the present application provide a speech noise reduction method, device, electronic device, and storage medium, which can solve the technical problem of difficulty in balancing the requirements of real-time performance and low resource consumption on a microprocessor.

[0004] In a first aspect, this embodiment provides a speech noise reduction method, including: Acquire a first time domain signal including a target sampling point in the speech to be denoised; Performing frequency band segmentation processing on the first time domain signal to obtain a plurality of sub-band time domain signals; each of the sub-band time domain signals includes a signal corresponding to the target sampling point; determining, according to the intensity parameter corresponding to each of the sub-band time domain signals, a first sub-band time domain signal of human voice and a second sub-band time domain signal of noise among the multiple sub-band time domain signals; performing noise reduction processing on the second sub-band time domain signal to obtain a third sub-band time domain signal; A second time domain signal of the speech to be denoised after noise reduction is determined based on the first sub-band time domain signal and the third sub-band time domain signal.

[0005] In some embodiments, determining the second sub-band time domain signal that is noise among the plurality of sub-band time domain signals according to the intensity parameter corresponding to each of the sub-band time domain signals includes: Determining whether the value of the intensity parameter is less than a preset intensity threshold, and obtaining a first determination result; If the first judgment result indicates that the value of the strength parameter of the sub-band time domain signal is less than the strength threshold, the sub-band time domain signal is determined to be the second sub-band time domain signal.

[0006] In some embodiments, performing noise reduction processing on the second sub-band time domain signal to obtain a third sub-band time domain signal includes: Determining whether a value of a bandwidth parameter of the second sub-band time domain signal is greater than a preset bandwidth threshold, to obtain a second determination result; When the second judgment result indicates that the value of the bandwidth parameter of the second sub-band time domain signal is greater than the bandwidth threshold, noise reduction processing is performed on the second sub-band time domain signal according to a preset first noise reduction coefficient to obtain the third sub-band time domain signal.

[0007] In some embodiments, the method further comprises: Obtaining initial time domain signal and initial energy parameters; determining an average intensity parameter based on the initial time domain signal and the initial energy parameter; The intensity threshold is determined according to the average intensity parameter and a preset intensity coefficient.

[0008] In some embodiments, after determining the second sub-band time domain signal that is noise among the multiple sub-band time domain signals according to the intensity parameter corresponding to each of the sub-band time domain signals, the method further includes: The intensity threshold is updated based on the intensity parameter of the second sub-band time domain signal to obtain an updated intensity threshold.

[0009] In some embodiments, determining the noise-reduced second time domain signal based on the first sub-band time domain signal and the third sub-band time domain signal includes: Performing noise reduction processing on the first sub-band time domain signal according to a preset second noise reduction coefficient to obtain a fourth sub-band time domain signal; The third sub-band time domain signal and the fourth sub-band time domain signal are combined to obtain the second time domain signal.

[0010] In some embodiments, the combining the third sub-band time domain signal and the fourth sub-band time domain signal to obtain the second time domain signal includes: determining an overlapping interval between the third sub-band time domain signal and the fourth sub-band time domain signal; weighting the first signal value of the third sub-band time domain signal in the overlapping interval based on a preset first gain parameter to obtain a first weighted value; and weighting the second signal value of the fourth sub-band time domain signal in the overlapping interval based on a preset second gain parameter to obtain a second weighted value; the sum of the value of the first gain parameter and the value of the second gain parameter being a preset threshold; A target signal value of the second time domain signal is determined according to the first weighted value and the second weighted value.

[0011] In a second aspect, this embodiment further provides a speech noise reduction device, comprising: A first acquisition module is used to acquire a first time domain signal including a target sampling point in the speech to be denoised; a segmentation module, configured to perform frequency band segmentation processing on the first time domain signal to obtain a plurality of sub-band time domain signals; each of the sub-band time domain signals includes a signal corresponding to the target sampling point; a first determining module, configured to determine, according to the intensity parameter corresponding to each of the sub-band time domain signals, a first sub-band time domain signal of human voice and a second sub-band time domain signal of noise among the multiple sub-band time domain signals; a first noise reduction module, configured to perform noise reduction processing on the second sub-band time domain signal to obtain a third sub-band time domain signal; The second determining module is configured to determine a second time domain signal of the speech to be denoised after noise reduction based on the first sub-band time domain signal and the third sub-band time domain signal.

[0012] In a third aspect, this embodiment further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the method described above is implemented.

[0013] In a fourth aspect, this embodiment further provides a computer-readable storage medium on which a computer program is stored. The computer program is loaded by a processor to execute the steps in the described method.

[0014] In an embodiment of the present invention, a first time domain signal including a target sampling point is subjected to frequency band segmentation processing to obtain multiple sub-band time domain signals. A first sub-band time domain signal representing a human voice and a second sub-band time domain signal representing noise are distinguished based on intensity parameters corresponding to the sub-band time domain signals. After noise reduction processing is performed on the second sub-band time domain signal, the noise-reduced second time domain signal is determined in combination with the first sub-band time domain signal. This can achieve application to a single-channel structure without the need for reference noise, reduce computational complexity to reduce resource consumption, and simultaneously provide targeted processing for noise in different frequency bands, thereby improving the technical problem in related technologies of finding it difficult to strike a balance between real-time performance and low microprocessor resource requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 A flowchart of a speech noise reduction method provided in an embodiment of the present application; Figure 2 Another flowchart of a speech noise reduction method provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of a speech noise reduction device provided in an embodiment of the present application; Figure 4 A schematic diagram of the hardware structure of a speech noise reduction device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0018] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more features. In the description of the present application, "multiple" means two or more, unless otherwise clearly and specifically defined.

[0019] “A and / or B” includes the following three combinations: A only, B only, and a combination of A and B.

[0020] The use of "suitable for" or "configured to" in this application is intended to be open and inclusive language, and does not exclude devices that are adapted or configured to perform additional tasks or steps. In addition, the use of "based on" is intended to be open and inclusive, as a process, step, calculation, or other action that is "based on" one or more conditions or values ​​may, in practice, be based on additional conditions or values ​​beyond these.

[0021] In this application, the word "exemplary" is used to mean "serving as an example, illustration, or illustration." Any embodiment described in this application as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is given to enable any person skilled in the art to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that one of ordinary skill in the art can recognize that the present application can be implemented without using these specific details. In other instances, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in this application.

[0022] Low-latency noise reduction is a function of a recorder or audio capture device that is used to reduce background noise in audio to improve audio clarity. It also ensures that there is no excessive delay during audio transmission due to the introduction of algorithms, thereby ensuring the real-time nature of audio data.

[0023] Taking into account the real-time nature and the ability to run on a microprocessor with low computing resources, this application proposes a time-domain low-latency speech noise reduction algorithm that can not only meet the low-latency requirements of audio noise reduction in battlefield or event live broadcasts, stage live broadcasts, and crew recordings, but can also run on processors with smaller resources, reducing costs.

[0024] Figure 1 A flow chart of a speech noise reduction method provided in an embodiment of the present application is shown as follows: Figure 1 As shown, the audio file marking method provided in the embodiment of the present application may include but is not limited to the following steps and combinations of the following steps.

[0025] Step 101: Acquire a first time domain signal including a target sampling point in the speech to be denoised.

[0026] In this embodiment, the speech to be de-noised may be the original speech containing noise, and may include both valid speech such as human voices that need to be retained and interfering speech such as environmental noise and electronic noise. The speech to be de-noised may be determined based on actual conditions and is not limited here. As an example, the speech to be de-noised may come from an audio acquisition device such as a microphone or a recording device. The target sampling point may be a specific sampling point selected as a processing object in the time domain waveform of the speech to be de-noised, and the target sampling point may be any one or more sampling points in the speech to be de-noised. As an example, the target sampling point may be a target number of continuous sampling points, wherein the target number may be 10. The first time domain signal may be a time domain signal extracted from the speech to be de-noised and containing the above-mentioned target sampling point, wherein the time domain signal may be a waveform signal with time as the horizontal axis and signal amplitude as the vertical axis, and the time domain signal may directly reflect the characteristics of the signal changing over time.

[0027] In this embodiment, obtaining the first time domain signal including the target sampling point from the speech to be de-noised may include performing analog-to-digital conversion on the analog signal of the speech to be de-noised to obtain a sequence of discrete sampling points arranged in chronological order; sequentially storing the sampling points into a buffer starting from the start position of the sampling point sequence, forming an initial window when the number of sampling points in the buffer reaches a target number; and determining the sampling points corresponding to the initial window as the first time domain signal. It should be noted that when a new sampling point is input, a sliding window mechanism is used to remove the oldest sampling point stored in the buffer and simultaneously store the new sampling point, so that the buffer always maintains the target number of sampling points, forming an updated window, and determining the sampling points corresponding to the updated window as the first time domain signal.

[0028] Step 102: Perform frequency band segmentation processing on the first time domain signal to obtain multiple sub-band time domain signals; each sub-band time domain signal includes a signal corresponding to a target sampling point.

[0029] In this embodiment, the sub-band time domain signal may be a time domain signal obtained by frequency band splitting the first time domain signal. The sub-band time domain signal may correspond to a time domain signal in a specific frequency range (frequency band). The sub-band time domain signal may still use time as the horizontal axis and amplitude as the vertical axis, retaining the characteristics of the time domain signal, but only including the signal of a certain frequency band in the original first time domain signal.

[0030] In this embodiment, the frequency band splitting processing of the first time domain signal may be performed by using multiple biquad filters with different passband ranges to filter the first time domain signal separately to obtain multiple sub-band time domain signals. Because the first time domain signal includes a target sampling point, after being filtered by multiple filters, the sub-band time domain signals output by each filter include the signal of the target sampling point in the corresponding frequency band, that is, each sub-band time domain signal includes the signal corresponding to the target sampling point. The biquad filter may be a digital filter implemented based on a second-order difference equation, which has the characteristics of simple structure, low computational complexity, and easy design and implementation. By configuring coefficients, low-pass, band-pass, or high-pass filtering functions can be achieved, and it is suitable for extracting signals in specific frequency bands.

[0031] The present application performs frequency band segmentation on the first time domain signal through a biquad filter to obtain multiple sub-band time domain signals containing signals corresponding to target sampling points. Compared with the orthogonal mirror image (QMF) filter bank in the related art, the application significantly reduces the amount of calculation and is easier to implement on a microprocessor. At the same time, it is convenient to accurately divide the frequency band where noise is more distributed and the frequency band where human voice is mainly distributed.

[0032] Step 103: Determine, according to the intensity parameter corresponding to each sub-band time domain signal, a first sub-band time domain signal of human voice and a second sub-band time domain signal of noise among the multiple sub-band time domain signals.

[0033] In this embodiment, the intensity parameter may be a physical quantity used to describe the strength of a sub-band time domain signal. As an example, the intensity parameter may be the energy, power, or root mean square (RMS) level of the sub-band time domain signal. The value of the intensity parameter may directly reflect the energy accumulation or average intensity of the signal in the sub-band time domain signal. By analyzing the intensity parameter of each sub-band time domain signal, it is possible to distinguish between sub-band time domain signals that primarily include human voices and sub-band time domain signals that primarily include noise among multiple sub-band time domain signals. The first sub-band time domain signal may be a human voice sub-band time domain signal determined to primarily include human voices based on the intensity parameter; the second sub-band time domain signal may be a noise sub-band time domain signal determined to primarily include noise based on the intensity parameter.

[0034] In some embodiments, determining a second sub-band time domain signal that is noise among the multiple sub-band time domain signals according to the intensity parameter corresponding to each sub-band time domain signal includes: Determine whether the value of the intensity parameter is less than a preset intensity threshold, and obtain a first determination result; If the first judgment result indicates that the value of the intensity parameter of the sub-band time domain signal is less than the intensity threshold, the sub-band time domain signal is determined to be the second sub-band time domain signal.

[0035] In this embodiment, the preset intensity threshold may be a critical value used to distinguish between a sub-band time-domain signal and a vocal sub-band time-domain signal. As an example, the intensity threshold may be a threshold value pre-set through analysis of a large amount of sample data and may be dynamically adjusted based on actual scenarios. Before determining whether the intensity parameter value is less than the preset intensity threshold, the method further includes calculating an intensity parameter for each sub-band time-domain signal. The process for calculating the intensity parameters may be determined based on actual circumstances and is not limited herein.

[0036] In this embodiment, the value of the intensity parameter of each sub-band time domain signal can be compared with a preset intensity threshold to obtain a first judgment result; if the value of the intensity parameter of a sub-band time domain signal is less than the intensity threshold, the intensity of the sub-band time domain signal is low, which meets the characteristics of the noise signal, and the sub-band time domain signal is determined to be the second sub-band time domain signal.

[0037] In an embodiment of the present invention, a second sub-band time domain signal is determined to be noise by comparing and judging the intensity parameter of the sub-band time domain signal with a preset intensity threshold. The difference in intensity characteristics between human voice and noise is utilized to achieve accurate identification of the noise sub-band time domain signal, thereby achieving targeted positioning of the noise sub-band time domain signal. This improves the technical problem in related noise reduction methods that it is difficult to accurately distinguish between noise and human voice, resulting in insufficient noise reduction targeting or damage to effective speech, and improves the accuracy of noise reduction processing.

[0038] In some embodiments, the method further comprises: Obtaining initial time domain signal and initial energy parameters; determining an average intensity parameter based on the initial time domain signal and the initial energy parameter; The intensity threshold is determined according to the average intensity parameter and a preset intensity coefficient.

[0039] In this embodiment, the initial time domain signal may be a time domain signal acquired during the initial processing of the speech to be noise-reduced. The initial time domain signal may be a time domain signal used for noise estimation and calculation of an intensity threshold. The initial time domain signal may typically be the first few frames of time domain signal acquired upon powering up the audio acquisition device. Because the noise signals contained in the first few frames of time domain signal are relatively stable and do not contain human voice signals, they can serve as the basis for noise estimation. It should be noted that the initial time domain signal may include signals corresponding to initial sampling points. The initial sampling points may be a target number of sampling points.

[0040] In this embodiment, obtaining the initial time domain signal and the initial energy parameter may include obtaining the initial time domain signal of the first m frames of the speech to be denoised, where m is greater than or equal to 2; and determining the initial energy parameter based on the sampling value of each initial sampling point. The process of determining the initial energy parameter based on the initial sampling value can be determined based on actual conditions and is not limited here. Determining the average intensity parameter based on the initial time domain signal and the initial energy parameter may include calculating the arithmetic mean of the initial energy parameter values ​​of the first m frames of the initial time domain signal to obtain the value of the average intensity parameter to achieve noise estimation.

[0041] In this embodiment, the preset intensity coefficient may be a coefficient pre-set based on the energy characteristic difference between human voice and noise. The preset intensity coefficient may be used to convert the average intensity parameter into an intensity threshold that can distinguish between human voice and noise. As an example, the preset intensity coefficient may be 2 or 3. Determining the intensity threshold based on the average intensity parameter and the preset intensity coefficient may include multiplying the average intensity parameter and the preset intensity coefficient as the intensity threshold.

[0042] In an embodiment of the present invention, the average intensity parameter is calculated by using the initial time domain signal and the initial energy parameter, the intensity threshold is determined in combination with the preset intensity coefficient, and the intensity threshold of the current environment is determined based on the average intensity parameter of the initial time domain signal of the previous few frames, so that the intensity threshold is more in line with the actual noise environment, thereby improving the technical problem of low accuracy in distinguishing human voices from noise in different noise environments caused by the fixed intensity threshold in related technologies.

[0043] In some embodiments, after determining the second sub-band time domain signal that is noise among the multiple sub-band time domain signals according to the intensity parameter corresponding to each sub-band time domain signal, the method further includes: The intensity threshold is updated based on the intensity parameter of the second sub-band time domain signal to obtain an updated intensity threshold.

[0044] In this embodiment, after determining the second sub-band time domain signal, the arithmetic mean of the intensity parameter value of the second sub-band time domain signal and the intensity threshold value can be calculated to obtain an updated average intensity parameter, and the intensity threshold value can be re-determined based on the average intensity parameter and the preset intensity coefficient; alternatively, the intensity parameter value of the second sub-band time domain signal and the intensity threshold value can be weighted to obtain an updated average intensity parameter, and the intensity threshold value can be re-determined based on the average intensity parameter and the preset intensity coefficient. It should be noted that when the ambient noise increases, the value of the intensity parameter of the second sub-band time domain signal increases, the value of the updated average intensity parameter increases, and thus the intensity threshold value increases; when the ambient noise decreases, the value of the intensity parameter of the second sub-band time domain signal decreases, the value of the updated average intensity parameter decreases, and thus the intensity threshold value decreases.

[0045] In an embodiment of the present invention, by updating the intensity threshold based on the intensity parameter of the second sub-band time domain signal and dynamically adjusting the intensity threshold in combination with the current noise energy, real-time adaptation of the intensity threshold to changing environmental noise can be achieved, which can improve the technical problem in the related art of reduced accuracy in distinguishing human voices from noise when the noise environment changes due to the fixed intensity threshold.

[0046] Step 104: Perform noise reduction processing on the second sub-band time domain signal to obtain a third sub-band time domain signal.

[0047] In this embodiment, an adaptive noise reduction strategy can be employed based on the noise characteristics (e.g., noise intensity, bandwidth, frequency distribution, etc.) of the second sub-band time-domain signal. For example, different noise reduction coefficients can be selected based on the bandwidth parameters of the sub-band time-domain signal. A stronger attenuation coefficient can be used for broadband noise sub-band time-domain signals, while a weaker attenuation coefficient can be used for narrowband noise sub-band time-domain signals. Another example is that the degree of noise reduction can be dynamically adjusted based on the noise intensity parameter. When the noise intensity is high, the noise reduction can be enhanced. In another example, filtering can be used to further remove noise signals from the sub-band time-domain signal. Through these filtering processes, the noise energy in the second sub-band time-domain signal can be suppressed, ultimately resulting in a noise-reduced third sub-band time-domain signal.

[0048] In some embodiments, performing noise reduction processing on the second sub-band time domain signal to obtain a third sub-band time domain signal includes: Determine whether a value of a bandwidth parameter of the second sub-band time domain signal is greater than a preset bandwidth threshold, and obtain a second determination result; When the second judgment result indicates that the value of the bandwidth parameter of the second sub-band time domain signal is greater than the bandwidth threshold, the second sub-band time domain signal is denoised according to a preset first noise reduction coefficient to obtain a third sub-band time domain signal.

[0049] In this embodiment, the bandwidth parameter may be a parameter used to describe the width of the frequency range covered by the second sub-band time-domain signal. The value of the bandwidth parameter may be the difference between the highest frequency and the lowest frequency of the second sub-band time-domain signal. The bandwidth parameter may directly reflect the frequency coverage range of the sub-band. The preset bandwidth threshold may be a critical value used to distinguish whether the second sub-band time-domain signal is a wideband sub-band time-domain signal or a narrowband sub-band time-domain signal.

[0050] In this embodiment, before determining whether the value of the bandwidth parameter of the second sub-band time domain signal is greater than a preset bandwidth threshold, the method further includes calculating the difference between the highest frequency and the lowest frequency of the second sub-band time domain signal to obtain the value of the bandwidth parameter of the second sub-band time domain signal. The first noise reduction coefficient may be a preset noise reduction attenuation coefficient for a broadband noise sub-band time domain signal whose bandwidth parameter is greater than the bandwidth threshold. The value of the first noise reduction coefficient can be set based on the requirement for broadband noise suppression and is not limited herein. It should be noted that the first noise reduction coefficients of different second sub-band time domain signals may be different.

[0051] In this embodiment, the value of the bandwidth parameter of each second sub-band time domain signal can be compared with a preset bandwidth threshold to obtain a second judgment result. If the value of the bandwidth parameter of a second sub-band time domain signal is greater than the bandwidth threshold, it indicates that the second sub-band time domain signal is a broadband noise sub-band time domain signal, and the second sub-band time domain signal is a sub-band time domain signal with a relatively large noise distribution. The second sub-band time domain signal is subjected to noise reduction processing using a preset first noise reduction coefficient. For example, the first noise reduction coefficient is multiplied by the value of the amplitude parameter of the second sub-band time domain signal to obtain the value of the amplitude parameter of the third sub-band time domain signal in which the noise is effectively suppressed.

[0052] In an embodiment of the present invention, by determining the size relationship between the bandwidth parameter of the second sub-band time domain signal and a preset bandwidth threshold, a first noise reduction coefficient is used to perform noise reduction processing on the noise sub-band time domain signal having a larger bandwidth parameter value. This can achieve targeted suppression of broadband sub-bands with a larger noise distribution, and can improve technical problems in related technologies such as the lack of differentiation in noise suppression for different bandwidth sub-bands, resulting in insufficient broadband noise suppression or excessive attenuation of narrowband speech signals, thereby improving the accuracy and effectiveness of noise reduction.

[0053] Step 105: Determine a second time domain signal of the speech to be denoised after noise reduction based on the first sub-band time domain signal and the third sub-band time domain signal.

[0054] In this embodiment, the first sub-band time domain signal and the noise-reduced third sub-band time domain signal can be integrated to form a complete signal covering the full frequency range of the original speech to be noise-reduced. The integration method includes but is not limited to: performing in-band noise suppression on the first sub-band time domain signal to obtain the noise-reduced first sub-band time domain signal; merging the frequency range of the noise-reduced first sub-band time domain signal and the third sub-band time domain signal in a time domain order. In some embodiments, for the overlapping interval of the first sub-band time domain signal and the third sub-band time domain signal, gain weighting can be used for smooth transition to avoid signal mutation; for the non-overlapping interval, the signal characteristics of each first sub-band time domain signal and the third sub-band time domain signal after processing can be directly retained, and finally a second time domain signal with continuous time domain and complete frequency is formed through time domain superposition.

[0055] In some embodiments, determining the second time domain signal after noise reduction based on the first sub-band time domain signal and the third sub-band time domain signal includes: Performing noise reduction processing on the first sub-band time domain signal according to a preset second noise reduction coefficient to obtain a fourth sub-band time domain signal; The third sub-band time domain signal and the fourth sub-band time domain signal are combined to obtain a second time domain signal.

[0056] In this embodiment, the preset second noise reduction coefficient may be a noise reduction attenuation coefficient preset for the first sub-band time domain signal. The second noise reduction coefficient may be the same as or different from the first noise reduction coefficient. The attenuation strength of the second noise reduction coefficient may be weaker than the attenuation strength of the first noise reduction coefficient. The second noise reduction coefficient may function to suppress noise within the first sub-band time domain signal without damaging the primary human voice. The first noise reduction coefficient and the second noise reduction coefficient may represent the noise reduction depth for the sub-band time domain signal.

[0057] In this embodiment, noise reduction processing is performed on the first subband time domain signal according to a preset second noise reduction coefficient to obtain a fourth subband time domain signal. The second noise reduction coefficient is multiplied by the amplitude parameter of the first subband time domain signal to reduce in-band noise while retaining vocal energy, thereby obtaining the value of the amplitude parameter of the fourth subband time domain signal. The third subband time domain signal and the fourth subband time domain signal are combined to obtain a second time domain signal. The amplitude parameter values ​​of sampling points corresponding to the same moment in the third and fourth subband time domain signals are superimposed to recombine signals in different frequency bands to form a second time domain signal covering the full frequency range of the original signal. The second time domain signal may be a complete speech signal after noise reduction.

[0058] In an embodiment of the present invention, by performing in-band noise suppression on the first sub-band time domain signal according to the second noise reduction coefficient to obtain a fourth sub-band time domain signal, and merging the fourth sub-band time domain signal with the third sub-band time domain signal to obtain a second time domain signal, it is possible to effectively suppress the noise in the human voice sub-band time domain signal while retaining the main human voice signal. This can improve the technical problem in the related art of only focusing on noise sub-band noise reduction while ignoring the noise in the human voice sub-band, resulting in the final signal still containing in-band noise, and improve the purity of the speech signal after noise reduction.

[0059] In some embodiments, combining the third sub-band time domain signal and the fourth sub-band time domain signal to obtain the second time domain signal includes: determining an overlapping interval of the third sub-band time domain signal and the fourth sub-band time domain signal; weighting the first signal value of the third sub-band time domain signal in the overlapping interval based on a preset first gain parameter to obtain a first weighted value; and weighting the second signal value of the fourth sub-band time domain signal in the overlapping interval based on a preset second gain parameter to obtain a second weighted value; the sum of the value of the first gain parameter and the value of the second gain parameter being a preset threshold value; A target signal value of the second time domain signal is determined according to the first weighted value and the second weighted value.

[0060] In this embodiment, the overlapping interval may be a range of sampling points where the third subband time-domain signal and the fourth subband time-domain signal overlap in the time domain. The overlapping interval may be a region where both subband signals have valid signal values ​​in the same time interval. The first gain parameter may be a preset weighting coefficient for the third subband time-domain signal within the overlapping interval. The value of the first gain parameter may dynamically change with the position of the sampling point within the overlapping interval. For example, the value of the first gain parameter may decrease as the position of the sampling point changes. The value of the first gain parameter may be between 1 and 0.2.

[0061] In this embodiment, the second gain parameter may be a preset weighting coefficient for the fourth subband time domain signal within the overlapping interval. The value of the second gain parameter may dynamically change with the sampling point position. For example, the value of the second gain parameter may increase as the sampling point position changes. The value of the second gain parameter may be 0.2-1, and the sum of the second gain parameter and the first gain parameter is always equal to a preset threshold. The preset threshold can be determined based on actual conditions and is not limited here. As an example, the preset threshold may be 1. The first signal value may be the value of the amplitude parameter of the third subband time domain signal at the corresponding sampling point within the overlapping interval; the second signal value may be the value of the amplitude parameter of the fourth subband time domain signal at the corresponding sampling point within the overlapping interval.

[0062] In this embodiment, determining the overlapping interval of the third subband time domain signal and the fourth subband time domain signal may include analyzing the time domain distribution of the third subband time domain signal and the fourth subband time domain signal to obtain the overlapping interval of the third subband time domain signal and the fourth subband time domain signal on the time axis. A first gain parameter may be assigned to the sampling points of the third subband time domain signal within the overlapping interval, and a second gain parameter may be assigned to the sampling points of the fourth subband time domain signal; a first signal value of the third subband time domain signal in the overlapping interval is multiplied by the corresponding first gain parameter to obtain a first weighted value, and a second signal value of the fourth subband time domain signal is multiplied by the corresponding second gain parameter to obtain a second weighted value; the first weighted value and the second weighted value are added for each sampling point to obtain a target signal value for the sampling point, and the signal value of the subband time domain signal corresponding to the non-overlapping interval may be directly retained to form the second time domain signal as a whole.

[0063] In an embodiment of the present invention, by determining the overlapping interval of the third sub-band time domain signal and the fourth sub-band time domain signal, and using complementary gain parameters for weighted merging, the signal values ​​in the overlapping interval are smoothly transitioned, thereby achieving seamless merging of the sub-band time domain signals, thereby improving the technical problem in the related art that direct merging of sub-band signals leads to signal mutations in the transition area, the introduction of ticking sounds or distortion, and improving the auditory quality of the speech signal after noise reduction.

[0064] The following describes the speech noise reduction method provided by the embodiments of the present application.

[0065] This application aims to provide a low-latency speech noise reduction method. This method is achieved by performing noise estimation, sub-band segmentation, level detection, threshold determination, and then noise reduction processing and noise updating on the noisy speech. Using 10 sampling points and 48kHz audio, the delay is less than 1ms. This method achieves low latency requirements while operating on low-resource platforms, making it applicable to various audio scenarios. Figure 2 Another flow chart of a speech noise reduction method provided in an embodiment of the present application is as follows: Figure 2 As shown, the speech noise reduction method includes: Start a real-time audio data stream. Initialize the noise estimate and calculate the initial threshold T. Exemplarily, the noise estimation method uses the average of the energy parameters of the previous frames as the initial noise estimate. The energy parameter of each frame is E = (1 / n) * sqrt (x1 * x1 + x2 * x2 ... + xn * xn), where x is the sampling point value within a frame and n is the frame length. The initial threshold T can be set to 2 or 3 times the noise estimate. Cache 10 sampling points as a frame of time domain signal. Exemplarily, a sliding window technique can be used, with every 10 sampling points as a frame of time domain signal. Data samples are calculated for each frame of time domain signal, for example, to extract statistics reflecting the characteristics of the current audio segment.

[0066] The time domain signal is sub-band segmented to obtain multiple sub-band time domain signals. For example, the purpose of sub-band segmentation is to use a digital filter to segment the time domain signal into frequency bands, which can suppress the frequency bands with more noise distribution, thereby retaining the human voice frequency band part. Common sub-band segmentation methods are to use low-pass, band-pass, and high-pass filters for segmentation. A biquad filter can be used to divide the time domain signal into multiple sub-bands such as 0-400hz, 400-800, 800-2000, 2000-4000 and 4000-8000. The energy values ​​of multiple sub-band time domain signals are counted. For example, the energy value of the signal can be calculated based on the level value of the signal, and the level value is usually calculated using the RMS method. The noise reduction depth can be set for each sub-band time domain signal.

[0067] Determine whether the energy value of the sub-band time-domain signal is less than a threshold value T. For example, in most scenarios, the energy of human voice is greater than that of noise. If the energy of human voice is less than that of noise, the voice will be masked by the noise. Based on this criterion, an initial threshold value T is set through data analysis such as the noise estimation described above, and then dynamically adjusted. Threshold T is primarily used to distinguish between noise and human voice. If it is above the threshold, the sub-band time-domain signal is a voice sub-band time-domain signal; if it is below the threshold, the sub-band time-domain signal is a noise sub-band time-domain signal.

[0068] If so, the noise energy value is recorded to update the noise estimate. Exemplarily, when the current frame sub-band time-domain signal is determined to be a noise sub-band time-domain signal, the changing ambient noise can be dynamically identified. The noise estimate is updated by statistically analyzing the noise energy. Noise reduction is performed on the noise sub-band time-domain signal based on the noise reduction depth. Exemplarily, based on the threshold determination result, the human voice sub-band time-domain signal and the noise sub-band time-domain signal are determined. If the signal is a noise sub-band time-domain signal, noise reduction is performed on the frame sub-band time-domain signal using the set noise reduction coefficient.

[0069] If not, in-band noise suppression is performed on the vocal subband time-domain signal based on the noise reduction depth. Exemplarily, the noise portion of the vocal subband time-domain signal is suppressed using a set noise reduction coefficient. Speech post-processing is performed. Exemplarily, the speech segment is smoothed using a fade-in / fade-out technique to avoid audio data loss or the introduction of clicks due to tail swallowing or discontinuities.

[0070] This application provides a low-latency noise reduction solution architecture. It initializes a level threshold and noise estimate from a 48K sampling rate audio data stream, caches 10 sampling points (0.2ms latency), and uses a bandpass filter to divide the audio into subbands. The average level within the current frequency band is then calculated. Combining sliding window technology and threshold judgment, it determines whether a human voice is currently present. If so, post-processing and filtering are performed on the human voice time domain signal to suppress in-band noise. If not, the noise level is calculated, and the level threshold and noise estimate are updated. Noise reduction is then performed simultaneously, followed by speech post-processing and filtering.

[0071] Figure 3 A schematic diagram of the structure of a speech noise reduction device provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, the speech noise reduction device 300 includes: A first acquisition module 301 is configured to acquire a first time domain signal including a target sampling point in the speech to be denoised; The segmentation module 302 is configured to perform frequency band segmentation processing on the first time domain signal to obtain a plurality of sub-band time domain signals; each sub-band time domain signal includes a signal corresponding to a target sampling point; A first determining module 303 is configured to determine, according to the intensity parameter corresponding to each sub-band time domain signal, a first sub-band time domain signal of a human voice and a second sub-band time domain signal of noise among the multiple sub-band time domain signals; A first noise reduction module 304 is configured to perform noise reduction processing on the second sub-band time domain signal to obtain a third sub-band time domain signal; The second determining module 305 is configured to determine a second time domain signal after noise reduction of the speech to be denoised based on the first sub-band time domain signal and the third sub-band time domain signal.

[0072] In some embodiments, the first determination module 303 is further used to determine whether the value of the intensity parameter is less than a preset intensity threshold to obtain a first judgment result; if the first judgment result indicates that the value of the intensity parameter of the sub-band time domain signal is less than the intensity threshold, the sub-band time domain signal is determined to be a second sub-band time domain signal.

[0073] In some embodiments, the first noise reduction module 304 is further used to determine whether the value of the bandwidth parameter of the second sub-band time domain signal is greater than a preset bandwidth threshold to obtain a second judgment result; when the second judgment result indicates that the value of the bandwidth parameter of the second sub-band time domain signal is greater than the bandwidth threshold, the second sub-band time domain signal is subjected to noise reduction processing according to a preset first noise reduction coefficient to obtain a third sub-band time domain signal.

[0074] In some embodiments, the speech noise reduction device 300 further includes: a second acquisition module for acquiring an initial time domain signal and an initial energy parameter; a third determination module for determining an average intensity parameter based on the initial time domain signal and the initial energy parameter; and a fourth determination module for determining an intensity threshold based on the average intensity parameter and a preset intensity coefficient.

[0075] In some embodiments, the speech noise reduction apparatus 300 further includes: an updating module, configured to update the intensity threshold based on the intensity parameter of the second sub-band time domain signal to obtain an updated intensity threshold.

[0076] In some embodiments, the second determination module 305 is further used to perform noise reduction processing on the first sub-band time domain signal according to a preset second noise reduction coefficient to obtain a fourth sub-band time domain signal; and combine the third sub-band time domain signal and the fourth sub-band time domain signal to obtain a second time domain signal.

[0077] In some embodiments, the second determination module 305 is further used to determine the overlapping interval of the third subband time domain signal and the fourth subband time domain signal; weight the first signal value of the third subband time domain signal in the overlapping interval based on a preset first gain parameter to obtain a first weighted value; and weight the second signal value of the fourth subband time domain signal in the overlapping interval based on a preset second gain parameter to obtain a second weighted value; the sum of the value of the first gain parameter and the value of the second gain parameter is a preset threshold; and determine the target signal value of the second time domain signal based on the first weighted value and the second weighted value.

[0078] To implement the method of the embodiment of the present application, Figure 4 A hardware structure diagram of a speech noise reduction device provided in an embodiment of the present application is shown as follows: Figure 4 As shown, an embodiment of the present application further provides a speech noise reduction device 40 that may include: a memory 401 for storing a computer program; and a processor 402 for implementing any of the aforementioned methods when executing the computer program. For example, processor 402 may be used to implement the steps of any of the aforementioned methods, which will not be described in detail here.

[0079] It should be noted that the speech noise reduction device and the speech noise reduction method provided in the above embodiments belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0080] Of course, in actual application, Figure 4 As shown, the speech noise reduction device 40 may also include: at least one network interface 403. The various components in the speech noise reduction device are coupled together via a bus system 404. It is understood that the bus system 404 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 404 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 4Various buses are labeled as bus system 404. There may be at least one processor 402. The network interface 403 is used for wired or wireless communication between the speech noise reduction device and other devices. The memory 401 in the embodiments of this application is used to store various types of data to support the operation of the speech noise reduction device. The methods disclosed in the embodiments of this application can be applied to or implemented by the processor 402. The processor 402 may be an integrated circuit chip with signal processing capabilities. During implementation, the steps of the methods described above can be performed by hardware integrated logic circuits or software instructions in the processor 402. The processor 402 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc. The processor 402 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly implemented and executed by a combination of hardware and software modules in a single-chip microcomputer. The software module may be located in a storage medium located in memory 401. Processor 402 reads the information in memory 401 and, in conjunction with its hardware, completes the steps of the aforementioned method. In an exemplary embodiment, speech noise reduction device 40 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0081] Specifically, embodiments of the present application provide a computer-readable storage medium having a computer program stored thereon, such as a memory 401 storing the computer program. The computer program can be executed by a processor 402 to perform the aforementioned method steps. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface mount storage, optical disk, or CD-ROM.

[0082] In addition, all functional units in the embodiments of the present application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the above-mentioned integrated units can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0083] Those skilled in the art will appreciate that all or part of the steps of the above-mentioned method embodiments may be implemented by hardware associated with program instructions, and the aforementioned program may be stored in a computer-readable storage medium. When the program is executed, the program executes the steps of the above-mentioned method embodiments. The aforementioned storage medium includes various media that can store program codes, such as mobile storage devices, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0084] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application, or the part that contributes to the relevant technology, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods of each embodiment of the present application. The aforementioned storage medium includes: various media that can store program code, such as mobile storage devices, ROM, RAM, disks or optical disks.

[0085] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the detailed description of other embodiments above and will not be repeated here.

[0086] The above is a detailed introduction to a speech noise reduction method, device, electronic device and storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present application.

Claims

1. A speech noise reduction method, characterized in that: include: Acquire a first time domain signal including a target sampling point in the speech to be denoised; Performing frequency band segmentation processing on the first time domain signal to obtain a plurality of sub-band time domain signals; each of the sub-band time domain signals includes a signal corresponding to the target sampling point; determining, according to the intensity parameter corresponding to each of the sub-band time domain signals, a first sub-band time domain signal of human voice and a second sub-band time domain signal of noise among the multiple sub-band time domain signals; performing noise reduction processing on the second sub-band time domain signal to obtain a third sub-band time domain signal; A second time domain signal of the speech to be denoised after noise reduction is determined based on the first sub-band time domain signal and the third sub-band time domain signal.

2. The method according to claim 1, characterized in that The determining, according to the intensity parameter corresponding to each of the sub-band time domain signals, a second sub-band time domain signal that is noise among the multiple sub-band time domain signals comprises: Determining whether the value of the intensity parameter is less than a preset intensity threshold, and obtaining a first determination result; If the first judgment result indicates that the value of the strength parameter of the sub-band time domain signal is less than the strength threshold, the sub-band time domain signal is determined to be the second sub-band time domain signal.

3. The method according to claim 1, characterized in that The performing noise reduction processing on the second sub-band time domain signal to obtain a third sub-band time domain signal includes: Determining whether a value of a bandwidth parameter of the second sub-band time domain signal is greater than a preset bandwidth threshold, to obtain a second determination result; When the second judgment result indicates that the value of the bandwidth parameter of the second sub-band time domain signal is greater than the bandwidth threshold, noise reduction processing is performed on the second sub-band time domain signal according to a preset first noise reduction coefficient to obtain the third sub-band time domain signal.

4. The method according to claim 2, characterized in that The method further comprises: Obtaining initial time domain signal and initial energy parameters; determining an average intensity parameter based on the initial time domain signal and the initial energy parameter; The intensity threshold is determined according to the average intensity parameter and a preset intensity coefficient.

5. The method according to claim 2, characterized in that After determining the second sub-band time domain signal that is noise among the multiple sub-band time domain signals according to the intensity parameter corresponding to each of the sub-band time domain signals, the method further includes: The intensity threshold is updated based on the intensity parameter of the second sub-band time domain signal to obtain an updated intensity threshold.

6. The method according to claim 1, characterized in that The determining the second time domain signal after noise reduction based on the first sub-band time domain signal and the third sub-band time domain signal includes: Performing noise reduction processing on the first sub-band time domain signal according to a preset second noise reduction coefficient to obtain a fourth sub-band time domain signal; The third sub-band time domain signal and the fourth sub-band time domain signal are combined to obtain the second time domain signal.

7. The method according to claim 6, characterized in that The combining the third sub-band time domain signal and the fourth sub-band time domain signal to obtain the second time domain signal includes: determining an overlapping interval between the third sub-band time domain signal and the fourth sub-band time domain signal; weighting the first signal value of the third sub-band time domain signal in the overlapping interval based on a preset first gain parameter to obtain a first weighted value; and weighting the second signal value of the fourth sub-band time domain signal in the overlapping interval based on a preset second gain parameter to obtain a second weighted value; the sum of the value of the first gain parameter and the value of the second gain parameter being a preset threshold; A target signal value of the second time domain signal is determined according to the first weighted value and the second weighted value.

8. A speech noise reduction device, characterized in that: include: A first acquisition module is used to acquire a first time domain signal including a target sampling point in the speech to be denoised; a segmentation module, configured to perform frequency band segmentation processing on the first time domain signal to obtain a plurality of sub-band time domain signals; each of the sub-band time domain signals includes a signal corresponding to the target sampling point; a first determining module, configured to determine, according to the intensity parameter corresponding to each of the sub-band time domain signals, a first sub-band time domain signal of human voice and a second sub-band time domain signal of noise among the multiple sub-band time domain signals; a first noise reduction module, configured to perform noise reduction processing on the second sub-band time domain signal to obtain a third sub-band time domain signal; The second determining module is configured to determine a second time domain signal of the speech to be denoised after noise reduction based on the first sub-band time domain signal and the third sub-band time domain signal.

9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any one of the methods according to claims 1-7.

Images