A speech enhancement post-processing method and device based on harmonic structure prediction

Abstract

The present invention discloses a speech enhancement post-processing method and device based on harmonic structure prediction, which belongs to the field of information processing. The method comprises the following steps: S1: performing short-time Fourier transform on the speech signal of a microphone to obtain a time-frequency domain expression; S2: performing harmonic loss estimation and correction on the time-frequency domain signal to obtain an estimated power spectrum density; S3: estimating a time-frequency masking value according to the power spectrum density; S4: obtaining a frequency domain estimation of a target speech according to the estimated time-frequency masking value, and then obtaining a time domain estimation of the target speech. The present invention can predict the lost harmonic structure to a certain extent, and the restored speech is more in line with the characteristics of close-speaking speech, and has higher intelligibility and speech perception quality.

Description

Technical Field

[0001] The invention belongs to the field of information processing, and in particular relates to a speech enhancement post-processing method and device based on harmonic structure prediction. Background Art

[0002] In many applications such as voice conferencing systems, background noise can reduce the communication quality of the call system. Using algorithms to suppress the noise signal collected by the microphone is one of the key technologies necessary for conference system-related applications. However, while suppressing noise, the voice signal will also be damaged. Therefore, while suppressing noise, it is also necessary to consider how to enhance the voice signal, especially the harmonic structure of the voice.

[0003] In the prior art, noise suppression and speech enhancement are key technologies for voice communication quality in conference systems or conference equipment. The traditional signal processing method is to track the noise power spectrum density and speech power spectrum density in the signal, and then construct a masking value from 0 to 1 in the frequency domain based on Wiener filtering. After masking the original signal, the purpose of suppressing background noise is achieved. In order to improve the shortcomings of traditional signal processing methods that are ineffective for non-stationary noise, the technology of time-frequency masking estimation based on deep learning is becoming more and more mature and has been applied more and more. The main idea is to estimate the time-frequency masking value directly from the mixed signal by training a noisy data set to a pure speech signal. At present, deep learning is superior to traditional signal processing methods in noise suppression. Regardless of whether it is a traditional signal processing method or a deep learning method, there is a potential risk of speech distortion. Since the energy of the speech signal is mainly distributed in the harmonic structure, how to enhance the speech signal through harmonic structure prediction is of great significance to improving the quality of voice communication.

[0004] At present, the main disadvantages of the methods for estimating time-frequency masking values ​​in the prior art are as follows: 1. The existing time-frequency masking methods tend to ignore the harmonic structure of speech, resulting in partial harmonic loss and affecting communication quality; 2. In long-distance speaking scenarios, since speech is affected by reverberation, high-frequency harmonics will become weaker, and the existing time-frequency masking methods cannot restore the weakened high-frequency harmonics.

[0005] The information disclosed in this background technology section is only intended to enhance the understanding of the overall background of the invention and should not be regarded as an acknowledgment or any form of suggestion that the information constitutes the prior art already known to a person skilled in the art. Summary of the invention

[0006] The purpose of the present invention is to provide a speech enhancement post-processing method and device based on harmonic structure prediction, which can predict the lost harmonic structure to a certain extent, and the restored speech is more in line with the characteristics of close-talking speech, with higher intelligibility and speech perception quality.

[0007] In order to achieve the above object, the present invention provides a speech enhancement post-processing method based on harmonic structure prediction, comprising the following steps:

[0008] S1: Perform short-time Fourier transform on the speech signal of the microphone to obtain the time-frequency domain expression;

[0009] S2: Estimating and correcting the harmonic loss of the time-frequency domain signal to obtain the estimated power spectrum density;

[0010] S3: Estimate the time-frequency masking value based on the power spectrum density;

[0011] S4: Obtain a frequency domain estimation of the target speech according to the estimated time-frequency masking value, and then obtain a time domain estimation of the target speech.

[0012] In one embodiment of the present invention, before step S1, the method further includes: acquiring a voice signal x(n) from a microphone;

[0013] The process of performing short-time Fourier transform on the voice signal x(n) of the microphone in step S1 is as follows:

[0014]

[0015] In one embodiment of the present invention, step S2 specifically includes the following steps:

[0016] S201: Use the time-frequency masking value M(l, k) estimated by deep learning to calculate the masked time domain signal The specific calculation process is as follows:

[0017]

[0018] S202: masked time domain signal After half-wave rectification, Fourier transform is performed to obtain an estimate of harmonic loss The specific calculation formula is as follows:

[0019]

[0020]

[0021] S203: Correct the harmonic loss estimation, and the correction process is as follows:

[0022]

[0023] S204: for each frequency band k, using a uniform smoothing factor α, estimating a power spectrum density; wherein the power spectrum density includes a power spectrum density of background noise, a power spectrum density of speech after time-frequency masking, and a power spectrum density of harmonic loss;

[0024] The estimation process of power spectral density is as follows:

[0025] ρ v (k) = αρ v (k)+(1-α)(1-M(l,k))|X(l,k)| 2

[0026] ρ s (k) = αρ s (k)+(1-α)M(l,k)|X(l,k)| 2

[0027]

[0028] In one embodiment of the present invention, in step S3, for each frequency band k, the time-frequency mask value G(l, k) is estimated, and the estimation process is as follows:

[0029]

[0030] In one embodiment of the present invention, in step S4, the frequency domain estimation of the target speech The acquisition process is as follows:

[0031] y^(″l,k″)=G(l,k)X(l,k),

[0032] After that, the target speech time domain estimation is obtained through inverse Fourier transform, and the process is as follows:

[0033]

[0034] The present invention also provides a speech enhancement post-processing device based on harmonic structure prediction, comprising a signal decomposition module, a loss estimation module, a masking calculation module and a speech synthesis module:

[0035] The signal decomposition module is used to perform short-time Fourier transform on the speech signal of the microphone to obtain the time-frequency domain expression;

[0036] The loss estimation module is used to estimate and correct the harmonic loss of the time-frequency domain signal to obtain an estimated power spectrum density; it includes a masking calculation module, a harmonic loss calculation module, a loss estimation correction module and a power spectrum density calculation module;

[0037] The masking calculation module is used to estimate the time-frequency masking value according to the power spectrum density;

[0038] The speech synthesis module is used to obtain a frequency domain estimation of the target speech according to the estimated time-frequency masking value, and then obtain a time domain estimation of the target speech.

[0039] In one embodiment of the present invention, the signal decomposition module is further used to obtain a voice signal x(n) from a microphone;

[0040] In the signal decomposition module, the process of performing short-time Fourier transform on the voice signal x(n) of the microphone is as follows:

[0041]

[0042] In one embodiment of the present invention, the masking calculation module is used to calculate the masked time domain signal using the time-frequency masking value M(l, k) estimated by deep learning. The specific calculation process is as follows:

[0043]

[0044] The harmonic loss calculation module is used to calculate the masked time domain signal After half-wave rectification, Fourier transform is performed to obtain an estimate of harmonic loss The specific calculation formula is as follows:

[0045]

[0046]

[0047] The loss estimation correction module is used to perform harmonic loss estimation correction, and the correction process is as follows:

[0048]

[0049] The power spectrum density calculation module is used to estimate the power spectrum density for each frequency band k using a uniform smoothing factor α; wherein the power spectrum density includes the power spectrum density of background noise, the power spectrum density of speech after time-frequency masking, and the power spectrum density of harmonic loss.

[0050] The estimation process of power spectral density is as follows:

[0051] ρ v (k) = αρ v (k)+(1-α)(1-M(l,k))|X(l,k)| 2

[0052] ρ s (k) = αρ s (k)+(1-α)M(l,k)|X(l,k)| 2

[0053]

[0054] In one embodiment of the present invention, the masking calculation module estimates the time-frequency masking value G(l, k) for each frequency band k, and the estimation process is as follows:

[0055]

[0056] In one embodiment of the present invention, in the speech synthesis module, the frequency domain estimation of the target speech The acquisition process is as follows:

[0057] y^(″l,k″)=G(l,k)X(l,k),

[0058] The target speech time domain estimation is obtained by inverse Fourier transform, and the process is as follows:

[0059]

[0060] The present invention provides a speech enhancement post-processing method and device based on harmonic structure prediction, which has the following beneficial effects:

[0061] 1. In the present invention, the time-frequency masking information obtained based on deep learning is used to estimate the lost harmonic components, obtain more lost harmonic components, and construct new time-frequency masking information, which can effectively restore the harmonic components and improve the speech perception quality.

[0062] 2. The present invention adopts a novel time-frequency masking estimation, taking into account the specific needs of voice communication, taking into account noise suppression, and restoring the lost harmonics to a certain extent, thereby achieving better communication quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 This is a flowchart of the speech enhancement post-processing method based on harmonic structure prediction in this specific implementation.

[0064] Figure 2 It is a schematic diagram of the Hamming window function used in this specific implementation.

[0065] Figure 3 It is a schematic diagram of a speech enhancement post-processing device based on harmonic structure prediction in this specific implementation manner. DETAILED DESCRIPTION

[0066] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with specific implementation methods.

[0067] like Figure 1 As shown, one embodiment of the present invention is a speech enhancement post-processing method based on harmonic structure prediction.

[0068] The specific implementation steps include the following four steps:

[0069] S1: Perform short-time Fourier transform on the speech signal of the microphone to obtain the time-frequency domain expression.

[0070] The voice signal of the microphone is a digital signal after the sound pressure collected by the microphone passes through the ADC.

[0071] Before step S1, the process also includes acquiring a speech signal from a microphone. The acquired speech signal is as follows: Assume that x(n) represents the original time domain signal picked up by the microphone array element in real time, where n represents a time tag.

[0072] Specifically, the process of performing short-time Fourier transform on the microphone speech signal x(n) to obtain the time-frequency domain expression is as follows:

[0073]

[0074] Where N is the frame length, N=512 is selected; w(n) is a Hamming window of length 512, where n represents the time tag, that is, the time sequence number, so w(n) represents the value at each corresponding time sequence number n; 1 is the time frame sequence number, in frames; k is the frequency band sequence number, where the frequency band refers to the signal component corresponding to a certain frequency; j represents the imaginary unit X(l, k) is the speech signal of the mth microphone, the spectrum of the kth frequency band in the lth frame.

[0075] In the present invention, the Hamming window function used is as follows: Figure 2 shown.

[0076] Through the above step S1, the time domain signal of the speech signal of the microphone can be converted into a time-frequency domain signal for processing in subsequent steps.

[0077] S2: Estimation and correction of harmonic loss of time-frequency domain signals to obtain estimated power spectral density.

[0078] Specifically, the time-frequency masking value estimated by deep learning is used to calculate the masked time domain signal, estimate and correct the harmonic loss, and then estimate the power spectral density.

[0079] Specifically, this step S2 includes the following steps:

[0080] S201: Use the time-frequency masking value M(l, k) estimated by deep learning to calculate the masked time domain signal The specific calculation process is as follows:

[0081]

[0082] Among them, time-frequency masking M(l, k) is a commonly used solution for speech enhancement based on deep learning.

[0083] The time domain signal outputted in step S201 has certain harmonic loss, that is, the energy in some frequency bands of the periodic harmonic frequency bands (fundamental frequency and its multiples) is significantly weakened, but most of the harmonic structure is retained, which can be used to calculate the lost harmonic structure in subsequent steps.

[0084] The difference between step S201 and step S1 is that this inverse Fourier transform does not require a windowing function.

[0085] S202: masked time domain signal After half-wave rectification, Fourier transform is performed to obtain an estimate of harmonic loss The specific calculation formula is as follows:

[0086]

[0087]

[0088] Among them, sign() represents a half-wave rectification operation.

[0089] In step S202, the adjacent harmonic frequency bands can be smoothed by half-wave rectification and then Fourier transformation, and the damaged harmonics can be partially restored. There is a certain underestimation, which will be corrected in the subsequent step S203.

[0090] S203: Performing harmonic loss estimation correction, the correction process is as follows:

[0091]

[0092] The harmonic loss estimated in the previous step S202 has a certain error, and this step S203 can correct the estimation result.

[0093] The principle of the correction method adopted in step S203 is to use a time-frequency masking method, and the masking value M(l, k) is always less than 1. Therefore, if the estimation of harmonic loss The value of is less than its original value Y(l, k), indicating that the harmonics have not been restored and the original estimate is still used as the masking value. The result of step S203 can be used to update the power spectrum density and the time-frequency masking estimate in subsequent steps.

[0094] S204: For each frequency band k, a uniform smoothing factor α is used to estimate the power spectrum density, wherein the power spectrum density includes the power spectrum density of the background noise, the power spectrum density of the speech after time-frequency masking, and the power spectrum density of the harmonic loss.

[0095] The estimation process of power spectral density is as follows:

[0096] ρ v (k) = αρ v (k)+(1-α)(1-M(l,k))|X(l,k)| 2

[0097] ρ s (k) = αρ s (k)+(1-α)M(l,k)|X(l,k)| 2

[0098]

[0099] Among them, ρ v (k),ρ s (k) and ρ h (k) represents the power spectral density of background noise, speech after time-frequency masking, and harmonic loss, respectively. h (k) can be understood as the average result of harmonic loss over a period of time, so ρ h (k) Harmonic recovery will have better robustness. α is a smoothing factor between adjacent frames, and its value range is between 0 and 1. If the value is too small, the power spectrum density estimation will change too much, resulting in instability. If the value is too high, the energy estimation will be too stable, and the ability to model non-stationary signals will be reduced. The preferred embodiment of the present invention is α = 0.95, which can balance stability and the ability to model non-stationary noise.

[0100] The output structure of step S204 can be used in subsequent steps to construct masking values ​​that can restore the harmonic structure.

[0101] S3: Estimate the time-frequency masking value based on the power spectrum density.

[0102] Specifically, for each frequency band k, the time-frequency mask value G(l, k) is estimated, and the estimation process is as follows:

[0103]

[0104] Wherein, max() represents taking the larger value of the two. In step S3, It is used to enhance the harmonic structure, and the calculated time-frequency masking value is used to obtain speech spectrum estimation in subsequent steps.

[0105] S4: Obtain a frequency domain estimation of the target speech according to the estimated time-frequency masking value, and then obtain a time domain estimation of the target speech.

[0106] Among them, the frequency domain estimation of the target speech The acquisition process is as follows:

[0107] y^(″l,k″)=G(l,k)X(l,k)

[0108] After that, the target speech time domain estimation is obtained through inverse Fourier transform, and the process is as follows:

[0109]

[0110] Through step S4, the time domain estimated signal can be directly converted into a voltage signal through digital-to-analog conversion, and the speaker plays the enhanced voice signal.

[0111] Through the above steps S1-S4 of the present invention, signal time-frequency decomposition, harmonic loss estimation, time-frequency masking calculation and target speech synthesis can be achieved, and ultimately the quality of speech communication can be improved.

[0112] like Figure 3 As shown, one embodiment of the present invention is a speech enhancement post-processing device based on harmonic structure prediction, including a signal decomposition module 1, a loss estimation module 2, a masking calculation module 3 and a speech synthesis module 4.

[0113] The signal decomposition module 1 is used to perform short-time Fourier transform on the speech signal of the microphone to obtain a time-frequency domain expression.

[0114] The signal decomposition module 1 can also be used to obtain the voice signal and echo reference signal of the microphone. The obtained voice signal is as follows: Assume that x(n) represents the original time domain signal picked up by the microphone array element in real time, where n represents the time label.

[0115] Specifically, the method for performing short-time Fourier transform is as follows:

[0116] Perform short-time Fourier transform on the time domain signal x(n) to obtain the time-frequency domain expression:

[0117]

[0118] Where N is the frame length, N=512 is selected; w(n) is a Hamming window of length 512, where n represents the time tag, that is, the time sequence number, so w(n) represents the value at each corresponding time sequence number n; 1 is the time frame sequence number, in frames; k is the frequency band sequence number, where the frequency band refers to the signal component corresponding to a certain frequency; j represents the imaginary unit X(l, k) is the speech signal of the mth microphone, the spectrum of the kth frequency band in the first frame.

[0119] In the present invention, the Hamming window function used is as follows: Figure 2 shown.

[0120] The signal decomposition module 1 can convert the time domain signal of the microphone's speech signal into a time-frequency domain signal.

[0121] The loss estimation module 2 is used to estimate the harmonic loss of the time-frequency domain signal, estimate the harmonic components of the loss in the time-frequency domain signal, and obtain the estimated power spectrum density. The loss estimation module 2 includes a masking calculation module, a harmonic loss calculation module, a loss estimation correction module, and a power spectrum density calculation module.

[0122] Specifically, the masking calculation module is used to use the time-frequency masking value M(l, k) estimated by deep learning to calculate the masked time domain signal The specific calculation process is as follows:

[0123]

[0124] The time domain signal output by the masking calculation module has a certain harmonic loss, that is, the energy in some frequency bands of the periodic harmonic frequency bands (fundamental frequency and its multiples) is significantly weakened, but most of the harmonic structure is retained, which can be used for subsequent calculation of the lost harmonic structure.

[0125] The difference between the mask calculation module and the signal decomposition module 1 is that the inverse Fourier transform does not require a windowing function.

[0126] Harmonic loss calculation module, for masked time domain signal After half-wave rectification, Fourier transform is performed to obtain an estimate of harmonic loss The specific calculation process is as follows:

[0127]

[0128]

[0129] Among them, sign() represents a half-wave rectification operation.

[0130] The harmonic loss calculation module can smooth adjacent harmonic frequency bands by half-wave rectification and then Fourier transform, and can partially restore damaged harmonics. There is a certain underestimation in the estimation of harmonic loss, which will be corrected later.

[0131] The loss estimation correction module is used to perform harmonic loss estimation correction. The correction process is as follows:

[0132]

[0133] The harmonic loss estimated in the harmonic loss calculation module has certain errors, and the loss estimation correction module can correct the estimation result.

[0134] The principle of the correction method adopted by the loss estimation correction module is to use the time-frequency masking method, and the masking value M(l, k) is always less than 1. Therefore, if the estimate of harmonic loss The value of is less than its original value X(l, k), indicating that the harmonics have not been restored and the original estimate is still used as the masking value. The result of the loss estimation correction module can be used to subsequently update the power spectrum density and update the time-frequency masking estimate.

[0135] The power spectrum density calculation module is used to estimate the power spectrum density for each frequency band k using a uniform smoothing factor α, wherein the power spectrum density includes the power spectrum density of background noise, the power spectrum density of speech after time-frequency masking, and the power spectrum density of harmonic loss.

[0136] The estimation process of power spectral density is as follows:

[0137] ρ v (k) = αρ v (k)+(1-α)(1-M(l,k))|X(l,k)| 2

[0138] ρ s (k) = αρ s (k)+(1-α)M(l,k)|X(l,k)| 2

[0139]

[0140] Among them, ρ v (k),ρ s (k) and ρ h (k) represents the power spectral density of background noise, speech after time-frequency masking, and harmonic loss, respectively. h (k) can be understood as the average result of harmonic loss over a period of time, so ρ h (k) Harmonic recovery will have better robustness. α is a smoothing factor between adjacent frames, and its value range is between 0 and 1. The present invention preferably sets α=0.95. If the value is too small, the power spectrum density estimation will change too much, resulting in an unstable defect. If the value is too high, the energy estimation will be too stable, and the ability to model non-stationary signals will be reduced.

[0141] The separation matrix calculation module 3 is used to estimate the time-frequency masking value according to the power spectrum density.

[0142] Specifically, for each frequency band k, the time-frequency masking value G(l, k) is estimated, and the estimation process is as follows:

[0143]

[0144] Wherein, max() represents taking the larger value of the two. In the separation matrix calculation module 3, It is used to enhance the harmonic structure, and the calculated time-frequency masking value is used to obtain speech spectrum estimation in subsequent steps.

[0145] The speech synthesis module 4 is used to obtain a frequency domain estimation of the target speech according to the estimated time-frequency masking value, and then obtain a time domain estimation of the target speech.

[0146] Among them, the frequency domain estimation of the target speech The acquisition process is as follows:

[0147] y^(″l,k″)=G(l,k)X(l,k)

[0148] After that, the target speech time domain estimation is obtained through inverse Fourier transform, and the process is as follows:

[0149]

[0150] Through the speech synthesis module 4, the time domain estimated signal can be directly converted into a voltage signal through digital-to-analog conversion, and the enhanced speech signal can be played by the speaker.

[0151] The speech enhancement post-processing device based on harmonic structure prediction of the present invention can realize signal time-frequency decomposition, harmonic loss estimation, time-frequency masking calculation and target speech synthesis, and finally improve the quality of speech communication.

[0152] The foregoing description of specific exemplary embodiments of the present invention is for the purpose of illustration and demonstration. These descriptions are not intended to limit the present invention to the precise form disclosed, and it is clear that many changes and variations can be made based on the above teachings. The purpose of selecting and describing the exemplary embodiments is to explain the specific principles of the present invention and its practical application, so that those skilled in the art can realize and utilize various different exemplary embodiments of the present invention and various different selections and changes. The scope of the present invention is intended to be limited by the claims and their equivalents.

Claims

1. A speech enhancement post-processing method based on harmonic structure prediction, characterized in that: The following steps are involved: S1: Perform short-time Fourier transform on the speech signal of the microphone to obtain the time-frequency domain expression; S2: Estimating and correcting the harmonic loss of the time-frequency domain signal to obtain the estimated power spectrum density; S3: Estimate the time-frequency masking value based on the power spectrum density; S4: obtaining a frequency domain estimation of the target speech according to the estimated time-frequency masking value, and then obtaining a time domain estimation of the target speech; Before step S1, the method further includes: obtaining a voice signal x(n) from a microphone; The process of performing short-time Fourier transform on the voice signal x(n) of the microphone in step S1 is as follows: Wherein, N is the frame length, N=512; w(n) is the Hamming window function of length 512, n is the time label; l is the time frame number; k is the frequency band number; j represents the imaginary unit; X(l, k) is the spectrum of the kth frequency band in the lth frame and the mth microphone signal; The step S2 specifically includes the following steps: S201: Use the first time-frequency masking value M(l, k) estimated by deep learning to calculate the masked time domain signal The specific calculation process is as follows: S202: masked time domain signal After half-wave rectification, Fourier transform is performed to obtain an estimate of harmonic loss The specific calculation formula is as follows: Among them, sign() represents the half-wave rectification operation; S203: Correct the estimation of harmonic loss. The correction process is as follows: S204: for each frequency band k, using a uniform smoothing factor α, estimating a power spectrum density; wherein the power spectrum density includes a power spectrum density of background noise, a power spectrum density of speech after time-frequency masking, and a power spectrum density of harmonic loss; The estimation process of power spectral density is as follows: ρ v (k)=αρ v (k)+(1-α)(1-M(l,k))|X(l,k)| 2 ρ s (k)=αρ s (k)+(1-α)M(l,k)|X(l,k)| 2 Among them, ρ v (k),ρ s (k) and ρ h (k) The power spectral density of background noise, speech after time-frequency masking, and harmonic loss, respectively; In step S3, for each frequency band k, the second time-frequency mask value G(l, k) is estimated, and the estimation process is as follows: Among them, max() represents taking the larger value of the two.

2. The speech enhancement post-processing method based on harmonic structure prediction according to claim 1, characterized in that: In step S4, the frequency domain estimation of the target speech The acquisition process is as follows: After that, the target speech time domain estimation is obtained through inverse Fourier transform, and the process is as follows:

3. A speech enhancement post-processing device based on harmonic structure prediction, characterized in that: Including signal decomposition module, loss estimation module, masking calculation module and speech synthesis module: The signal decomposition module is used to perform short-time Fourier transform on the speech signal of the microphone to obtain the time-frequency domain expression; The loss estimation module is used to estimate and correct the harmonic loss of the time-frequency domain signal to obtain an estimated power spectrum density; it includes a masking calculation module, a harmonic loss calculation module, a loss estimation correction module and a power spectrum density calculation module; The masking calculation module is used to estimate the time-frequency masking value according to the power spectrum density; The speech synthesis module is used to obtain a frequency domain estimation of the target speech according to the estimated time-frequency masking value, and then obtain a time domain estimation of the target speech; The signal decomposition module is also used to obtain a voice signal x(n) from a microphone; In the signal decomposition module, the process of performing short-time Fourier transform on the voice signal x(n) of the microphone is as follows: Wherein, N is the frame length, N=512; w(n) is the Hamming window function of length 512, n is the time label; l is the time frame number; k is the frequency band number; j represents the imaginary unit; X(l, k) is the spectrum of the kth frequency band in the lth frame and the mth microphone signal; The masking calculation module is used to calculate the masked time domain signal using the first time-frequency masking value M(l, k) estimated by deep learning The specific calculation process is as follows: The harmonic loss calculation module is used to calculate the masked time domain signal After half-wave rectification, Fourier transform is performed to obtain an estimate of harmonic loss The specific calculation formula is as follows: Among them, sign() represents the half-wave rectification operation; The loss estimation correction module is used to correct the estimation of harmonic loss, and the correction process is as follows: The power spectrum density calculation module is used to estimate the power spectrum density for each frequency band k using a uniform smoothing factor α; wherein the power spectrum density includes the power spectrum density of background noise, the power spectrum density of speech after time-frequency masking, and the power spectrum density of harmonic loss. The estimation process of power spectral density is as follows: ρ v (k)=αρ v (k)+(1-α)(1-M(l,k))|X(l,k)| 2 ρ s (k)=αρ s (k)+(1-α)M(l,k)|X(l,k)| 2 Among them, ρ v (k),ρ s (k) and ρ h (k) The power spectral density of background noise, speech after time-frequency masking, and harmonic loss, respectively; The mask calculation module estimates the second time-frequency mask value G(l, k) for each frequency band k, and the estimation process is as follows: Among them, max() represents taking the larger value of the two.

4. The speech enhancement post-processing device based on harmonic structure prediction according to claim 3, characterized in that: In the speech synthesis module, the frequency domain estimation of the target speech The acquisition process is as follows: The target speech time domain estimation is obtained by inverse Fourier transform, and the process is as follows:

Images