Speech enhancement method and apparatus, electronic device and storage medium

Abstract

A voice enhancement method and apparatus, an electronic device, and a storage medium, belonging to the technical field of audio noise reduction. The method comprises: extracting first excitation information and first envelope information of a first frequency domain signal corresponding to a first speech signal, wherein the first speech signal comprises a noise signal and a first clean speech signal (201); performing optimal excitation codebook matching on the first excitation information, so as to obtain second excitation information (202); performing optimal envelope codebook matching on the first envelope information, so as to obtain second envelope information (203); on the basis of the first excitation information, the second excitation information and the second envelope information, synthesizing target spectrum information (204); and according to the target spectrum information, performing speech enhancement processing on the first speech signal (205).

Description

Voice enhancement method, device, electronic device and storage medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent application No. 202310833021.0 filed in China on July 6, 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present application belongs to the technical field of audio noise reduction, and specifically relates to a speech enhancement method, device, electronic device and storage medium. Background Art

[0004] Currently, when electronic devices use Wiener filtering and statistical models to reduce the noise of speech signals, they can use a priori signal-to-noise ratio (SNR) to reduce the noise and produce a noise-free speech signal. Generally, an accurate SNR requires an accurate power spectrum of the clean speech signal. However, in reality, electronic devices only produce noisy speech signals.

[0005] Therefore, in related technologies, electronic devices can estimate the a priori signal-to-noise ratio through a decision-directed method, that is, the electronic device can estimate the a priori signal-to-noise ratio through the a posteriori signal-to-noise ratio.

[0006] However, in the process of obtaining the posterior signal-to-noise ratio, if the power of the clean speech signal is close to the power of the noise signal, the electronic device cannot obtain an accurate posterior signal-to-noise ratio, resulting in a large error in the electronic device estimating the prior signal-to-noise ratio through the posterior signal-to-noise ratio. As a result, the electronic device has poor accuracy in determining the prior signal-to-noise ratio, which in turn leads to poor noise reduction effect on the speech signal.

[0007] Summary of the Invention

[0008] The purpose of the embodiments of the present application is to provide a speech enhancement method, device, electronic device and storage medium, which can improve the noise reduction effect of electronic devices when performing speech noise reduction.

[0009] In a first aspect, an embodiment of the present application provides a speech enhancement method, which spectrum information determination method includes: extracting first excitation information and first envelope information of a first frequency domain signal corresponding to a first speech signal, wherein the first speech signal includes a noise signal and a first clean speech signal; performing optimal excitation codebook matching on the first excitation information to obtain second excitation information; performing optimal envelope codebook matching on the first envelope information to obtain second envelope information; synthesizing target spectrum information based on the first excitation information, the second excitation information and the second envelope information; and performing speech enhancement processing on the first speech signal according to the target spectrum information.

[0010] In a second aspect, an embodiment of the present application provides a speech enhancement device, which includes: an extraction module, a matching module, a synthesis module, and a processing module. The extraction module is used to extract first excitation information and first envelope information of a first frequency domain signal corresponding to a first speech signal, wherein the first speech signal includes a noise signal and a first clean speech signal. The matching module is used to perform optimal excitation codebook matching on the first excitation information extracted by the extraction module to obtain second excitation information; and perform optimal envelope codebook matching on the first envelope information extracted by the extraction module to obtain second envelope information. The synthesis module is used to synthesize target spectrum information based on the first excitation information, the second excitation information, and the second envelope information. The processing module is used to perform speech enhancement processing on the first speech signal according to the target spectrum information.

[0011] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0012] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0013] In a fifth aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the method described in the first aspect.

[0014] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the method described in the first aspect.

[0015] In an embodiment of the present application, an electronic device can extract first excitation information and first envelope information from a first frequency domain signal corresponding to a first voice signal, wherein the first voice signal includes a noise signal and a first clean voice signal; thereby, the electronic device can perform optimal excitation codebook matching on the first excitation information to obtain second excitation information, and perform optimal envelope codebook matching on the first envelope information to obtain second envelope information; further, the electronic device can synthesize target spectrum information based on the first excitation information, the second excitation information, and the second envelope information, and then the electronic device can perform speech enhancement processing on the first voice signal according to the target spectrum information. In this solution, the electronic device can perform optimal excitation codebook matching on the first excitation information and perform optimal envelope codebook matching on the first envelope information, so that the electronic device can obtain the second excitation information and the second envelope information corresponding to the first clean voice signal, and then can obtain the target spectrum information through the first excitation information, the second excitation information, and the second envelope information; so that the electronic device can obtain an accurate priori signal-to-noise ratio corresponding to the first clean voice signal based on the target spectrum information, and then the electronic device can perform speech enhancement processing based on the accurate priori signal-to-noise ratio, thereby improving the noise reduction effect of the electronic device in performing speech noise reduction. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] FIG1 is a flow chart of a method for speech enhancement according to an embodiment of the present application;

[0017] FIG2 is a second flow chart of a speech enhancement method provided in an embodiment of the present application;

[0018] FIG3 is a third flow chart of a speech enhancement method provided in an embodiment of the present application;

[0019] FIG4 is one of the structural diagrams of a deep neural network provided in an embodiment of the present application;

[0020] FIG5 is a second structural diagram of a deep neural network provided in an embodiment of the present application;

[0021] FIG6 is a third structural diagram of a deep neural network provided in an embodiment of the present application;

[0022] FIG7 is a schematic structural diagram of a speech enhancement device provided in an embodiment of the present application;

[0023] FIG8 is a schematic diagram of a hardware structure of an electronic device provided in an embodiment of the present application;

[0024] FIG9 is a second schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the accompanying drawings in the embodiments of this application to clearly describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0026] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0027] The following, in conjunction with the accompanying drawings, describes in detail the speech enhancement method, device, electronic device, and storage medium provided in the embodiments of the present application through specific embodiments and their application scenarios.

[0028] The speech enhancement method, device, electronic device, and storage medium provided in the embodiments of the present application can be applied to voice call scenarios in which calls are made using mobile electronic devices such as speakers, conference terminals, and headphones.

[0029] Currently, when a user makes a voice call through an electronic device, the electronic device may perform noise reduction processing on the voice signal in the voice call in order to ensure the quality of the voice call, so that the electronic device can receive a voice signal with better call quality.

[0030] Typically, when electronic devices perform speech noise reduction, they can use Wiener filtering and statistical models, hereinafter referred to as traditional noise reduction methods, to reduce the noise of noisy speech signals. These methods are widely used due to their simplicity, effectiveness, ease of real-time calculation, and low engineering computational complexity.

[0031] Specifically, assume that the noisy speech signal received by the electronic device through the microphone is: x(t) = s(t) + n(t)

[0032] Where s(t) is an unknown clean speech signal and n(t) is a random noise signal. The electronic device can input the noisy speech signal into a frame-based windowing and Fast Fourier Transform (FFT) algorithm. Thus, the electronic device can obtain a frequency domain signal corresponding to the noisy speech signal. The frequency domain signal is: X(f,k) = FFT[x(t)] = s(f,k) + n(f,k)

[0033] Here, k represents the frame number and f represents the frequency.

[0034] After the electronic device obtains the frequency domain signal corresponding to the noisy speech signal, the electronic device can define the posterior signal-to-noise ratio: And define the prior signal-to-noise ratio: Among them, P nn (f,k) is the estimated value of the noise spectrum corresponding to the noisy speech signal, P xx (f,k) is the spectrum of the noisy speech signal (known), P ss (f,k) is the clean speech power spectrum (unknown).

[0035] In general, the noise power spectrum estimation method is usually designed as follows: the noisy speech signal is represented as a complex form of amplitude and phase, X(f,k) = r·exp(j·θ).

[0036] The spectrum of the noise signal is expressed as a complex form of amplitude and phase, N(f,k) = n·exp(j·δ).

[0037] Given a noisy speech signal X(f,k) observation, according to the Bayesian formula, the minimum mean square error (MMSE) estimate of the noise power spectrum can be expected by the following formula 1:

[0038] Where f(X|n,δ) is the conditional probability of the clean speech signal, and f(n,δ) is the conditional probability of the noisy signal.

[0039] Assuming that the clean speech signal and the noise signal obey independent complex Gaussian distributions, the conditional probabilities in the numerator and denominator can be expressed by the following formula 2 and formula 3:

[0040] The electronic device then converts the noise spectrum estimate into a priori signal-to-noise ratio form through an iterative update method, which can be specifically achieved by the following formula 4:

[0041] After obtaining the noise power spectrum, the electronic device can use it to update the prior signal-to-noise ratio of the current frame, thereby further obtaining the noise reduction gain G based on the statistical model. It is commonly used in the following two forms:

[0042] Method 1: Noise reduction gain G in the form of Wiener filtering winner (f,k), the noise reduction gain Where ξ(f,k) is the prior signal-to-noise ratio.

[0043] Method 2: Noise reduction gain in the form of logarithmic magnitude spectrum MMSE estimation, the noise reduction gain

[0044] in,

[0045] The electronic device multiplies the noise reduction gain by the frequency domain signal corresponding to the noisy speech signal, and then performs an inverse Fourier transform (IFFT) to obtain the noise-reduced speech signal. This can be specifically achieved using the following formula 5:

[0046] Generally, electronic devices can use the aforementioned traditional noise reduction methods to reduce the noise of speech signals to obtain a noise-free speech signal. Specifically, the aforementioned traditional noise reduction methods demonstrate that the prior signal-to-noise ratio ξ(f,k) plays a key role in estimating both the noise power spectrum and the noise reduction gain. In other words, the prior signal-to-noise ratio determines the effectiveness of noise reduction. To obtain an accurate priori SNR, it must be based on the power spectrum of the clean speech signal. However, in actual scenarios, electronic devices generally only receive noisy speech signals. Therefore, in order to obtain an accurate priori SNR, electronic devices can obtain the priori SNR through a decision-directed approach. The idea is to first calculate the posterior SNR, use the posterior SNR as an estimate of the prior SNR, and then calculate the prior SNR of the current audio frame by smoothing it with the prior SNR of the previous audio frame. This can be achieved using the following formula 6: ξ(f,k)=α*ξ(f,k-1)+(1-α)*max(0,γ(f,k)-1) (Formula 6)

[0047] Among them, γ(f,k)-1 is the posterior signal-to-noise ratio, ξ(f,k-1) is the prior signal-to-noise ratio of the previous frame, and α is the smoothing coefficient.

[0048] It should be noted that the above-mentioned previous frame and current frame are obtained by the electronic device performing frame dividing and windowing on the noisy speech signal when the electronic device converts the noisy speech signal into a frequency domain signal. For example, the electronic device can divide the noisy speech signal into 10 audio segments, and any audio segment of the 10 audio segments is a frame audio segment.

[0049] However, in the above method, the measurement of the posterior signal-to-noise ratio is accurate under high signal-to-noise ratio conditions (speech power is greater than noise power). As the signal-to-noise ratio gradually decreases (speech power approaches noise power), the error in the signal-to-noise ratio estimation increases. When the speech power is less than the noise power, the posterior signal-to-noise ratio is 0, and the electronic equipment no longer has the ability to measure. Moreover, due to the existence of the smoothing coefficient α, slow iterative updates help stabilize the prior signal-to-noise ratio estimation, but will introduce delays, while fast iterative updates will cause the prior signal-to-noise ratio estimation to be unstable, resulting in jitter in the noise reduction gain and causing "musical noise" in the subjective listening experience. The above defects result in traditional noise reduction methods being able to obtain accurate noise reduction gain only in high signal-to-noise ratio scenarios. In low signal-to-noise ratio scenarios, the error in the signal-to-noise ratio estimation will cause problems such as speech damage and insufficient noise reduction gain, making it difficult to meet application requirements.

[0050] In the speech enhancement method, device, electronic device and storage medium provided in the embodiment of the present application, the electronic device can perform optimal excitation codebook matching on the first excitation information and optimal envelope codebook matching on the first envelope information, so that the electronic device can obtain the second excitation information and the second envelope information corresponding to the first clean speech signal, and then can obtain the target spectrum information through the first excitation information, the second excitation information and the second envelope information; so that the electronic device can obtain the accurate priori signal-to-noise ratio corresponding to the first clean speech signal based on the target spectrum information, and then the electronic device can perform speech enhancement processing based on the accurate priori signal-to-noise ratio, thereby improving the noise reduction effect of the electronic device for speech noise reduction. It can be understood that since the embodiment of the present application does not involve the posterior signal-to-noise ratio, whether the electronic device is in a high signal-to-noise ratio scenario or in a low signal-to-noise ratio scenario, the electronic device can obtain a relatively accurate priori signal-to-noise ratio based on the target spectrum information of the clean speech signal; in this way, the electronic device can obtain an accurate noise reduction gain based on the accurate priori signal-to-noise ratio, so as to obtain a speech signal with better noise reduction effect.

[0051] The execution subject of the speech enhancement method provided in the embodiment of the present application may be a speech enhancement device, which may be an electronic device or a functional module in an electronic device. The technical solution provided in the embodiment of the present application is described below using an electronic device as an example.

[0052] The present invention provides a method for speech enhancement, and Figure 1 shows a flowchart of the method for speech enhancement provided by the present invention. As shown in Figure 1, the method for speech enhancement provided by the present invention may include the following steps 201 to 205.

[0053] Step 201: The electronic device extracts first excitation information and first envelope information of a first frequency domain signal corresponding to a first speech signal.

[0054] In the embodiment of the present application, the first speech signal includes a noise signal and a first clean speech signal.

[0055] For example, the first voice signal may be a voice signal collected by an electronic device through a microphone. Generally, such a temporally continuous signal is usually a time domain signal.

[0056] Optionally, in an embodiment of the present application, the first frequency domain signal may be the entire frequency domain signal corresponding to the first voice signal, or may be a partial frequency domain signal corresponding to the first voice signal.

[0057] In the embodiment of the present application, after obtaining the first voice signal, the electronic device may obtain a first frequency domain signal corresponding to the first voice signal through a short-time Fourier transform (STFT) algorithm.

[0058] Illustratively, after the electronic device obtains the first speech signal, the electronic device may perform framing processing, windowing processing, and FFT transformation, namely the above-mentioned STFT, on the first speech signal, thereby obtaining a frequency domain representation of the first speech signal.

[0059] For example, assuming that the sampling frequency of the first speech signal is 16 kHz, the frame length is set to 32 ms, the overlap is 8 ms, the window function is set to a Hanning window, and the FFT length is 512 points, the electronic device can obtain a frequency domain representation of the first speech signal using the STFT algorithm by setting the above parameters.

[0060] As another example, after obtaining the complete first voice signal, the electronic device can obtain the first frequency domain signal corresponding to the complete first voice signal through the above-mentioned STFT algorithm; or, in the process of receiving the first voice signal, the electronic device can obtain the partial first frequency domain signal corresponding to the partial first voice signal in real time through the above-mentioned STFT algorithm, so that when the electronic device finishes receiving the first voice information, the electronic device can add the partial first frequency domain signals corresponding to each partial first voice signal to obtain the first frequency domain signal.

[0061] In some embodiments of the present application, the first frequency domain signal includes frequency domain signals corresponding to L audio frames, where L is a positive integer.

[0062] In some embodiments of the present application, when converting a time-domain signal into a frequency-domain signal using the STFT algorithm, the electronic device segments the time-domain signal to obtain multiple audio segments of the time-domain signal. This is done by framing the time-domain signal. Subsequently, a windowing process and an FFT transform are performed to obtain the frequency-domain signal corresponding to each audio segment.

[0063] Optionally, in an embodiment of the present application, when converting the first voice signal into the first frequency domain signal, the electronic device may perform frame processing on the first voice signal to obtain L audio frames corresponding to the L audio segments of the first voice signal. Then, the frequency domain signal corresponding to each audio frame is obtained through windowing and FFT transformation. Finally, the frequency domain signals corresponding to the L audio frames are superimposed to obtain the first frequency domain signal.

[0064] Step 202: The electronic device performs optimal excitation codebook matching on the first excitation information to obtain second excitation information.

[0065] In the embodiment of the present application, the second excitation information is excitation information corresponding to the clean speech signal in the first speech signal.

[0066] In the embodiment of the present application, the electronic device can perform optimal excitation codebook matching on the first excitation information through a network model to obtain the second excitation information.

[0067] Optionally, in an embodiment of the present application, the electronic device may obtain second excitation information corresponding to the clean speech signal through a first algorithm.

[0068] Exemplarily, the first algorithm may be a peak detection algorithm.

[0069] It is understood that the electronic device can obtain the second excitation information through the excitation code book; or the electronic device can also obtain the second excitation information without the excitation code book. The specific details can be determined according to actual usage and are not limited in the embodiments of the present application.

[0070] Step 203: The electronic device performs optimal envelope codebook matching on the first envelope information to obtain second envelope information.

[0071] In the embodiment of the present application, the second envelope information is the envelope information corresponding to the clean speech signal in the first speech signal.

[0072] In the embodiment of the present application, the electronic device may perform optimal envelope codebook matching on the first envelope information through a network model to obtain the second envelope information.

[0073] Optionally, in an embodiment of the present application, the electronic device may obtain second excitation information corresponding to the clean speech signal through a second algorithm.

[0074] Exemplarily, the second algorithm may be a vector distance algorithm.

[0075] It is understood that the electronic device can obtain the second envelope information through the envelope codebook; or the electronic device can also obtain the second envelope information without the envelope codebook. The specific method can be determined according to actual usage and is not limited in the embodiments of the present application.

[0076] Step 204: The electronic device synthesizes target spectrum information based on the first excitation information, the second excitation information, and the second envelope information.

[0077] In some embodiments of the present application, the target spectrum information is used to generate a target clean speech signal.

[0078] In some embodiments of the present application, the electronic device may synthesize target spectrum information by linear prediction synthesis.

[0079] Optionally, in some embodiments of the present application, the second envelope information includes: gain information of the first vibration frequency corresponding to the first frequency domain signal; the second excitation information includes: frequency information of the first vibration frequency; the first excitation information includes: frequency information of the M vibration frequencies, the M vibration frequencies include the first vibration frequency, and M is a positive integer.

[0080] Exemplarily, in combination with FIG1 , as shown in FIG2 , the above step 203 may be implemented specifically through the following steps 203a and 203b.

[0081] Step 203a: The electronic device calculates target gain information based on the first excitation information and the second excitation information.

[0082] In some embodiments of the present application, the electronic device may transform the first excitation information and the second excitation information from the cepstrum domain to the power spectrum domain, and then obtain the target gain information by taking the difference.

[0083] For example, the electronic device may obtain the target gain information by using the following formula 7, which may be: EG res (f,k)=FFT[E s ]-FFT[E x ] (Formula 7)

[0084] Among them, EG res (f,k) is the target gain information, E s is the second stimulus information, E x This is the first motivational information.

[0085] Step 203b: The electronic device synthesizes target spectrum information based on the second envelope information, the target gain information, and the residual signal.

[0086] In some embodiments of the present application, the residual signal is obtained after the electronic device performs linear filtering on the first frequency domain signal.

[0087] In some embodiments of the present application, the electronic device may restore the second envelope information to an LPC linear prediction coefficient, and then synthesize the target spectrum information through the LPC linear prediction coefficient, the target gain information, and the residual signal.

[0088] For example, the electronic device may restore the second envelope information to the LPC linear prediction coefficient by using the following formula 8. Specifically, formula 8 is: lpc_α2(n)=LSFtoLPC(A s ) (Formula 8)

[0089] Among them, lpc_α2(n) is the LPC linear prediction coefficient, A s is the second envelope information.

[0090] As another example, the electronic device may synthesize the target spectrum information using the LPC linear prediction coefficient, the target gain information, and the residual signal using the following formula 9. Specifically, formula 9 is:

[0091] in, is the target spectrum information, X res (f,k) is the residual signal, EG res (f,k) is the target gain information, and lpc_α2(n) is the LPC linear prediction coefficient.

[0092] In an embodiment of the present application, the electronic device can obtain target gain information using the first excitation information and the second excitation information, and then synthesize target spectrum information using the second envelope information, the target gain information, and the residual signal. It will be understood that since the second excitation information and the second envelope information are both information corresponding to a clean speech signal, the target spectrum information obtained by the electronic device is also target spectrum information corresponding to a clean speech signal. Furthermore, the electronic device can perform speech enhancement processing on the first speech signal based on the target spectrum information, thereby obtaining a speech signal with a better noise reduction effect.

[0093] Step 205: The electronic device performs speech enhancement processing on the first speech signal according to the target spectrum information.

[0094] In an embodiment of the present application, the electronic device can calculate a priori signal-to-noise ratio based on the target spectrum information and the noisy speech spectrum information, estimate the noise power spectrum through the priori signal-to-noise ratio, and then obtain the noise reduction gain through the noise power spectrum.

[0095] It can be understood that the above-mentioned noise reduction gain is used to perform speech enhancement processing on the first speech signal.

[0096] For example, the target spectrum information is obtained in the electronic device And, after the noisy speech spectrum information X(f,k), the electronic device can calculate the power spectrum information of the target spectrum information respectively and the power spectrum information P of the noisy speech spectrum information xx (f,k)=|X(f,k) 2 |, and calculate the prior signal-to-noise ratio, which is ξ(f,k)=P ss (f,k) / P nn (f,k)

[0097] The electronic device can use the statistical model in the traditional noise reduction method to estimate the noise power spectrum, which can be specifically achieved by the following formula 10:

[0098] Among them, P nn (f,k) is the noise power spectrum, ξ(f,k-1) is the prior signal-to-noise ratio of the previous audio frame, P nn (f,k-1) is the noise power spectrum of the previous audio frame, P xx (f,k) is the power spectrum information of the noisy speech spectrum information.

[0099] Then, the electronic device calculates the noise reduction gain G(f,k) through MMSE estimation, which can be specifically implemented by the following formula 11:

[0100] in,

[0101] Finally, the electronic device can multiply the noise reduction gain by the noise reduction signal spectrum to obtain the signal spectrum after noise reduction processing. This can be achieved by the following formula 12: S(f,k) = X(f,k)·G(f,k) (Formula 12)

[0102] Where S(f,k) is the signal spectrum after noise reduction processing, G(f,k) is the noise reduction gain, and X(f,k) is the spectrum of the noisy speech signal.

[0103] Optionally, in some embodiments of the present application, after obtaining the signal spectrum after noise reduction processing, the electronic device can perform Fourier transform on the signal spectrum after noise reduction processing to obtain the time domain signal after noise reduction processing, that is, the above-mentioned target clean speech signal.

[0104] In the speech enhancement method provided in an embodiment of the present application, an electronic device can extract first excitation information and first envelope information from a first frequency domain signal corresponding to a first speech signal, wherein the first speech signal includes a noise signal and a first clean speech signal; thereby, the electronic device can perform optimal excitation codebook matching on the first excitation information to obtain second excitation information, and perform optimal envelope codebook matching on the first envelope information to obtain second envelope information; further, the electronic device can synthesize target spectrum information based on the first excitation information, the second excitation information, and the second envelope information, and then the electronic device can perform speech enhancement processing on the first speech signal according to the target spectrum information. In this solution, the electronic device can perform optimal excitation codebook matching on the first excitation information and perform optimal envelope codebook matching on the first envelope information, so that the electronic device can obtain the second excitation information and the second envelope information corresponding to the first clean speech signal, and then can obtain the target spectrum information through the first excitation information, the second excitation information, and the second envelope information; so that the electronic device can obtain an accurate priori signal-to-noise ratio corresponding to the first clean speech signal based on the target spectrum information, and then the electronic device can perform speech enhancement processing based on the accurate priori signal-to-noise ratio, thereby improving the noise reduction effect of the electronic device in performing speech noise reduction.

[0105] Optionally, in an embodiment of the present application, in combination with FIG1 , as shown in FIG3 , the above step 202 may be specifically implemented through the following step 202a; or, the above step 203 may be specifically implemented through the following step 203c.

[0106] Step 202a: The electronic device determines second excitation information that matches the first excitation information from a preset excitation codebook.

[0107] In the embodiment of the present application, the preset excitation codebook is obtained by training the excitation information set of the clean speech signal to obtain NC representative clean speech excitation information subsets.

[0108] It can be understood that the above preset excitation codebook includes excitation information of NC clean speech signals.

[0109] In an embodiment of the present application, the electronic device may process each clean speech signal in a clean speech data set to obtain a preset excitation codebook, wherein the clean speech data set includes at least N clean speech signals, where N is a positive integer.

[0110] In the embodiment of the present application, the preset excitation codebook includes multiple excitation codebooks, and each excitation codebook corresponds to a pitch period of a clean speech signal.

[0111] Optionally, in the embodiment of the present application, the speech enhancement method provided in the embodiment of the present application further includes the following step 301, and the above-mentioned step 202a can be specifically implemented through the following steps 202a1 and 202a2.

[0112] Step 301: The electronic device performs linear prediction analysis on the first frequency domain signal to obtain first excitation information.

[0113] In an embodiment of the present application, the electronic device may perform linear prediction analysis on the first frequency domain signal to obtain a residual signal, where the residual signal corresponds to the first excitation information.

[0114] Optionally, in some embodiments of the present application, before extracting the first excitation information and first envelope information from the first frequency domain signal corresponding to the first speech signal, the electronic device may perform pre-noise reduction processing on the first frequency domain signal, thereby obtaining the first excitation information and first envelope information based on the pre-noise reduction processed first frequency domain signal. In this way, the electronic device can extract the first excitation information and first envelope information from the pre-noise reduction processed first frequency domain signal, thereby obtaining excitation information and envelope information that are closer to a clean speech signal.

[0115] For example, taking the first audio frame among the L audio frames corresponding to the above-mentioned first frequency domain signal as an example, when the electronic device performs pre-noise reduction processing on the frequency domain signal corresponding to the first audio frame, it can perform pre-noise reduction processing on the frequency domain signal corresponding to the first audio frame based on the noise reduction gain of the second audio frame.

[0116] In this way, the electronic device can obtain excitation information and envelope information that are closer to a clean frequency domain signal based on the frequency domain signal corresponding to the first audio frame after pre-noise reduction processing. It should be noted that the second audio frame is the preceding adjacent audio frame of the first audio frame.

[0117] For example, assuming that the frequency domain corresponding to the first audio frame is represented by X(f, k), and the noise reduction gain of the second audio frame is G(f, k-1), the electronic device can perform pre-noise reduction processing on the first audio frame according to the following formula 13. The specific formula is: X c (f,k) = X(f,k)·G(f,k-1) (Formula 13)

[0118] Among them, X c (f, k) is the frequency domain representation of the first audio frame after noise reduction, X(f, k) is the frequency domain representation of the first audio frame without noise reduction, and G(f, k-1) is the noise reduction gain of the second audio frame.

[0119] It should be noted that, if the first audio frame is the first audio frame among the L audio frames, the noise reduction gain of the second audio frame is preset.

[0120] It should be noted that when the electronic device performs pre-noise reduction processing on the first frequency domain signal, it also performs pre-noise reduction processing on the frequency domain signal corresponding to each of the L audio frames. The pre-noise reduction processing process for the frequency domain signal corresponding to each of the L audio frames can refer to the pre-noise reduction processing process for the frequency domain signal corresponding to the first audio frame, and will not be further described here.

[0121] Optionally, in an embodiment of the present application, the above step 301 can be specifically implemented through the following steps 301a and 301b.

[0122] Step 301a: The electronic device performs linear prediction processing on the first frequency domain signal to obtain a residual signal corresponding to the first frequency domain signal.

[0123] In the embodiment of the present application, the electronic device can obtain a residual signal corresponding to the first frequency domain signal through the autocorrelation coefficient corresponding to the first frequency domain signal.

[0124] Step 301b: The electronic device performs frequency domain conversion processing on the residual signal to obtain a residual signal in the cepstrum domain.

[0125] In the embodiment of the present application, the residual signal in the cepstral domain is the first excitation information.

[0126] Exemplarily, taking the first audio frame of the L audio frames corresponding to the above-mentioned first frequency domain signal as an example, the electronic device can obtain the autocorrelation coefficient corresponding to the first audio frame based on the frequency domain signal corresponding to the first audio frame, and then obtain the linear prediction coding (LPC) linear prediction coefficient based on the autocorrelation coefficient, and then transform the LPC linear prediction coefficient into the frequency domain and perform filtering processing to obtain a linear prediction residual signal, and convert the residual signal into the cepstrum domain to obtain the excitation information of the frequency domain signal corresponding to the first audio frame.

[0127] For example, taking the first audio frame after pre-noise reduction processing as an example, the electronic device can obtain the autocorrelation coefficient corresponding to the first audio frame through the IFFT algorithm, which can be specifically implemented by the following formula 14: acc (n) = IFFT[|X c (f,k)| 2 ] (Formula 14)

[0128] Among them, R acc(n) is the autocorrelation coefficient corresponding to the first audio frame after pre-noise reduction processing, X c (f, k) is the frequency domain representation of the first audio frame after pre-noise reduction processing;

[0129] The autocorrelation coefficient can be calculated by the Levinson-Durbin recursive formula to obtain the LPC linear prediction coefficient, which can be specifically achieved by the following formula 15: lpc_α(n)=LevinsonDurbin(R acc (n)) (Formula 15)

[0130] Wherein, lpc_α(n) is the LPC linear prediction coefficient;

[0131] The electronic device can transform the LPC linear prediction coefficient into the frequency domain and perform filtering to obtain a linear prediction residual signal, which can be specifically implemented by the following formula 16: res (f,k)=X c (f,k)·(1-FFT[lpc_α(n)]) (Formula 16)

[0132] Among them, X res (f,k) is the LPC residual signal;

[0133] It should be noted that, in order to prevent the LPC linear prediction coefficients from containing time domain information, the electronic device may perform a frequency domain transform on the LPC linear prediction coefficients;

[0134] The residual signal is converted into the cepstrum domain to obtain the excitation information of the frequency domain signal corresponding to the first audio frame after the pre-noise reduction process, which can be specifically achieved by the following formula 17: x =IFFT[20*log10(|X res (f,k)|)] (Formula 17)

[0135] Among them, E x is the excitation information of the frequency domain signal corresponding to the first audio frame after the pre-noise reduction processing;

[0136] Step 202a1: The electronic device determines the pitch period of the first frequency domain signal based on the peak value of the first excitation information.

[0137] In the embodiment of the present application, the peak value is correlated with the pitch period. Therefore, after obtaining the peak value of the first excitation information, the electronic device can determine the pitch period of the first frequency domain signal based on the correlation.

[0138] Exemplarily, the above association relationship is preset by the electronic device; or determined by the user.

[0139] Optionally, in an embodiment of the present application, the first excitation information may include M pitch periods, where the M pitch periods respectively correspond to M excitation information in the first excitation information, and M is a positive integer.

[0140] For example, the above correspondence may be one-to-one or one-to-many, that is, one pitch period corresponds to one excitation information or one pitch period corresponds to K excitation information, where K is less than M.

[0141] Step 202a2: The electronic device determines second excitation information corresponding to the pitch period from the preset excitation codebook.

[0142] In the embodiment of the present application, the second incentive information is clean voice incentive information.

[0143] In the embodiment of the present application, the electronic device determines a first pitch period of the first excitation information.

[0144] Exemplarily, the first speech signal carries mechanical wave energy, the wavelength of the mechanical wave is called a fundamental wave, and the period corresponding to the fundamental wave is called a fundamental pitch period.

[0145] Optionally, in an embodiment of the present application, the number of the above-mentioned first incentive information may be one or more.

[0146] In some embodiments of the present application, after obtaining the first excitation information, the electronic device may search for frequency values ​​of multiple excitation information and then obtain the first fundamental frequency period corresponding to the first excitation information through the peak value of the frequency of the first excitation information.

[0147] In the embodiment of the present application, the electronic device may use the excitation information of the second clean speech signal in the above-mentioned preset excitation codebook as the second excitation information.

[0148] In this embodiment of the present application, the pitch period corresponding to the second clean speech signal is the first pitch period.

[0149] In the embodiment of the present application, the electronic device can find the excitation codebook that best matches the first pitch period in the preset excitation codebook, and obtain the excitation information of the clean speech signal corresponding to the noisy speech signal, that is, the above-mentioned second excitation information.

[0150] For example, assuming that the first excitation information is a vector with a length of 1 / 2 FFT length, the fundamental pitch period F0 of the first audio signal can be determined by searching for the peak position of the highest vibration frequency; then, the second excitation information can be obtained by finding the codebook that best matches the fundamental pitch period F0 from the preset excitation codebook using the following formula 18. Formula 18 can be specifically: E s =E codebook(F0) (Formula 18)

[0151] Among them, E s is the second stimulus information, E codebook () is the preset excitation codebook, and F0 is the fundamental frequency period of the first audio signal.

[0152] In an embodiment of the present application, the electronic device can determine the second excitation information from a preset excitation codebook based on the fundamental period of the first audio signal, so that the electronic device can estimate the spectrum information of the clean speech signal based on the second excitation information, thereby improving the accuracy of the spectrum information obtained by the electronic device.

[0153] Optionally, in the embodiment of the present application, the above step 202a can be specifically implemented through the following step 202a3.

[0154] Step 202a3: The electronic device inputs the first stimulus information into the first network model to obtain second stimulus information.

[0155] In the embodiment of the present application, the first network model is an incentive matching model.

[0156] In an embodiment of the present application, the first feature matching model may include a preset excitation codebook.

[0157] Exemplarily, the above-mentioned incentive matching model can be a first deep neural network model.

[0158] In an embodiment of the present application, the electronic device can input the above-mentioned first stimulus information into the first deep neural network model to obtain the second stimulus information.

[0159] Exemplarily, as shown in FIG4 , the above-mentioned first deep neural network model may include a normalization layer (LayerNorm, LN), represented by LN in FIG4 , a first linear layer, represented by Linear1 in FIG4 , a long short-term memory network (Long Short-Term Memory, LSTM) layer, represented by LSTM in FIG4 , and a second linear layer, represented by Linear2 in FIG4 .

[0160] As another example, the electronic device can input the first excitation information into the normalization layer, so that the electronic device can normalize the first excitation information to obtain the normalized first excitation information. The electronic device inputs the normalized first excitation information into the first linear layer, and the first linear layer can perform feature extraction and dimensionality reduction on the normalized first excitation information to output first feature information of size 128. Then, the electronic device performs time series modeling on the first feature information through the LSTM layer, and the second linear layer is responsible for feature mapping the modeled first feature information output by the LSTM with the preset excitation codebook in the first deep neural network model, thereby determining the excitation feature of the clean speech signal corresponding to the noisy speech signal from the preset excitation codebook, and outputting the excitation information of the clean speech signal through the excitation feature, that is, the above-mentioned second excitation information.

[0161] It should be noted that the above LSTM layer is a unidirectional LSTM network with a layer number of 2 to more fully explore the implicit information of the time series. The number of hidden layer nodes of the two-layer LSTM network is set to 128.

[0162] In some embodiments of the present application, the electronic device may provide a deep neural network model that can directly predict the excitation information of a clean speech signal corresponding to a noisy speech signal, avoiding interference caused by noise aliasing into the speech signal. The entire neural network reasoning process includes the process of restoring clean excitation information, and has more accurate prediction values. Moreover, the deep neural network model adopts a sequence model with time memory capability, which can solve long-distance dependencies in speech signals, thereby better coping with the complexity and long delay of speech signals.

[0163] Step 203c: The electronic device determines second envelope information matching the first envelope information from a preset envelope codebook.

[0164] In the embodiment of the present application, the preset envelope codebook is obtained by training the envelope information set of the clean speech signal to obtain NA representative clean speech envelope information subsets.

[0165] In some embodiments of the present application, the electronic device may process the speech signal in the above-mentioned clean speech data set to obtain a preset envelope codebook.

[0166] In the embodiment of the present application, the electronic device uses the envelope information of the third clean speech signal in the above-mentioned preset envelope codebook as the second envelope information.

[0167] In some embodiments of the present application, the third clean speech signal is: a clean speech signal having the smallest difference between the length of the envelope information and the length of the first envelope information in the preset envelope codebook.

[0168] Optionally, in an embodiment of the present application, the voice enhancement method provided in the embodiment of the present application further includes the following step 401, and the above-mentioned step 203c can be specifically implemented through the following step 203c1.

[0169] Step 401: The electronic device performs linear prediction analysis on the first frequency domain signal to obtain first envelope information.

[0170] In the embodiment of the present application, the electronic device can obtain line spectrum frequency pairs (LSFs) corresponding to the first frequency domain signal through the autocorrelation coefficient corresponding to the first frequency domain signal, and the LSFs correspond to the first envelope information.

[0171] Optionally, in an embodiment of the present application, the above step 401 can be specifically implemented through the following steps 401a and 401b.

[0172] Step 401a: The electronic device performs linear prediction processing on the first frequency domain signal to obtain linear prediction coding coefficients corresponding to the first frequency domain signal.

[0173] Step 401b: The electronic device converts the linear prediction coding coefficients to obtain line spectrum frequency pairs corresponding to the linear prediction coding coefficients.

[0174] In the embodiment of the present application, the line spectrum frequency pair corresponding to the above-mentioned linear prediction coding coefficient is the first envelope information.

[0175] For example, taking the first audio frame after pre-noise reduction processing as an example, the electronic device can obtain the autocorrelation coefficient corresponding to the first audio frame through the IFFT algorithm, which can be specifically implemented by the above formula 14;

[0176] The autocorrelation coefficient can be calculated by the Levinson-Durbin recursive formula to obtain the LPC linear prediction coefficient, which can be specifically achieved by the above formula 15;

[0177] The LPC linear prediction coefficient is converted into LSF to obtain the envelope information of the frequency domain signal corresponding to the first audio frame after pre-noise reduction processing, which can be specifically achieved by the following formula 19: x =LPCtoLSF(lpc_α(n)) (Formula 19)

[0178] Among them, A x It is the envelope information of the frequency domain signal corresponding to the first audio frame after the pre-noise reduction processing.

[0179] It can be understood that the electronic device can perform the same processing as above formulas 14 to 19 on the frequency domain signal corresponding to each audio frame in the L audio frames, thereby obtaining the excitation information and envelope information of the frequency domain signal corresponding to each audio frame.

[0180] Step 203c1: The electronic device determines the clean speech envelope information in the preset envelope codebook that has the smallest vector distance with the first envelope information as the second envelope information.

[0181] In the embodiment of the present application, the electronic device may determine the second envelope information from a preset envelope codebook in a targeted manner.

[0182] Optionally, in an embodiment of the present application, the above-mentioned target method may include any one of the following: Euclidean distance, cosine similarity, normalized Euclidean distance, Hamming distance, and string similarity.

[0183] For example, assuming that the first envelope information is a vector of length P, in the preset envelope codebook, the electronic device can find the best match according to the minimum Euclidean distance of the vector, thereby obtaining the envelope information of the clean speech signal corresponding to the noisy speech signal, that is, the second envelope information mentioned above, which can be specifically implemented by the following formula 20:

[0184] Among them, A s is the second envelope information, A codebook () is the preset envelope codebook, A x is the first envelope information.

[0185] In some embodiments of the present application, the electronic device can obtain second envelope information from a preset envelope codebook, so that the electronic device can estimate the spectrum information of the clean speech signal based on the second envelope information, thereby improving the accuracy of the spectrum information obtained by the electronic device.

[0186] Optionally, in the embodiment of the present application, the above step 203c can be specifically implemented through the following step 203a2.

[0187] Step 203c2: The electronic device inputs the first envelope information into the second network model to obtain second excitation information.

[0188] In the embodiment of the present application, the second network model is an envelope matching model.

[0189] In the embodiment of the present application, the second feature matching model includes the preset envelope codebook.

[0190] Exemplarily, as shown in Figure 5, the above-mentioned second deep neural network model may include a normalization layer (LayerNorm, LN), represented by LN in Figure 5, a first linear (Linear) layer, represented by Linear1 in Figure 5, a long short-term memory network (Long Short-Term Memory, LSTM) layer, represented by LSTM in Figure 5, a second linear (Linear) layer, represented by Linear2 in Figure 5 and an activation function (Sigmoid) layer, represented by Sigmoid in Figure 5.

[0191] For example, the activation function in the above Sigmoid layer is

[0192] It should be noted that, since the first envelope information is calculated using a line spectrum frequency pair, its value is between 0 and 1, the present invention selects Simoid as the activation function, and the result obtained is also between 0 and 1.

[0193] As another example, the electronic device may input the first envelope information into a normalization layer, so that the electronic device may normalize the first envelope information to obtain the normalized first envelope information. The electronic device inputs the normalized first envelope information into a first linear layer, and the first linear layer may perform feature extraction and dimensionality reduction on the normalized first envelope information to output second feature information of size P. Then, the electronic device performs time series modeling on the second feature information through the LSTM layer. The second linear layer is responsible for feature mapping the modeled second feature information output by the LSTM with the preset envelope codebook in the second deep neural network model, and matching the modeled second feature information through the activation function in the activation function layer to obtain the second envelope information.

[0194] It should be noted that the above LSTM layer is a unidirectional LSTM network with a layer number of 2 to more fully explore the implicit information of the time series. The number of hidden layer nodes of the two-layer LSTM network is set to 128.

[0195] In some embodiments of the present application, an electronic device provides a deep neural network model that can directly predict the envelope information of a clean speech signal, avoiding interference caused by noise aliasing into the speech signal. The entire neural network reasoning process includes the process of restoring the clean envelope information, resulting in more accurate predictions. Furthermore, the deep neural network model uses a sequence model with temporal memory capabilities to resolve long-range dependencies in speech signals and better cope with the complexity and long latency of speech signals.

[0196] Optionally, in the embodiment of the present application, the above-mentioned step 202a and step 203a can be specifically implemented by the following step 501.

[0197] Step 501: The electronic device inputs the first excitation information and the first envelope information into a third network model to obtain second excitation information and second envelope information.

[0198] In the embodiment of the present application, the third network model is a cross-matching model.

[0199] In an embodiment of the present application, the electronic device can input the first excitation information and the first envelope information into a third deep neural network, so as to simultaneously obtain the second excitation information and the second envelope information through one deep neural network.

[0200] For example, the third deep neural network may include two branches, namely a first branch and a second branch, the first branch is used to obtain the second envelope information, and the second branch is used to obtain the second excitation information.

[0201] As another example, as shown in FIG6 , the first branch includes a first normalization layer (denoted by LN1 in FIG6 ) and a first linear layer (denoted by Linear1 in FIG6 ). The second branch includes a second normalization layer (denoted by LN2 in FIG6 ) and a second linear layer (denoted by Linear2 in FIG6 ). The electronic device then uses a feature concatenation layer (denoted by + in FIG6 ) to concatenate the feature information of the envelope information output by the first linear layer with the feature information of the excitation information output by the second linear layer. The electronic device then inputs the concatenated feature information of the envelope information and the feature information of the excitation information, hereinafter referred to as the fourth feature information, into an LSTM layer, performs time series modeling on the fourth feature information, and inputs the modeled fourth feature information into the third linear layer in the first branch (denoted by Linear3 in FIG6 ) and the fourth linear layer in the second branch (denoted by Linear4 in FIG6 ). Then, the electronic device obtains the second envelope information through the activation function layer in the first branch, represented by Sigmoid in Figure 6, and the electronic device obtains the second excitation information through the fourth linear layer in the second branch. In this way, the electronic device can predict the envelope information of the clean speech signal and the excitation information of the clean speech signal respectively through different branch networks.

[0202] In some embodiments of the present application, an electronic device provides a deep neural network model that can directly predict the envelope characteristics of clean speech information and the excitation characteristics of clean speech signals, avoiding interference caused by noise aliasing into the speech signal. The entire neural network reasoning process includes the processes of envelope restoration and excitation restoration, which has more accurate prediction values. Moreover, the deep neural network model uses a sequence model with time memory capability to resolve long-distance dependencies in speech signals and better cope with the complexity and long delay of speech signals.

[0203] Optionally, in an embodiment of the present application, the speech enhancement method provided in the embodiment of the present application further includes the following steps 501 to 504.

[0204] Step 501: The electronic device obtains a set of clean speech signals.

[0205] In the embodiment of the present application, the above-mentioned clean speech signal set includes at least one clean speech signal.

[0206] Optionally, in an embodiment of the present application, the above-mentioned clean speech signal set may be stored in an electronic device; or sent to the electronic device by other electronic devices.

[0207] Step 502: The electronic device performs linear prediction processing on each clean speech signal in the clean speech signal set to obtain a feature information set corresponding to the clean speech signal set.

[0208] In an embodiment of the present application, the electronic device can obtain the LPC coefficient corresponding to each clean speech signal, and then obtain a feature information set based on the LPC coefficient.

[0209] In an embodiment of the present application, the above-mentioned feature information set includes excitation information of the above-mentioned at least one clean speech signal.

[0210] Step 503: The electronic device performs peak detection on the characteristic information set to obtain a pitch period corresponding to the characteristic information set.

[0211] In the embodiment of the present application, the electronic device can perform peak detection on the frequency information of all excitation information in the feature information set, thereby obtaining the peak value corresponding to each excitation information, and further obtaining the fundamental frequency period corresponding to each excitation information in the feature information set.

[0212] Step 504: The electronic device clusters the feature information in the feature information set based on the pitch period to obtain a preset excitation codebook.

[0213] Exemplarily, the electronic device can perform linear prediction analysis on each clean speech signal in the clean speech data set to obtain the LPC coefficient, and obtain the residual signal after linear filtering, and then perform logarithmic operation on the residual signal and convert it to the cepstral domain to obtain excitation information, and collect all excitation information to obtain an excitation information set. Then, by performing peak detection on each excitation information in the excitation information set, the peak position of each excitation information is determined, and the peak position corresponds to the fundamental pitch period F0 of the above-mentioned clean speech. The excitation information in the excitation information set is clustered according to F0, and the excitation information with the same F0 is classified as the same cluster excitation information. The excitation information within the cluster is weighted averaged to obtain the excitation codebook under the F0.

[0214] Exemplarily, the electronic device may perform weighted averaging on the excitation information of the same cluster by using the following formula 21 to obtain the excitation codebook under the F0. Specifically, the formula is:

[0215] Among them, E codebook (f) is the preset excitation codebook, pitch(E x ) is the same cluster of excitation information, E x is any stimulus information in the same cluster of stimulus information.

[0216] It can be understood that one pitch period corresponds to one excitation codebook, and the preset excitation codebook is a collection of excitation codebooks corresponding to K pitch periods, where K is a positive integer.

[0217] It should be noted that the execution order of the above steps 501 to 504 can be before the above step 201, or before the above step 202, which can be determined according to actual usage and is not limited in the embodiment of the present application.

[0218] In the embodiment of the present application, since the preset excitation codebook contains the excitation information of the clean speech signal, the electronic device can determine the second excitation information from the preset excitation codebook, thereby ensuring that the electronic device can obtain the target spectrum information through the excitation information of the clean speech signal, and thus obtain a speech signal with better noise reduction effect.

[0219] Optionally, in the embodiment of the present application, the speech enhancement method provided in the embodiment of the present application further includes the following steps 601 to 603.

[0220] Step 601: The electronic device obtains a set of clean speech signals.

[0221] In the embodiment of the present application, the above-mentioned clean speech signal set includes at least one clean speech signal.

[0222] Optionally, in an embodiment of the present application, the above-mentioned clean speech signal set may be stored in an electronic device; or sent to the electronic device by other electronic devices.

[0223] Optionally, in an embodiment of the present application, the above-mentioned clean speech signal set and the clean speech signal set in the above-mentioned step 501 can be the same speech signal set; or, they can be different speech signal sets.

[0224] Step 602: The electronic device performs linear prediction processing on each clean speech signal in the clean speech signal set to obtain a feature information set corresponding to the clean speech signal set.

[0225] In an embodiment of the present application, the electronic device can obtain the LPC coefficient corresponding to each clean speech signal, and then obtain a feature information set based on the LPC coefficient.

[0226] In an embodiment of the present application, the above-mentioned feature information set includes envelope information of the above-mentioned at least one clean speech signal.

[0227] Step 603: The electronic device clusters the feature information set using a preset algorithm to obtain a preset envelope codebook.

[0228] For example, the electronic device can perform linear prediction analysis on the speech signals in the clean speech dataset to obtain LPC coefficients, convert them into LSF coefficients as envelope information, collect all the envelope information, and thus obtain an envelope information set. The envelope information set is then clustered using an unsupervised machine learning method to obtain a sparse representation of a preset envelope codebook.

[0229] It should be noted that the electronic device clusters the envelope information set to obtain the cluster center of the envelope information set, which means finding the most representative envelope information in the envelope information set.

[0230] It should be noted that the execution order of the above steps 601 to 603 can be before the above step 201, or before the above step 202, which can be determined according to actual usage and is not limited in the embodiment of the present application.

[0231] In the embodiment of the present application, since the preset envelope codebook contains the envelope information of the clean speech signal, the electronic device can determine the second envelope information from the preset envelope codebook, thereby ensuring that the electronic device can obtain the target spectrum information through the envelope information of the clean speech signal, and thus obtain a speech signal with better noise reduction effect.

[0232] For example, the spectrum information determination method provided in the embodiment of the present application is explained below through a specific example. The spectrum information determination method provided in the embodiment of the present application is explained below using an audio frame as a unit.

[0233] Step 20: The electronic device performs short-time Fourier transform on the first speech signal to obtain a first frequency domain signal corresponding to the first speech signal.

[0234] In this embodiment of the present application, the first frequency domain signal includes L audio frames.

[0235] Step 21: The electronic device performs pre-noise reduction and linear prediction analysis on the first audio frame to obtain first excitation information and first envelope information of the first audio frame.

[0236] In the embodiment of the present application, the first audio frame is any one of the L audio frames.

[0237] Step 22: Excitation codebook matching.

[0238] In the embodiment of the present application, the electronic device obtains second excitation information that matches the first excitation information from a preset excitation codebook.

[0239] Step 23: Envelope codebook matching.

[0240] In the embodiment of the present application, the electronic device obtains second envelope information matching the first envelope information from a preset envelope codebook.

[0241] Step 24: The electronic device obtains first spectrum information based on the LPC linear prediction coefficient, the first excitation information, the second excitation information, and the residual information.

[0242] In the embodiment of the present application, the above-mentioned LPC linear prediction coefficient is obtained based on the second envelope information.

[0243] Step 25: The electronic device obtains a priori signal-to-noise ratio based on the first spectrum information and spectrum information of the noisy speech signal.

[0244] Step 26: The electronic device obtains a noise reduction gain based on the priori signal-to-noise ratio, and obtains a signal spectrum after noise reduction processing based on the noise reduction gain.

[0245] Step 27: The electronic device performs STFT on the spectrum of the signal after noise reduction processing to obtain a time domain signal after noise reduction processing, that is, a target clean speech signal corresponding to the first audio frame.

[0246] It should be noted that the specific implementation of the above steps 20 to 27 can be found in the above embodiments, and will not be described here in detail to avoid repetition.

[0247] In an embodiment of the present application, the electronic device can perform optimal excitation codebook matching on the first excitation information, and perform optimal envelope codebook matching on the first envelope information, so that the electronic device can obtain the second excitation information and the second envelope information corresponding to the first clean voice signal, and then can obtain the target spectrum information through the first excitation information, the second excitation information and the second envelope information; so that the electronic device can obtain the accurate prior signal-to-noise ratio corresponding to the first clean voice signal based on the target spectrum information, and then the electronic device can perform speech enhancement processing based on the accurate prior signal-to-noise ratio, thereby improving the noise reduction effect of the electronic device for voice noise reduction.

[0248] It should be noted that the speech enhancement method provided in the embodiments of the present application can be performed by a speech enhancement device, an electronic device, or a functional module or entity in an electronic device. In the embodiments of the present application, the speech enhancement device provided in the embodiments of the present application is described by taking the speech enhancement method performed by the speech enhancement device as an example.

[0249] FIG7 shows a possible structural diagram of a speech enhancement device involved in an embodiment of the present application. As shown in FIG7 , the speech enhancement device 70 may include: an extraction module 71 , a matching module 72 , a synthesis module 73 and a processing module 74 .

[0250] Among them, the extraction module 71 is used to extract the first excitation information and the first envelope information of the first frequency domain signal corresponding to the first speech signal, where the first speech signal includes a noise signal and a first clean speech signal. The matching module 72 is used to perform optimal excitation codebook matching on the first excitation information to obtain second excitation information; and to perform optimal envelope codebook matching on the first envelope information to obtain second envelope information. The synthesis module 73 is used to synthesize the target spectrum information based on the first excitation information, the second excitation information, and the second envelope information. The processing module 74 is used to perform speech enhancement processing on the first speech signal based on the target spectrum information.

[0251] In one possible implementation, the matching module 72 is specifically configured to determine second excitation information that matches the first excitation information from a preset excitation codebook; or to determine second envelope information that matches the first envelope information from a preset envelope codebook; wherein the preset excitation codebook is a set of excitation information of a clean speech signal trained to obtain representative NC subsets of clean speech excitation information; and the preset envelope codebook is a set of envelope information of a clean speech signal trained to obtain representative NA subsets of clean speech envelope information.

[0252] In one possible implementation, the processing module 74 is further configured to perform linear prediction analysis on the first frequency domain signal to obtain first excitation information. The matching module 72 is specifically configured to determine the pitch period of the first frequency domain signal based on the peak value of the first excitation information; and to determine second excitation information corresponding to the pitch period from a preset excitation codebook, where the second excitation information is clean speech excitation information.

[0253] In one possible implementation, the above-mentioned processing module 74 is specifically used to perform linear prediction processing on the first frequency domain signal to obtain a residual signal corresponding to the first frequency domain signal; and perform frequency domain conversion processing on the residual signal to obtain a residual signal in the cepstral domain; wherein the residual signal in the cepstral domain is the first excitation information.

[0254] In one possible implementation, the processing module 74 is further configured to perform linear prediction analysis on the first frequency domain signal to obtain the first envelope information. The matching module 72 is specifically configured to determine, as the second envelope information, the clean speech envelope information in the preset envelope codebook that has the smallest vector distance to the first envelope information.

[0255] In one possible implementation, the above-mentioned processing module 74 is specifically used to perform linear prediction processing on the first frequency domain signal to obtain linear prediction coding coefficients corresponding to the first frequency domain signal; and convert the linear prediction coding coefficients to obtain line spectrum frequency pairs corresponding to the linear prediction coding coefficients; the line spectrum frequency pairs corresponding to the linear prediction coding coefficients are the first envelope information.

[0256] In one possible implementation, the speech enhancement device provided in an embodiment of the present application further includes: an acquisition module configured to acquire a clean speech signal set, wherein the clean speech signal set includes at least one clean speech signal. The processing module 74 is further configured to perform linear prediction processing on each clean speech signal in the clean speech signal set to obtain a feature information set corresponding to the clean speech signal set; perform peak detection on the feature information set to obtain a pitch period corresponding to the feature information set; and cluster feature information in the feature information set based on the pitch period to obtain a preset excitation codebook.

[0257] In one possible implementation, the speech enhancement device provided in an embodiment of the present application further includes: an acquisition module configured to acquire a clean speech signal set, the clean speech signal set including at least one clean speech signal. The processing module 74 is further configured to perform linear prediction processing on each clean speech signal in the clean speech signal set to obtain a feature information set corresponding to the clean speech signal set; and to cluster the feature information set using a preset algorithm to obtain a preset envelope codebook.

[0258] In a possible implementation, the matching module 72 is specifically configured to input the first stimulus information into the first network model to obtain the second stimulus information; wherein the first network model is a stimulus matching model.

[0259] In a possible implementation, the matching module 72 is specifically configured to input the first envelope information into the second network model to obtain the second excitation information; wherein the second network model is an envelope matching model.

[0260] In a possible implementation, the matching module 72 is specifically configured to input the first excitation information and the first envelope information into a third network model to obtain second excitation information and second envelope information; wherein the third network model is a cross-matching model.

[0261] An embodiment of the present application provides a speech enhancement device, which can perform optimal excitation codebook matching on first excitation information and optimal envelope codebook matching on first envelope information, so that the speech enhancement device can obtain second excitation information and second envelope information corresponding to the first clean speech signal, and then can obtain target spectrum information through the first excitation information, the second excitation information and the second envelope information; so that the speech enhancement device can obtain an accurate priori signal-to-noise ratio corresponding to the first clean speech signal based on the target spectrum information, and then the speech enhancement device can perform speech enhancement processing based on the accurate priori signal-to-noise ratio, thereby improving the noise reduction effect of the speech enhancement device in performing speech noise reduction.

[0262] The speech enhancement device in the embodiment of the present application can be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other device other than a terminal. For example, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a mobile Internet device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc. It can also be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., and the embodiment of the present application does not specifically limit it.

[0263] The speech enhancement device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0264] The speech enhancement device provided in the embodiment of the present application can implement each process implemented in the above method embodiment. To avoid repetition, it will not be described here.

[0265] Optionally, as shown in Figure 8, an embodiment of the present application also provides an electronic device 90, including a processor 91 and a memory 92, and the memory 92 stores a program or instruction that can be run on the processor 91. When the program or instruction is executed by the processor 91, the various steps of the above-mentioned spectrum information determination method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0266] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0267] FIG9 is a schematic diagram of the hardware structure of an electronic device implementing an embodiment of the present application.

[0268] The electronic device 100 includes but is not limited to components such as a radio frequency unit 101 , a network module 102 , an audio output unit 103 , an input unit 104 , a sensor 105 , a display unit 106 , a user input unit 107 , an interface unit 108 , a memory 109 , and a processor 110 .

[0269] Those skilled in the art will appreciate that the electronic device 100 may further include a power source (such as a battery) for powering various components. The power source may be logically connected to the processor 110 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The electronic device structure shown in FIG9 does not limit the electronic device. The electronic device may include more or fewer components than shown, or may combine certain components or arrange the components differently, which will not be described in detail here.

[0270] Among them, the processor 110 is used to extract the first excitation information and the first envelope information of the first frequency domain signal corresponding to the first speech signal, where the first speech signal includes a noise signal and a first clean speech signal; perform optimal excitation codebook matching on the first excitation information to obtain second excitation information; perform optimal envelope codebook matching on the first envelope information to obtain second envelope information; synthesize target spectrum information based on the first excitation information, the second excitation information and the second envelope information; and perform speech enhancement processing on the first speech signal according to the target spectrum information.

[0271] An embodiment of the present application provides an electronic device, which can perform optimal excitation codebook matching on first excitation information and optimal envelope codebook matching on first envelope information, so that the electronic device can obtain second excitation information and second envelope information corresponding to the first clean speech signal, and then can obtain target spectrum information through the first excitation information, the second excitation information and the second envelope information; so that the electronic device can obtain an accurate priori signal-to-noise ratio corresponding to the first clean speech signal based on the target spectrum information, and then the electronic device can perform speech enhancement processing based on the accurate priori signal-to-noise ratio, thereby improving the noise reduction effect of the electronic device in performing speech noise reduction.

[0272] Optionally, in an embodiment of the present application, the above-mentioned processor 110 is specifically used to determine the second excitation information that matches the first excitation information from a preset excitation codebook; or, to determine the second envelope information that matches the first envelope information from a preset envelope codebook; wherein, the preset excitation codebook is obtained by training an excitation information set of a clean speech signal to obtain a representative NC subset of clean speech excitation information; the preset envelope codebook is obtained by training an envelope information set of a clean speech signal to obtain a representative NA subset of clean speech envelope information.

[0273] Optionally, in an embodiment of the present application, the above-mentioned processor 110 is also used to perform linear prediction analysis on the first frequency domain signal to obtain first excitation information; and determine the fundamental period of the first frequency domain signal based on the peak value of the first excitation information; and determine the second excitation information corresponding to the fundamental period from a preset excitation codebook, where the second excitation information is clean speech excitation information.

[0274] Optionally, in an embodiment of the present application, the above-mentioned processor 110 is specifically used to perform linear prediction processing on the first frequency domain signal to obtain a residual signal corresponding to the first frequency domain signal; and perform frequency domain conversion processing on the residual signal to obtain a residual signal in the cepstral domain; wherein the residual signal in the cepstral domain is the first excitation information.

[0275] Optionally, in an embodiment of the present application, the above-mentioned processor 110 is also used to perform linear prediction analysis on the first frequency domain signal to obtain first envelope information; and determine the clean speech envelope information with the smallest vector distance with the first envelope information in the preset envelope codebook as the second envelope information.

[0276] Optionally, in an embodiment of the present application, the above-mentioned processor 110 is specifically used to perform linear prediction processing on the first frequency domain signal to obtain linear prediction coding coefficients corresponding to the first frequency domain signal; and convert the linear prediction coding coefficients to obtain line spectrum frequency pairs corresponding to the linear prediction coding coefficients; and the line spectrum frequency pairs corresponding to the linear prediction coding coefficients are the first envelope information.

[0277] Optionally, in an embodiment of the present application, the above-mentioned processor 110 is also used to obtain a clean speech signal set, which contains at least one clean speech signal; perform linear prediction processing on each clean speech signal in the clean speech signal set to obtain a feature information set corresponding to the clean speech signal set; perform peak detection on the feature information set to obtain a fundamental frequency period corresponding to the feature information set; and cluster the feature information in the feature information set based on the fundamental frequency period to obtain a preset excitation codebook.

[0278] Optionally, in an embodiment of the present application, the above-mentioned processor 110 is also used to obtain a clean speech signal set, which contains at least one clean speech signal; perform linear prediction processing on each clean speech signal in the clean speech signal set to obtain a feature information set corresponding to the clean speech signal set; and cluster the feature information set through a preset algorithm to obtain a preset envelope codebook.

[0279] Optionally, in an embodiment of the present application, the processor 110 is specifically configured to input the first incentive information into a first network model to obtain second incentive information; wherein the first network model is an incentive matching model.

[0280] Optionally, in an embodiment of the present application, the processor 110 is specifically configured to input the first envelope information into a second network model to obtain second excitation information; wherein the second network model is an envelope matching model.

[0281] Optionally, in an embodiment of the present application, the processor 110 is specifically configured to input the first excitation information and the first envelope information into a third network model to obtain second excitation information and second envelope information; wherein the third network model is a cross-matching model.

[0282] The electronic device provided in the embodiment of the present application can implement each process implemented in the above method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0283] The beneficial effects of various implementations in this embodiment can be specifically referred to the beneficial effects of the corresponding implementations in the above method embodiment. To avoid repetition, they will not be described here.

[0284] It should be understood that in an embodiment of the present application, the input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 106 may include a display panel 1061, and the display panel 1061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 107 includes a touch panel 1071 and at least one of other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0285] The memory 109 can be used to store software programs and various data. The memory 109 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, applications or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 109 may include a volatile memory or a non-volatile memory, or the memory 109 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM), and a direct memory bus random access memory (DRRAM). The memory 109 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0286] Processor 110 may include one or more processing units. Optionally, processor 110 integrates an application processor and a modem processor. The application processor primarily handles operations related to the operating system, user interface, and application programs, while the modem processor primarily processes wireless communication signals, such as a baseband processor. It is understood that the modem processor may not be integrated into processor 110.

[0287] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0288] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0289] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned method embodiment and achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0290] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0291] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned spectrum information determination method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0292] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0293] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0294] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. A speech enhancement method, the method comprising: Extracting first excitation information and first envelope information of a first frequency domain signal corresponding to a first speech signal, wherein the first speech signal includes a noise signal and a first clean speech signal; Performing optimal excitation codebook matching on the first excitation information to obtain second excitation information; Performing optimal envelope codebook matching on the first envelope information to obtain second envelope information; synthesizing target spectrum information based on the first excitation information, the second excitation information and the second envelope information; Perform speech enhancement processing on the first speech signal according to the target spectrum information.

2. The method according to claim 1, wherein: The performing optimal excitation codebook matching on the first excitation information to obtain the second excitation information comprises: Determining second excitation information matching the first excitation information from a preset excitation codebook; Or, performing optimal envelope codebook matching on the first envelope information to obtain second envelope information includes: Determine second envelope information matching the first envelope information from a preset envelope codebook; The preset excitation codebook is obtained by training the excitation information set of the clean speech signal to obtain NC representative clean speech excitation information subsets; The preset envelope codebook is obtained by training the envelope information set of the clean speech signal to obtain NA representative clean speech envelope information subsets.

3. The method according to claim 2, wherein: The method further comprises: Performing linear prediction analysis on the first frequency domain signal to obtain the first excitation information; The determining, from a preset excitation codebook, second excitation information matching the first excitation information comprises: Determining a pitch period of the first frequency domain signal based on a peak value of the first excitation information; The second excitation information corresponding to the pitch period is determined from the preset excitation codebook, where the second excitation information is clean speech excitation information.

4. The method according to claim 3, wherein: The performing linear prediction analysis on the first frequency domain signal to obtain the first excitation information includes: Performing linear prediction processing on the first frequency domain signal to obtain a residual signal corresponding to the first frequency domain signal; Performing frequency domain conversion processing on the residual signal to obtain a residual signal in the cepstrum domain; The residual signal in the cepstrum domain is the first excitation information.

5. The method according to claim 2, wherein: The method further comprises: Performing linear prediction analysis on the first frequency domain signal to obtain the first envelope information; The determining, from a preset envelope codebook, second envelope information matching the first envelope information comprises: The clean speech envelope information having the smallest vector distance with the first envelope information in the preset envelope codebook is determined as the second envelope information.

6. The method according to claim 5, wherein: The performing linear prediction analysis on the first frequency domain signal to obtain the first envelope information includes: Performing linear prediction processing on the first frequency domain signal to obtain a linear prediction coding coefficient corresponding to the first frequency domain signal; Converting the linear prediction coding coefficients to obtain line spectrum frequency pairs corresponding to the linear prediction coding coefficients; The line spectrum frequency pair corresponding to the linear prediction coding coefficient is the first envelope information.

7. The method according to claim 2, wherein: The method further comprises: Acquire a clean speech signal set, wherein the clean speech signal set includes at least one clean speech signal; Perform linear prediction processing on each clean speech signal in the clean speech signal set to obtain the clean speech signal. A feature information set corresponding to a clean speech signal set; Performing peak detection on the feature information set to obtain a pitch period corresponding to the feature information set; Based on the pitch period, the feature information in the feature information set is clustered to obtain the preset excitation codebook.

8. The method according to claim 2, wherein: The method further comprises: Acquire a clean speech signal set, wherein the clean speech signal set includes at least one clean speech signal; Performing linear prediction processing on each of the clean speech signals in the clean speech signal set to obtain a feature information set corresponding to the clean speech signal set; The characteristic information set is clustered by a preset algorithm to obtain the preset envelope codebook.

9. The method according to claim 1 or 2, wherein: The performing optimal excitation codebook matching on the first excitation information to obtain the second excitation information comprises: Inputting the first stimulus information into the first network model to obtain second stimulus information; Among them, the first network model is an incentive matching model.

10. The method according to claim 1 or 2, wherein: The performing optimal envelope codebook matching on the first envelope information to obtain second envelope information includes: Inputting the first envelope information into a second network model to obtain second excitation information; Wherein, the second network model is an envelope matching model.

11. The method according to claim 1, wherein: performing optimal excitation codebook matching on the first excitation information to obtain second excitation information; Performing optimal envelope codebook matching on the first envelope information to obtain second envelope information includes: Inputting the first excitation information and the first envelope information into a third network model to obtain second excitation information and second envelope information; Wherein, the third network model is a cross-matching model.

12. A speech enhancement device, comprising: Extraction module, matching module, synthesis module and processing module; The extraction module is used to extract first excitation information and first envelope information of a first frequency domain signal corresponding to a first speech signal, wherein the first speech signal includes a noise signal and a first clean speech signal; The matching module is used to perform optimal excitation codebook matching on the first excitation information extracted by the extraction module to obtain second excitation information; and perform optimal envelope codebook matching on the first envelope information extracted by the extraction module to obtain second envelope information; The synthesis module is used to synthesize target spectrum information based on the first excitation information, the second excitation information and the second envelope information; The processing module is used to perform speech enhancement processing on the first speech signal according to the target spectrum information.

13. The device according to claim 12, wherein: The matching module is specifically configured to determine second excitation information matching the first excitation information from a preset excitation codebook; or determine second envelope information matching the first envelope information from a preset envelope codebook; The preset excitation codebook is obtained by training the excitation information set of the clean speech signal to obtain NC representative clean speech excitation information subsets; The preset envelope codebook is obtained by training the envelope information set of the clean speech signal to obtain NA representative clean speech envelope information subsets.

14. The device according to claim 13, wherein: The speech enhancement device further comprises: a processing module; The processing module is used to perform linear prediction analysis on the first frequency domain signal to obtain the first excitation information; The matching module is specifically used to determine the fundamental pitch period of the first frequency domain signal based on the peak value of the first excitation information; and determine the second excitation information corresponding to the fundamental pitch period from the preset excitation codebook, wherein the second excitation information is clean speech excitation information.

15. The device according to claim 13, wherein: The speech enhancement device further comprises: a processing module; The processing module is specifically used to perform linear prediction analysis on the first frequency domain signal to obtain the first envelope information; The matching module is specifically configured to determine the clean speech envelope information having the smallest vector distance with the first envelope information in the preset envelope codebook as the second envelope information.

16. The device according to claim 12 or 13, wherein: The matching module is specifically used to input the first stimulus information into the first network model to obtain second stimulus information; Among them, the first network model is an incentive matching model.

17. The device according to claim 12 or 13, wherein: The matching module is specifically used to input the first envelope information into the second network model to obtain second excitation information; Wherein, the second network model is an envelope matching model.

18. The device according to claim 12, wherein: The matching module is specifically used to input the first excitation information and the first envelope information into a third network model to obtain second excitation information and second envelope information; Wherein, the third network model is a cross-matching model.

19. An electronic device comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the speech enhancement method as claimed in any one of claims 1 to 11.

20. A readable storage medium storing a program or an instruction, wherein the program or the instruction, when executed by a processor, implements the steps of the speech enhancement method according to any one of claims 1 to 11.

21. A computer program product, wherein the program product is executed by at least one processor to implement the speech enhancement method according to any one of claims 1 to 11.

22. A user equipment UE, comprising: the UE being configured to perform the speech enhancement method according to any one of claims 1 to 11.

Images