A speech enhancement method, device, electronic device and storage medium

Abstract

The present embodiment discloses a speech enhancement method, device, electronic device and storage medium. The method includes: obtaining the power spectrum and phase spectrum of the speech signal to be enhanced, and the power spectrum estimation value of the noise signal in the speech signal to be enhanced, wherein the speech signal to be enhanced includes a clean speech signal and a noise signal; determining the power spectrum estimation value of the clean speech signal according to the power spectrum of the speech signal to be enhanced and the power spectrum estimation value of the noise signal; determining the masking threshold in different frequency bands according to the power spectrum estimation value of the clean speech signal; determining the enhanced power spectrum estimation value of the clean speech signal according to the masking threshold and the power spectrum of the speech signal to be enhanced; determining the enhanced speech signal according to the phase spectrum of the speech signal to be enhanced and the enhanced power spectrum estimation value of the clean speech signal. The embodiment of the present invention can enhance the noise suppression result and improve the speech recognition effect by determining the masking threshold to obtain the enhanced speech signal.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of signal processing technology, and in particular to a speech enhancement method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of signal processing technology and speech recognition technology, speech enhancement technology in front-end preprocessing has become increasingly important. Generally speaking, when a device plays sound, noise will be heard along with the speech. However, the presence of noise will interfere with the speech and even affect the human ear's perception of the speech. Usually, speech enhancement methods are used to process speech signals containing noise.

[0003] At present, the main speech enhancement methods include spectral subtraction, wavelet transform, Wiener filter, etc. Among them, spectral subtraction can effectively suppress noise when the signal-to-noise ratio of the input signal is high, but when the signal-to-noise ratio is low, more noise remains, and half-wave rectification of the negative value obtained after spectral subtraction will cause the appearance of "music noise", which seriously affects the recognition effect of speech. In other words, in a non-stationary environment, many speech enhancement methods will have problems with tracking delay and large errors.

[0004] Therefore, how to suppress noise and enhance speech effects in a non-stationary environment is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention

[0005] The embodiments of the present invention provide a speech enhancement method, device, electronic device and storage medium, which can enhance noise suppression results and improve speech recognition effects.

[0006] In a first aspect, an embodiment of the present invention provides a speech enhancement method, comprising:

[0007] Acquire a power spectrum and a phase spectrum of a speech signal to be enhanced, and a power spectrum estimation value of a noise signal in the speech signal to be enhanced, wherein the speech signal to be enhanced includes a pure speech signal and a noise signal;

[0008] Determining a power spectrum estimation value of a clean speech signal according to the power spectrum of the speech signal to be enhanced and the power spectrum estimation value of the noise signal;

[0009] Determining masking thresholds in different frequency bands according to power spectrum estimation values ​​of the clean speech signal;

[0010] Determine an enhanced power spectrum estimation value of a clean speech signal according to the masking threshold and the power spectrum of the speech signal to be enhanced;

[0011] An enhanced speech signal is determined according to the phase spectrum of the speech signal to be enhanced and the enhanced power spectrum estimation value of the pure speech signal.

[0012] In a second aspect, an embodiment of the present invention further provides a speech enhancement device, comprising:

[0013] A parameter acquisition module, used to acquire a power spectrum and a phase spectrum of a speech signal to be enhanced, and a power spectrum estimation value of a noise signal in the speech signal to be enhanced, wherein the speech signal to be enhanced includes a pure speech signal and a noise signal;

[0014] A pure tone power spectrum estimation value determination module, used to determine a power spectrum estimation value of a pure speech signal according to the power spectrum of the speech signal to be enhanced and the power spectrum estimation value of the noise signal;

[0015] A masking threshold determination module, used to determine the masking thresholds in different frequency bands according to the power spectrum estimation value of the clean speech signal;

[0016] A pure tone enhanced power spectrum value determination module, used to determine an enhanced power spectrum estimation value of a pure speech signal according to the masking threshold and the power spectrum of the speech signal to be enhanced;

[0017] The enhanced speech signal determination module is used to determine the enhanced speech signal according to the phase spectrum of the speech signal to be enhanced and the enhanced power spectrum estimation value of the pure speech signal.

[0018] In a third aspect, an embodiment of the present invention further provides an electronic device, including:

[0019] one or more processors;

[0020] a storage device for storing one or more programs,

[0021] When one or more programs are executed by one or more processors, the one or more processors implement the speech enhancement method as in any embodiment of the present invention.

[0022] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a speech enhancement method as in any embodiment of the present invention.

[0023] The embodiment of the present invention discloses a method for speech enhancement. The method comprises: obtaining the power spectrum and phase spectrum of a speech signal to be enhanced, and the power spectrum estimation value of a noise signal in the speech signal to be enhanced, wherein the speech signal to be enhanced comprises a clean speech signal and a noise signal; determining the power spectrum estimation value of the clean speech signal according to the power spectrum of the speech signal to be enhanced and the power spectrum estimation value of the noise signal; determining the masking threshold in different frequency bands according to the power spectrum estimation value of the clean speech signal; determining the enhanced power spectrum estimation value of the clean speech signal according to the masking threshold and the power spectrum of the speech signal to be enhanced; determining the enhanced speech signal according to the phase spectrum of the speech signal to be enhanced and the enhanced power spectrum estimation value of the clean speech signal. The embodiment of the present invention can enhance the noise suppression result, improve the signal-to-noise ratio of the speech signal to be enhanced and the speech recognition effect by determining the masking threshold of different frequency bands of the speech signal to be enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a flow chart of a speech enhancement method provided by Embodiment 1 of the present invention;

[0025] Figure 2 is a structural schematic diagram of a speech enhancement device provided in Embodiment 2 of the present invention;

[0026] Figure 3 It is a structural schematic diagram of an electronic device provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0027] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only the parts related to the present application, rather than all structures, are shown in the accompanying drawings.

[0028] It should be mentioned before discussing the exemplary embodiments in more detail that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the steps as sequential processes, many of the steps therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the steps can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0029] Embodiment 1

[0030] Figure 11 is a flow chart of a speech enhancement method provided in the first embodiment of the present invention. This embodiment is applicable to the case where speech enhancement is performed on a speech signal to be enhanced. The method can be performed by a speech enhancement device, which can be implemented in software and / or hardware and can be configured in an electronic device, such as a background server or other device with communication and computing capabilities. Figure 1 As shown, the method specifically includes:

[0031] S110, obtaining a power spectrum and a phase spectrum of a speech signal to be enhanced, and a power spectrum estimation value of a noise signal in the speech signal to be enhanced, wherein the speech signal to be enhanced includes a pure speech signal and a noise signal.

[0032] The speech signal to be enhanced is acquired by at least one speech acquisition device in the speech acquisition site. The speech acquisition site may be a communication site in a noisy environment such as a conference room, a broadcasting studio, or a railway station, or a military communication site or a speech recognition site. For example, when a broadcaster is reporting the news, various sounds may appear in the broadcasting studio, such as traffic noise generated by passing vehicles outside the broadcasting studio building, or noise generated by the air conditioning system, lighting control system, camera, and staff walking back and forth in the broadcasting studio building. At this time, it is necessary to collect the speech signal in the broadcasting studio and perform speech enhancement on the announcer's speech signal.

[0033] The voice collection device may be a microphone or a detector. Specifically, there is no limit to the number of the voice collection devices, which may be one or more. When the number of the voice collection devices is two or more, in order to collect voice signals at different positions, there is no limit to the arrangement of the voice collection devices. For example, the voice collection devices may be arranged along the circumferential direction of the pure voice signal source in the voice signal to be enhanced. In addition, since there is uncertainty and randomness in the noise interference in the voice signal to be enhanced, the voice collection device may continuously collect the voice signal to be enhanced, or may intermittently collect the voice signal at shorter intervals.

[0034] Furthermore, in order to better perform speech enhancement processing on the speech signal to be enhanced, the collected speech signal to be enhanced needs to be converted into a frequency domain sound signal. For example, the speech signal to be enhanced can be converted into a frequency domain signal by using a Fourier transform or the like. The power spectrum is the power of the speech signal to be enhanced within a unit frequency band, and the phase spectrum is the phase of the speech signal to be enhanced within a unit frequency band. The power spectrum and phase spectrum of the speech signal to be enhanced can also be obtained by performing FFT (Fast Fourier Transform) on the speech signal to be enhanced obtained by the above collection.

[0035] Furthermore, the power spectrum estimation value of the noise signal in the speech signal to be enhanced is obtained by estimating the noise signal in the collected speech signal to be enhanced and processing it through FFT. Among them, the estimation method of the noise signal can be the following three: recursive average noise estimation algorithm, minimum tracking algorithm and histogram noise estimation algorithm.

[0036] It should be noted that the speech signal to be enhanced includes a pure speech signal and a noise signal. The pure speech signal refers to the desired speech signal, and the noise signal refers to all interference signals except the desired speech signal. For example, when a broadcaster is reporting news in a studio, the voice signal of the broadcaster is a pure speech signal, and the sounds generated by the air conditioning system, lighting control system, camera, and staff walking back and forth in the studio, as well as the sounds generated by vehicles passing by outside the studio, are noise signals.

[0037] S120: Determine a power spectrum estimation value of a clean speech signal according to the power spectrum of the speech signal to be enhanced and the power spectrum estimation value of the noise signal.

[0038] The power spectrum estimation value of the clean speech signal can be obtained by using spectrum subtraction, which is based on the assumption that the noise signal and the clean speech signal are uncorrelated and that the noise signal is stable, and is obtained by subtracting the power spectrum estimation value of the speech signal to be enhanced from the power spectrum estimation value of the noise signal.

[0039] Exemplarily, it can be assumed that the noise signal is stationary, that is, the expected value of the power spectrum of the noise signal during the period with a clean voice signal and during the period without a clean voice signal is equal; then, the power spectrum of the noise signal measured and calculated during the period without a clean voice signal is replaced with the power spectrum of the noise signal during the period with a clean voice signal; finally, the power spectrum of the voice signal to be enhanced is subtracted from the power spectrum estimate of the noise signal to obtain the power spectrum estimate of the clean voice signal. It can be understood that when the above difference is a negative value, it needs to be reset to zero.

[0040] S130: Determine masking thresholds in different frequency bands according to the power spectrum estimation value of the clean speech signal.

[0041] It should be noted that within a short period of time (e.g., 10ms to 30ms), the shape of a person's vocal cords and vocal tract is relatively stable, so the short-time spectrum of the acquired human voice signal is relatively stable. However, the speech acquisition site may be in a non-stationary environment. In order to ensure that the noise signal in the speech signal to be enhanced can be effectively suppressed in a non-stationary environment, it is necessary to divide the speech signal to be enhanced into several frequency bands according to its frequency domain. It is understandable that the pure speech signal (e.g., human voice signal) in each divided frequency band is quasi-stable.

[0042] The frequency band division method can be based on the Bark scale or the Mel scale. The Bark scale is the unit of perceived frequency. In Hertz, the frequency of the speech signal to be enhanced is mapped to 24 critical frequency bands of psychoacoustics. The width of each critical frequency band is a Bark scale. When the frequency band is divided by the Bark scale, the physical frequency needs to be converted into the psychoacoustic frequency. The Mel scale is a frequency band division method that is closer to the human auditory system.

[0043] For example, for different dedicated devices, the method of dividing the frequency band of the enhanced speech signal can be selected according to the application scenario. For example, when a broadcaster is reporting news in a studio, since the collected sound signals of each broadcaster and the noise signal in the studio are relatively stable, the frequency domain of the enhanced speech signal can be divided into 26 frequency bands according to the Mel scale.

[0044] In this embodiment, optionally, a masking threshold is determined based on the power spectrum estimation value of the clean speech signal, including: obtaining the values ​​of the power spectrum estimation value of the clean speech signal in two adjacent iterations in an iterative calculation; determining a parameter value of a frequency band based on the values ​​of two adjacent iterations of the power spectrum estimation value of the clean speech signal; and determining a masking threshold based on the parameter value of the frequency band and the power spectrum estimation value of the noise signal.

[0045] Wherein, the iterative calculation is a repetition of a set of instructions or certain steps, and the iterative relationship is iteratively calculated by limiting the iteration condition or the number of iterations to obtain the iteration variable. The acquisition method of the power spectrum estimation value of the clean speech signal in the iterative calculation between two adjacent iterations can be based on the period of the power spectrum estimation value of the clean speech signal. The parameter value of the frequency band is an estimated value that can show the proportion of the clean speech signal in the noisy speech signal. It can be an empirical value obtained from multiple experiments, or it can be obtained by calculating the relationship based on the values ​​of two adjacent iterations of the power spectrum estimation value of the clean speech signal. Of course, it is understandable that when there are multiple tones in the noisy speech signal at the same time, a masking effect will occur when the high tone completely covers the low tone. In addition, the masking threshold can be determined in a variety of ways. For example, the masking threshold can be output by establishing a neural network model, or the masking threshold can be calculated by establishing a relationship.

[0046] In this embodiment, optionally, determining the parameter value of the frequency band according to two adjacent iterative values ​​of the power spectrum estimation value of the clean speech signal includes:

[0047] The parameter values ​​of the frequency band are determined using the following formula:

[0048]

[0049] Where j is the number of iterations, i is the number of bands, is the power spectrum estimate of the pure speech signal in the i-th frequency band in the j-th iteration, is the power spectrum estimate of the pure speech signal in the i-th frequency band in the (j-1)th iteration, α j (i) is the parameter value of the i-th frequency band in the j-th iteration.

[0050] It can be understood that the parameter value is calculated by taking the values ​​of two adjacent iterations of the power spectrum estimation value of the pure speech signal in the iterative calculation. The advantage of this setting is that the parameter values ​​α of different frequency bands can be obtained. j (i) Using parameter values ​​of different frequency bands to calculate the masking thresholds of different frequency bands can make the masking threshold results more accurate.

[0051] In this embodiment, optionally, determining the masking threshold according to the parameter value of the frequency band and the power spectrum estimation value of the noise signal includes:

[0052] The masking threshold is determined using the following formula:

[0053]

[0054] Among them, N j (i) is the power spectrum estimate of the noise signal in the i-th frequency band in the j-th iteration, N (j-1) (i) is the power spectrum estimate of the noise signal in the i-th frequency band in the (j-1)th iteration, T j (i) is the masking threshold for the i-th frequency band in the j-th iteration.

[0055] It can be understood that in the above formula, the masking threshold T is set j Initial value of (i) By utilizing the human auditory effect to divide different frequency bands and then calculating the masking thresholds of different frequency bands, the noise signals of each frequency band can be suppressed according to the masking thresholds of different frequency bands, thereby improving the signal-to-noise ratio of the speech signal to be enhanced.

[0056] S140: Determine an enhanced power spectrum estimation value of the clean speech signal according to the masking threshold and the power spectrum of the speech signal to be enhanced.

[0057] The enhanced power spectrum estimation value of the clean speech signal is a power spectrum estimation value obtained after effectively suppressing the noise signal in the speech signal to be enhanced. Of course, there is no unique way to determine the enhanced power spectrum estimation value of the clean speech signal. For example, it can be determined by spectral subtraction, by Wiener filtering, or by using a masking threshold.

[0058] In this embodiment, optionally, determining the enhanced power spectrum estimation value of the clean speech signal according to the masking threshold and the power spectrum of the speech signal to be enhanced includes:

[0059] Determining a proportion parameter according to the masking threshold and the power spectrum of the speech signal to be enhanced;

[0060] An enhanced power spectrum estimation value of the pure speech signal is determined according to the proportion parameter and the power spectrum of the speech signal to be enhanced.

[0061] The proportion parameter is the proportion of the enhanced power spectrum estimation value of the pure speech signal in the speech signal to be enhanced.

[0062] In this embodiment, optionally, the following formula is used to determine the enhanced power spectrum estimation value of the clean speech signal:

[0063]

[0064] in, is the enhanced power spectrum estimate of the pure speech signal, P y (ω k ) is the power of the speech signal to be enhanced at the kth frequency point, α(ω k ) is the parameter value at the kth frequency point, μ(ω k ) is the adjustment value at the kth frequency point, is the proportion parameter.

[0065] It can be understood that the parameter value α(ω k ) is the adaptive adjustment parameter value of the enhanced power spectrum estimation value of the clean speech signal, which can be obtained according to experience or by calculation. The adjustment value μ(ω k ) is the enhanced power spectrum estimate of the output clean speech signal By using the Wiener-type function to adjust the speech signal to be enhanced, the problem of spectrum loss caused by excessive suppression of the speech signal to be enhanced by traditional algorithms such as spectral subtraction and Wiener filtering is avoided, the noise suppression result can be enhanced, the signal-to-noise ratio of the speech signal to be enhanced and the speech recognition effect can be improved, which is also beneficial to human subjective hearing and later speech recognition applications.

[0066] In this embodiment, optionally, the parameter value of the kth frequency point in the iterative process is determined by the following formula:

[0067]

[0068] By using the human auditory masking threshold, the parameter value of each frequency point is obtained, so that the enhanced power spectrum estimation value of the pure speech signal output according to the parameter value is more accurate.

[0069] S150: Determine an enhanced speech signal according to the phase spectrum of the speech signal to be enhanced and the enhanced power spectrum estimation value of the pure speech signal.

[0070] The enhanced speech signal is a high signal-to-noise ratio speech signal obtained after signal processing of the speech to be enhanced. The enhanced speech signal can be determined by performing IFFT (Inverse Fast Fourier Transform) calculation based on the phase spectrum of the speech signal to be enhanced and the enhanced power spectrum estimation value of the pure speech signal.

[0071] The technical solution of the embodiment of the present invention is to obtain the power spectrum and phase spectrum of the speech signal to be enhanced, and the power spectrum estimation value of the noise signal in the speech signal to be enhanced, wherein the speech signal to be enhanced includes a clean speech signal and a noise signal; determine the power spectrum estimation value of the clean speech signal according to the power spectrum of the speech signal to be enhanced and the power spectrum estimation value of the noise signal; determine the masking threshold in different frequency bands according to the power spectrum estimation value of the clean speech signal; determine the enhanced power spectrum estimation value of the clean speech signal according to the masking threshold and the power spectrum of the speech signal to be enhanced; and determine the enhanced speech signal according to the phase spectrum of the speech signal to be enhanced and the enhanced power spectrum estimation value of the clean speech signal. The embodiment of the present invention can enhance the noise suppression result and improve the signal-to-noise ratio of the speech signal to be enhanced and the speech recognition effect by determining the masking thresholds of different frequency bands of the speech signal to be enhanced.

[0072] Embodiment 2

[0073] Figure 2 Schematic diagram of the structure of a speech enhancement device provided by Embodiment 2 of the present invention. This embodiment is applicable to speech enhancement and speech recognition. Figure 2 As shown, the device comprises:

[0074] The parameter acquisition module 210 is used to acquire the power spectrum and phase spectrum of the speech signal to be enhanced, and the power spectrum estimation value of the noise signal in the speech signal to be enhanced, wherein the speech signal to be enhanced includes a pure speech signal and a noise signal;

[0075] A pure tone power spectrum estimation value determination module 220, configured to determine a power spectrum estimation value of a pure speech signal according to the power spectrum of the speech signal to be enhanced and the power spectrum estimation value of the noise signal;

[0076] A masking threshold determination module 230, configured to determine masking thresholds in different frequency bands according to the power spectrum estimation value of the clean speech signal;

[0077] A pure tone enhanced power spectrum value determination module 240, configured to determine an enhanced power spectrum estimation value of a pure speech signal according to the masking threshold and the power spectrum of the speech signal to be enhanced;

[0078] The enhanced speech signal determination module 250 is used to determine the enhanced speech signal according to the phase spectrum of the speech signal to be enhanced and the enhanced power spectrum estimation value of the pure speech signal.

[0079] The speech enhancement device provided by the embodiment of the present invention first obtains the power spectrum and phase spectrum of the speech signal to be enhanced, and the power spectrum estimation value of the noise signal in the speech signal to be enhanced, wherein the speech signal to be enhanced includes a clean speech signal and a noise signal; then determines the power spectrum estimation value of the clean speech signal according to the power spectrum of the speech signal to be enhanced and the power spectrum estimation value of the noise signal; then determines the masking threshold in different frequency bands according to the power spectrum estimation value of the clean speech signal; then determines the enhanced power spectrum estimation value of the clean speech signal according to the masking threshold and the power spectrum of the speech signal to be enhanced; finally, determines the enhanced speech signal according to the phase spectrum of the speech signal to be enhanced and the enhanced power spectrum estimation value of the clean speech signal. The embodiment of the present invention can enhance the noise suppression result and improve the signal-to-noise ratio of the speech signal to be enhanced and the speech recognition effect by determining the masking threshold of different frequency bands of the speech signal to be enhanced.

[0080] Furthermore, the masking threshold determination module 230 includes:

[0081] An iterative value acquisition unit, used to acquire the values ​​of two adjacent iterations of the power spectrum estimation value of the pure speech signal in the iterative calculation;

[0082] A parameter value determination unit, used to determine the parameter value of the frequency band according to the values ​​of two adjacent iterations of the power spectrum estimation value of the clean speech signal;

[0083] The masking threshold determination unit is used to determine the masking threshold according to the parameter value of the frequency band and the power spectrum estimation value of the noise signal.

[0084] Furthermore, the parameter value determination unit is specifically used to:

[0085] The parameter values ​​of the frequency band are determined using the following formula:

[0086]

[0087] Where j is the number of iterations, i is the number of bands, is the power spectrum estimate of the pure speech signal in the i-th frequency band in the j-th iteration, is the power spectrum estimate of the pure speech signal in the i-th frequency band in the (j-1)th iteration, α j (i) is the parameter value of the i-th frequency band in the j-th iteration.

[0088] Furthermore, the masking threshold determination unit is specifically used to:

[0089] The masking threshold is determined using the following formula:

[0090]

[0091] Among them, N j (i) is the power spectrum estimate of the noise signal in the i-th frequency band in the j-th iteration, N (j-1) (i) is the power spectrum estimate of the noise signal in the i-th frequency band in the (j-1)th iteration, T j (i) is the masking threshold for the i-th frequency band in the j-th iteration.

[0092] Furthermore, the pure tone enhancement power spectrum value determination module 240 includes:

[0093] A proportion parameter determination unit, configured to determine a proportion parameter according to the masking threshold and the power spectrum of the speech signal to be enhanced;

[0094] The pure tone enhanced power spectrum value determination unit is used to determine the enhanced power spectrum estimation value of the pure speech signal according to the proportion parameter and the power spectrum of the speech signal to be enhanced.

[0095] Furthermore, the pure tone enhancement power spectrum value determination unit is specifically used to:

[0096] The enhanced power spectrum estimate of the clean speech signal is determined using the following formula:

[0097]

[0098] in, is the enhanced power spectrum estimate of the pure speech signal, P y (ω k ) is the power of the speech signal to be enhanced at the kth frequency point, α(ω k ) is the parameter value at the kth frequency point, μ(ω k ) is the adjustment value at the kth frequency point, is the proportion parameter.

[0099] Furthermore, the parameter value of the kth frequency point in the iterative process is determined by the following formula:

[0100]

[0101] The speech enhancement device provided in the embodiment of the present invention can execute the speech enhancement method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the speech enhancement method.

[0102] Embodiment 3

[0103] Figure 3 is a schematic diagram of the structure of an electronic device provided by Embodiment 3 of the present invention, such as Figure 3 As shown, the device includes a processor 310, a memory 320, an input device 330 and an output device 340; the number of processors 310 in the device can be one or more. Figure 3 A processor 310 is taken as an example; the processor 310, memory 320, input device 330 and output device 340 in the device can be connected via a bus or other means. Figure 3 The example of connecting through bus is taken in the following.

[0104] The memory in the device is a computer-readable storage medium and can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the speech enhancement method in the embodiment of the present invention (for example, parameter acquisition module 210, pure tone power spectrum estimation value determination module 220, masking threshold determination module 230, pure tone enhancement power spectrum value determination module 240 and enhanced speech signal determination module 250). The processor 310 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 320, that is, realizing the above-mentioned speech enhancement method.

[0105] The memory 320 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 320 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 320 may further include a memory remotely arranged relative to the processor 310, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0106] The input device 330 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 340 may include a display device such as a display screen.

[0107] Furthermore, when one or more programs included in the electronic device are executed by the one or more processors 310, the program performs the following operations:

[0108] Acquire a power spectrum and a phase spectrum of a speech signal to be enhanced, and a power spectrum estimation value of a noise signal in the speech signal to be enhanced, wherein the speech signal to be enhanced includes a pure speech signal and a noise signal;

[0109] Determining a power spectrum estimation value of a clean speech signal according to the power spectrum of the speech signal to be enhanced and the power spectrum estimation value of the noise signal;

[0110] Determining masking thresholds in different frequency bands according to power spectrum estimation values ​​of the clean speech signal;

[0111] Determine an enhanced power spectrum estimation value of a clean speech signal according to the masking threshold and the power spectrum of the speech signal to be enhanced;

[0112] An enhanced speech signal is determined according to the phase spectrum of the speech signal to be enhanced and the enhanced power spectrum estimation value of the pure speech signal.

[0113] The speech enhancement method, device, electronic device, and storage medium provided in the above embodiments can execute the speech enhancement method provided in any embodiment of the present application, and have the corresponding functional modules and beneficial effects of executing the method. For technical details not described in detail in the above embodiments, please refer to the speech enhancement method provided in any embodiment of the present application.

[0114] Embodiment 4

[0115] Embodiment 3 of the present invention provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the speech enhancement method provided in all the embodiments of the present invention is implemented. The method includes:

[0116] Acquire a power spectrum and a phase spectrum of a speech signal to be enhanced, and a power spectrum estimation value of a noise signal in the speech signal to be enhanced, wherein the speech signal to be enhanced includes a pure speech signal and a noise signal;

[0117] Determining a power spectrum estimation value of a clean speech signal according to the power spectrum of the speech signal to be enhanced and the power spectrum estimation value of the noise signal;

[0118] Determining masking thresholds in different frequency bands according to power spectrum estimation values ​​of the clean speech signal;

[0119] Determine an enhanced power spectrum estimation value of a clean speech signal according to the masking threshold and the power spectrum of the speech signal to be enhanced;

[0120] An enhanced speech signal is determined according to the phase spectrum of the speech signal to be enhanced and the enhanced power spectrum estimation value of the pure speech signal.

[0121] Optionally, when the program is executed by a processor, it can also be used to execute the speech enhancement method provided by any embodiment of the present invention.

[0122] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable CD-ROM, an optical storage device, a magnetic storage device, or any suitable combination of the above. The computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device.

[0123] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to: electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0124] The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, radio frequency (RF), etc., or any suitable combination of the above.

[0125] Computer program code for performing the operation of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0126] It is worth noting that in the embodiment of the above-mentioned speech enhancement device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0127] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A speech enhancement method, It is characterized in that include: Acquire a power spectrum and a phase spectrum of a speech signal to be enhanced, and a power spectrum estimation value of a noise signal in the speech signal to be enhanced, wherein the speech signal to be enhanced includes a pure speech signal and a noise signal; Determining a power spectrum estimation value of a clean speech signal according to the power spectrum of the speech signal to be enhanced and the power spectrum estimation value of the noise signal; Determining masking thresholds in different frequency bands according to power spectrum estimation values ​​of the clean speech signal; Determine an enhanced power spectrum estimation value of a clean speech signal according to the masking threshold and the power spectrum of the speech signal to be enhanced; Determine an enhanced speech signal according to the phase spectrum of the speech signal to be enhanced and the enhanced power spectrum estimation value of the pure speech signal; The step of determining the masking thresholds in different frequency bands according to the power spectrum estimation value of the clean speech signal includes: Obtaining the values ​​of the power spectrum estimation value of the pure speech signal in two adjacent iterations in the iterative calculation; Determining a parameter value of a frequency band according to two adjacent iterative values ​​of a power spectrum estimation value of the clean speech signal; Determining a masking threshold according to the parameter value of the frequency band and the power spectrum estimation value of the noise signal; The step of determining the parameter value of the frequency band according to two adjacent iterative values ​​of the power spectrum estimation value of the clean speech signal includes: The parameter values ​​of the frequency band are determined using the following formula: Where j is the number of iterations, i is the number of bands, is the power spectrum estimate of the pure speech signal in the i-th frequency band in the j-th iteration, is the power spectrum estimate of the pure speech signal in the i-th frequency band in the (j-1)th iteration, α j (i) is the parameter value of the i-th frequency band in the j-th iteration; The step of determining the masking threshold according to the parameter value of the frequency band and the power spectrum estimation value of the noise signal comprises: The masking threshold is determined using the following formula: Among them, N j (i) is the power spectrum estimate of the noise signal in the i-th frequency band in the j-th iteration, N (j-1) (i) is the power spectrum estimate of the noise signal in the i-th frequency band in the (j-1)th iteration, T j (i) is the masking threshold for the i-th frequency band in the j-th iteration.

2. The method according to claim 1, It is characterized in that Determining an enhanced power spectrum estimation value of a clean speech signal according to the masking threshold and the power spectrum of the speech signal to be enhanced includes: Determining a proportion parameter according to the masking threshold and the power spectrum of the speech signal to be enhanced; An enhanced power spectrum estimation value of the pure speech signal is determined according to the proportion parameter and the power spectrum of the speech signal to be enhanced.

3. The method according to claim 2, It is characterized in that The enhanced power spectrum estimate of the clean speech signal is determined using the following formula: in, is the enhanced power spectrum estimate of the pure speech signal, P y (ω k ) is the power of the speech signal to be enhanced at the kth frequency point, α(ω k ) is the parameter value at the kth frequency point, μ(ω k ) is the adjustment value at the kth frequency point, is the proportion parameter.

4. The method according to claim 3, It is characterized in that The parameter value of the kth frequency point is determined by the following formula: Where N(i) is the power spectrum estimate of the noise signal in the i-th frequency band, T j (i) is the masking threshold for the i-th frequency band in the j-th iteration.

5. A speech enhancement device, It is characterized in that include: A parameter acquisition module, used to acquire a power spectrum and a phase spectrum of a speech signal to be enhanced, and a power spectrum estimation value of a noise signal in the speech signal to be enhanced, wherein the speech signal to be enhanced includes a pure speech signal and a noise signal; A pure tone power spectrum estimation value determination module, used to determine a power spectrum estimation value of a pure speech signal according to the power spectrum of the speech signal to be enhanced and the power spectrum estimation value of the noise signal; A masking threshold determination module, used to determine the masking thresholds in different frequency bands according to the power spectrum estimation value of the clean speech signal; A pure tone enhanced power spectrum value determination module, used to determine an enhanced power spectrum estimation value of a pure speech signal according to the masking threshold and the power spectrum of the speech signal to be enhanced; An enhanced speech signal determination module is used to determine an enhanced speech signal according to the phase spectrum of the speech signal to be enhanced and the enhanced power spectrum estimation value of the pure speech signal; Wherein, the masking threshold determination module includes: An iterative value acquisition unit, used to acquire the values ​​of two adjacent iterations of the power spectrum estimation value of the pure speech signal in the iterative calculation; A parameter value determination unit, used to determine the parameter value of the frequency band according to the values ​​of two adjacent iterations of the power spectrum estimation value of the clean speech signal; A masking threshold determination unit, configured to determine a masking threshold according to a parameter value of the frequency band and a power spectrum estimation value of a noise signal; The parameter value determination unit is specifically used for: The parameter values ​​of the frequency band are determined using the following formula: Where j is the number of iterations, i is the number of bands, is the power spectrum estimate of the pure speech signal in the i-th frequency band in the j-th iteration, is the power spectrum estimate of the pure speech signal in the i-th frequency band in the (j-1)th iteration, α j (i) is the parameter value of the i-th frequency band in the j-th iteration; Wherein, the masking threshold determination unit is specifically used for: The masking threshold is determined using the following formula: Among them, N j (i) is the power spectrum estimate of the noise signal in the i-th frequency band in the j-th iteration, N (j-1) (i) is the power spectrum estimate of the noise signal in the i-th frequency band in the (j-1)th iteration, T j (i) is the masking threshold for the i-th frequency band in the j-th iteration.

6. An electronic device, It is characterized in that include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the speech enhancement method as described in any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the program is executed by a processor, the speech enhancement method as described in any one of claims 1 to 4 is implemented.

Images