Speech enhancement method, device and equipment

Abstract

The present application discloses a speech enhancement method, a speech enhancement model construction method, and a conference device. The speech enhancement method constructs a speech enhancement model based on a neural network structure inspired by auditory perception; collects a sound signal; obtains a band-compressed high-frequency signal of the sound signal; obtains a complex ideal ratio mask in the frequency domain based on the low-frequency signal of the uncompressed frequency band of the sound signal and the band-compressed high-frequency signal through the model; and obtains a speech enhancement signal based on the mask and the sound signal. In this way, the frequency band compression processing inspired by auditory perception is used, and the frequency bands are unevenly distributed. In this way, the number of neural network parameters can be effectively reduced without losing the effect performance (speech recognition), so that the network model parameters are small and the amount of calculation is small, which is more suitable for real-time voice communication, especially on devices with limited hardware resources.

Description

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on March 22, 2022, with application number 202210284720X and invention name “Speech Enhancement Method, Device and Equipment”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of speech processing technology, and in particular to a speech enhancement method and device, a speech enhancement model construction method and device, and a conference terminal. Background Art

[0003] With the popularization of audio and video communication systems, we inevitably encounter a variety of complex room acoustic environments, especially the different reverberations in different spaces (rooms, conference rooms) and different types of environmental noise (steady-state noise, non-steady-state noise). Real-time speech enhancement technology has become a difficult problem that the industry urgently needs to solve, especially in the case of single-channel and limited hardware resources.

[0004] Single-channel real-time speech enhancement aims to remove interfering audio from speech. Audio and video communication scenarios often include ambient noise, especially non-stationary noise, and spatial reverberation, thereby improving speech intelligibility. Compared to traditional single-channel speech enhancement solutions, deep learning-based algorithms are more effective in suppressing non-stationary noise and removing later reverberation. However, this comes at the cost of excessive parameters and computational complexity in the neural network model, hindering the deployment and application of neural network frameworks on low-resource devices.

[0005] In summary, real-time speech enhancement technology based on deep learning has been widely used in audio and video products. How to achieve low resources, low parameters, and low computational complexity is still a direction that the industry needs to overcome. Summary of the Invention

[0006] The present application provides a speech enhancement method to solve the problem that the existing technology consumes a lot of computing resources and cannot be applied to low-resource devices. The present application also provides a speech enhancement device and a conference terminal.

[0007] This application provides a speech enhancement method, comprising:

[0008] Constructing a speech enhancement model based on a neural network structure inspired by auditory perception; the neural network structure based on auditory perception is a neural network structure whose input data includes a low-frequency signal in an uncompressed frequency band and a high-frequency signal after frequency band compression;

[0009] Collecting sound signals;

[0010] Obtaining a high-frequency signal of the sound signal that has undergone frequency band compression;

[0011] Obtaining a complex ideal ratio mask in the frequency domain based on the low-frequency signal of the uncompressed frequency band and the high-frequency signal after frequency band compression of the sound signal through the model;

[0012] A speech enhancement signal is obtained according to the mask and the sound signal.

[0013] Optionally, the frequency-band-compressed high-frequency signal includes: a Mel filter bank, a Bark filter bank, and an equivalent rectangular bandwidth ERB filter bank.

[0014] Optionally, obtaining a speech enhancement signal according to the mask and the sound signal includes:

[0015] Acquire a first speech enhancement signal according to the mask and the sound signal;

[0016] Perform frequency band expansion processing on the high frequency signal included in the first speech enhancement signal.

[0017] Optionally, also include:

[0018] Converting the sound signal into a time-frequency domain sound signal;

[0019] The obtaining of the high-frequency signal of the sound signal after frequency band compression includes:

[0020] Acquire a high-frequency signal of the time-frequency domain sound signal that has undergone frequency band compression;

[0021] The method of obtaining a complex ideal ratio mask in the frequency domain based on the low-frequency signal of the uncompressed frequency band and the high-frequency signal after frequency band compression of the sound signal by the model includes:

[0022] Obtaining a complex ideal ratio mask in the frequency domain based on the low-frequency signal of the uncompressed frequency band and the high-frequency signal after frequency band compression of the time-frequency domain sound signal through the model;

[0023] The method further comprises:

[0024] The speech enhancement signal is converted into a time-domain speech enhancement signal.

[0025] Optionally, the sound signal includes: a single-channel sound signal.

[0026] The present application also provides a speech processing device, comprising:

[0027] A model construction unit, configured to construct a speech enhancement model based on a neural network structure inspired by auditory perception; the neural network structure based on auditory perception is a neural network structure whose input data includes a low-frequency signal in an uncompressed frequency band and a high-frequency signal after frequency band compression;

[0028] A sound signal collecting unit, used for collecting sound signals;

[0029] A high-frequency signal band compression unit, configured to obtain a band-compressed high-frequency signal of the sound signal;

[0030] A model prediction unit, configured to obtain a complex ideal ratio mask in the frequency domain based on the low-frequency signal of the uncompressed frequency band and the high-frequency signal after frequency band compression of the sound signal through the model;

[0031] The speech enhancement signal acquisition unit is configured to acquire a speech enhancement signal according to the mask and the sound signal.

[0032] Optionally, the speech enhancement signal acquisition unit is specifically configured to acquire a first speech enhancement signal according to the mask and the sound signal; and perform frequency band expansion processing on a high-frequency signal included in the first speech enhancement signal.

[0033] Optionally, also include:

[0034] A time-frequency domain conversion unit, configured to convert the sound signal into a time-frequency domain sound signal;

[0035] The high-frequency signal band compression unit is specifically used to obtain the high-frequency signal of the time-frequency domain sound signal after the band compression;

[0036] The model prediction unit is specifically configured to obtain a complex ideal ratio mask in the frequency domain based on the low-frequency signal of the uncompressed frequency band and the high-frequency signal after frequency band compression of the time-frequency domain sound signal through the model;

[0037] The inverse time-frequency domain conversion unit is used to convert the speech enhancement signal into a time-domain speech enhancement signal.

[0038] This application also provides a conference device, including:

[0039] speaker;

[0040] microphone array;

[0041] processor; and

[0042] The memory is used to store a program for implementing the above-mentioned voice enhancement method. The terminal is powered on and runs the program of the method through the processor.

[0043] This application also provides a method for constructing a speech enhancement model, comprising:

[0044] Acquire a training data set; the training data includes a noisy sound signal and a speech enhancement signal, wherein the noisy sound signal includes a low-frequency signal in an uncompressed frequency band and a high-frequency signal after frequency band compression;

[0045] Construct the network structure of the speech enhancement model;

[0046] The network parameters of the speech enhancement model are trained according to the training data set.

[0047] The present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the above-mentioned various methods.

[0048] The present application also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to perform the above methods.

[0049] Compared with the prior art, this application has the following advantages:

[0050] The speech enhancement method provided in the embodiment of the present application is constructed by constructing a speech enhancement model based on a neural network structure inspired by auditory perception; the neural network structure inspired by auditory perception refers to a neural network structure whose input data includes a low-frequency signal of an uncompressed frequency band and a high-frequency signal that has been compressed by a frequency band; collecting a sound signal; obtaining a high-frequency signal of the sound signal that has been compressed by a frequency band; using the model, obtaining a complex ideal ratio mask in the frequency domain based on the low-frequency signal of the uncompressed frequency band and the high-frequency signal that has been compressed by a frequency band; and obtaining a speech enhancement signal based on the mask and the sound signal. In this way, the frequency band compression processing inspired by auditory perception is used, and the frequency bands are unevenly distributed. In this way, the number of neural network parameters can be effectively reduced without sacrificing the effect performance (speech recognition), so that the network model parameters are small and the amount of calculation is small, which is more suitable for real-time voice communication, especially on devices with limited hardware resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A flowchart of an embodiment of the speech enhancement method provided in this application;

[0052] Figure 2 A schematic diagram of a specific flow chart of an embodiment of the speech enhancement method provided in this application;

[0053] Figure 3 A schematic diagram of a model training embodiment of the speech enhancement method provided in this application;

[0054] Figure 4 Mel feature diagram of an embodiment of the speech enhancement method provided by this application;

[0055] Figure 5 Bark feature diagram of an embodiment of the speech enhancement method provided in this application;

[0056] Figure 6 Schematic diagram of ERB features of an embodiment of the speech enhancement method provided in this application;

[0057] Figure 7 A flowchart of an embodiment of a method for constructing a speech enhancement model provided in this application. DETAILED DESCRIPTION

[0058] The following description sets forth many specific details to facilitate a thorough understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of the present application. Therefore, the present application is not limited to the specific implementations disclosed below.

[0059] This application provides a speech enhancement method and apparatus, and a conference terminal. In the following embodiments, various solutions are described in detail one by one.

[0060] In the process of implementing the present invention, the inventors found that the existing solutions for speech enhancement using a neural network framework have at least the following problems: the frequency bands are evenly distributed, and are processed uniformly from low frequency to high frequency, resulting in a large number of required frequency bands, which directly affects the dimension of the neural network input and the follow-up of the dimensions of subsequent layers of the neural network to more accurately estimate the target output.

[0061] First embodiment

[0062] Please see Figure 1 , which is a flow chart of an embodiment of the speech enhancement method of the present application. In this embodiment, the method may include the following steps:

[0063] Step S101: Construct a speech enhancement model based on a neural network structure inspired by auditory perception.

[0064] The method can be applied to speaker location scenarios in conference terminal devices within audio and video conferencing systems. An audio and video conferencing system is a system where two or more individuals or groups in different locations use transmission lines and conference terminals to exchange audio, video, and file data, enabling instant and interactive communication and simultaneous conferencing. Therefore, an audio and video conferencing system is a typical real-time communication system.

[0065] The conference device may be a device with relatively low computing resource configuration.

[0066] In the method provided herein, the speech enhancement model employs a neural network structure inspired by auditory perception. This neural network structure, whose input data includes low-frequency signals in an uncompressed frequency band and high-frequency signals that have undergone frequency band compression, is used to estimate a complex ideal ratio mask M in the frequency domain.

[0067] Auditory perception heuristics refer to the study of the human auditory system to obtain the basic laws of speech perception. The method provided in this application applies this law to speech enhancement technology. Based on the inspiration of auditory perception, that is, the human auditory system has a higher perception of low frequencies and decreases in sensitivity as it moves towards high frequencies. Based on this law, the method provided in this application does not require a lot of evenly distributed frequency band accuracy for high-frequency bands. Instead, it can integrate a small number of frequency bands (such as one or two).

[0068] The high-frequency signal that has undergone frequency band compression includes but is not limited to at least one of the following: a Mel filter bank, a Bark filter bank, and an equivalent rectangular bandwidth ERB filter bank.

[0069] The human ear's sensitivity to sound varies with frequency, and the relationship between sensitivity and frequency is not simply linearly proportional, but rather approximately logarithmically. To better approximate the human ear's sound pickup characteristics, the frequency can be nonlinearly transformed to the Mel scale (or Bark scale or ERB scale), then divided into M equally spaced segments, and then back-transformed to the frequency domain. In practice, a set of bandpass filters can be used to extract features within these frequency segments.

[0070] like Figure 4 As shown, the Mel scale describes the nonlinear transformation of the human ear's frequency perception. Specifically, the following method can be used: select the required number M of Mel filters, convert the lowest frequency and cutoff frequency into Mel values ​​to obtain low_Mel and high_Mel, respectively; divide low_Mel and high_Mel into M equal parts; then, reverse-convert the equally divided Mel values ​​to obtain frequency values.

[0071] like Figure 5 As shown in , the Bark scale is another nonlinear scale used to describe the human ear's perception of frequency. Figure 6 As shown in Figure 2, the equivalent rectangular bandwidth (ERB) can describe the relationship between frequency-aware nonlinear transformations.

[0072] In one example, step S101 may include the following sub-steps:

[0073] Step S1011: Obtain a training data set.

[0074] The training data includes a noisy sound signal and a speech enhancement signal, wherein the noisy sound signal includes a low-frequency signal in an uncompressed frequency band and a high-frequency signal in a frequency band compressed.

[0075] Step S1013: Constructing a network structure of a speech enhancement model.

[0076] The network structure of the speech enhancement model can adopt network structures such as DNN, CNN, LSTM, SubNet, etc.

[0077] Step S1015: training the network parameters of the speech enhancement model according to the training data set.

[0078] The noisy sound signal is used as the input data of the model, and the corresponding speech enhancement signal is used as the output data of the model. The network parameters of the model can be trained through machine learning.

[0079] like Figure 3 As shown, in specific implementation, the spectrum compression module, neural network and spectrum expansion module can be trained together as a whole to realize an end-to-end speech enhancement model.

[0080] During specific implementation, the constructed model can be deployed to conference equipment. Since the model has the characteristics of small network model parameters and small computational complexity, it is more suitable for real-time voice communication, especially on conference equipment with limited hardware resources.

[0081] Step S103: collecting sound signals.

[0082] The sound signal may be a single-channel sound signal or a multi-channel sound signal.

[0083] In this embodiment, the sound signal collected by the microphone is a time-domain speech signal, which can be expressed as the following formula:

[0084] y(t)=x(t)+n(t)=s(t)*h(t)+n(t),

[0085] Where x is the speech signal with reverberation, s is the clean speech signal, n is the ambient noise, and h is the spatial impulse response (used to describe the reverberation effect).

[0086] Step S105: obtaining a high-frequency signal of the sound signal that has undergone frequency band compression.

[0087] The high-frequency signal of the sound signal that has undergone frequency band compression may include a Mel filter bank, a Bark filter bank, an equivalent rectangular bandwidth ERB filter bank, and the like.

[0088] The processing method for obtaining the high-frequency signal of the sound signal after frequency band compression has been described in step S101 and will not be repeated here.

[0089] Step S107: Obtaining a complex ideal ratio mask in the frequency domain through the model according to the low-frequency signal of the uncompressed frequency band and the high-frequency signal after frequency band compression of the sound signal.

[0090] The input data of the model includes low-frequency signals in the uncompressed frequency band and high-frequency signals after frequency band compression. This step uses the speech enhancement model constructed in step S101 to process the sound signal to be processed and obtain a complex ideal ratio mask in the frequency domain.

[0091] Step S109: Acquire a speech enhancement signal according to the masking and the sound signal.

[0092] According to the masking and the sound signal, a speech enhancement signal is obtained, which can be expressed as the following formula:

[0093]

[0094] Wherein, M represents the mask, Y represents the sound signal, and S represents the speech enhancement signal. Since obtaining the speech enhancement signal based on the mask and the noisy sound signal belongs to the prior art, it will not be described in detail here.

[0095] like Figure 2 As shown, the method provided in the embodiment of the present application may further include the following steps: converting the sound signal into a time-frequency domain sound signal. In this embodiment, the time-domain speech signal collected by the microphone is subjected to a short-time Fourier transform (STFT) to obtain a speech spectrum Y, which can be expressed as the following formula:

[0096] Y(t, f) = X(t, f) + N(t, f).

[0097] Since the time-frequency domain transformation of the sound signal belongs to the existing technology, it will not be described here in detail.

[0098] Accordingly, step S105 may be implemented in the following manner: obtaining a high-frequency signal of the time-frequency domain sound signal that has undergone frequency band compression.

[0099] Accordingly, step S107 can be implemented as follows: using the model, a complex ideal ratio mask in the frequency domain is obtained based on the low-frequency signal of the uncompressed frequency band and the high-frequency signal after frequency band compression of the time-frequency domain sound signal.

[0100] Accordingly, the method provided in the embodiment of the present application may further include the following step: converting the speech enhancement signal into a time-domain speech enhancement signal. In this embodiment, the processed time-domain speech signal, i.e., the single-channel enhanced speech signal, is obtained after inverse short-time Fourier transform (iSTFT) is performed.

[0101] In one example, step S109 can be implemented as follows: obtaining a first speech enhancement signal based on the mask and the sound signal; and performing frequency band expansion processing on the high-frequency signal included in the first speech enhancement signal. The frequency band expansion processing method has been described in step S101 and will not be repeated here.

[0102] Experiments have shown that by adopting the method provided in the embodiments of the present application, without affecting the performance (speech recognition), the input dimension of the neural network can be reduced by more than 20 times through uneven distribution of frequency bands, and the hidden layer and output layer can also reduce the corresponding dimensions, and the computational complexity can be reduced by more than 15 times.

[0103] As can be seen from the above embodiments, the speech enhancement method provided in the embodiments of the present application constructs a speech enhancement model based on a neural network structure inspired by auditory perception; the neural network structure inspired by auditory perception refers to a neural network structure whose input data includes a low-frequency signal in an uncompressed frequency band and a high-frequency signal that has been compressed by a frequency band; collects a sound signal; obtains a high-frequency signal of the sound signal that has been compressed by a frequency band; obtains a complex ideal ratio mask in the frequency domain based on the low-frequency signal in the uncompressed frequency band and the high-frequency signal that has been compressed by a frequency band through the model; and obtains a speech enhancement signal based on the mask and the sound signal. In this way, the frequency band compression processing inspired by auditory perception is used, and the frequency bands are unevenly distributed. In this way, the number of neural network parameters can be effectively reduced without sacrificing the effect performance (speech recognition), so that the network model parameters are small and the amount of calculation is small, which is more suitable for real-time voice communication, especially on devices with limited hardware resources.

[0104] Second embodiment

[0105] The above embodiments provide a method for speech enhancement. Correspondingly, this application also provides a speech enhancement device. This device corresponds to the above method embodiments. Since the device embodiments are generally similar to the method embodiments, the description is relatively simple. For relevant details, please refer to the description of the method embodiments. The device embodiments described below are merely illustrative.

[0106] The present application further provides a speech enhancement device, comprising:

[0107] A model construction unit, configured to construct a speech enhancement model based on a neural network structure inspired by auditory perception; the neural network structure based on auditory perception is a neural network structure whose input data includes a low-frequency signal in an uncompressed frequency band and a high-frequency signal after frequency band compression;

[0108] A sound signal collecting unit, used for collecting sound signals;

[0109] A high-frequency signal band compression unit, configured to obtain a band-compressed high-frequency signal of the sound signal;

[0110] A model prediction unit, configured to obtain a complex ideal ratio mask in the frequency domain based on the low-frequency signal of the uncompressed frequency band and the high-frequency signal after frequency band compression of the sound signal through the model;

[0111] The speech enhancement signal acquisition unit is configured to acquire a speech enhancement signal according to the mask and the sound signal.

[0112] Optionally, the speech enhancement signal acquisition unit is specifically configured to acquire a first speech enhancement signal according to the mask and the sound signal; and perform frequency band expansion processing on a high-frequency signal included in the first speech enhancement signal.

[0113] Optionally, also include:

[0114] A time-frequency domain conversion unit, configured to convert the sound signal into a time-frequency domain sound signal;

[0115] The high-frequency signal band compression unit is specifically used to obtain the high-frequency signal of the time-frequency domain sound signal after the band compression;

[0116] The model prediction unit is specifically configured to obtain a complex ideal ratio mask in the frequency domain based on the low-frequency signal of the uncompressed frequency band and the high-frequency signal after frequency band compression of the time-frequency domain sound signal through the model;

[0117] The inverse time-frequency domain conversion unit is used to convert the speech enhancement signal into a time-domain speech enhancement signal.

[0118] Third embodiment

[0119] In the above-mentioned embodiments, a speech enhancement method is provided. Correspondingly, this application also provides a conferencing device. This device corresponds to the above-mentioned method embodiments. Since the device embodiments are substantially similar to the method embodiments, the description is relatively simple. For relevant details, please refer to the description of the method embodiments. The device embodiments described below are merely illustrative.

[0120] The present application further provides a conference device comprising: a microphone; a processor; and a memory, wherein the memory is used to store a program for implementing the above-mentioned voice enhancement method, and the terminal is powered on and runs the program of the method through the processor.

[0121] The conference device may be an audio or video conference terminal or a sound pickup device.

[0122] Fourth embodiment

[0123] In the above-mentioned embodiment, a speech enhancement method is provided. Correspondingly, this application also provides a method for constructing a speech enhancement model. This method corresponds to the above-mentioned method embodiment. Since the method embodiment is substantially similar to the method embodiment, the description is relatively simple. For relevant details, please refer to the description of the method embodiment. The method embodiment described below is merely illustrative.

[0124] This application also provides a method for constructing a speech enhancement model, including:

[0125] Step S701: Obtain a training data set.

[0126] The training data includes a noisy sound signal and a speech enhancement signal, wherein the noisy sound signal includes a low-frequency signal in an uncompressed frequency band and a high-frequency signal in a frequency band compressed.

[0127] Step S703: Constructing a network structure of a speech enhancement model.

[0128] The network structure of the speech enhancement model can adopt network structures such as DNN, CNN, LSTM, SubNet, etc.

[0129] Step S705: training the network parameters of the speech enhancement model according to the training data set.

[0130] The noisy sound signal is used as the input data of the model, and the corresponding speech enhancement signal is used as the output data of the model. The network parameters of the model can be trained through machine learning.

[0131] During specific implementation, the constructed model can be deployed to conference equipment. Since the model has the characteristics of small network model parameters and small computational complexity, it is more suitable for real-time voice communication, especially on conference equipment with limited hardware resources.

[0132] Fifth embodiment

[0133] In the above-mentioned embodiments, a method for constructing a speech enhancement model is provided. Correspondingly, this application also provides a device for constructing a speech enhancement model. This device corresponds to the above-mentioned method embodiment. Since the device embodiment is substantially similar to the method embodiment, the description is relatively simple. For relevant details, please refer to the description of the method embodiment. The device embodiment described below is merely illustrative.

[0134] The present application further provides a speech enhancement model construction device, comprising:

[0135] A training data acquisition unit, configured to acquire a training data set; the training data includes a noisy sound signal and a speech enhancement signal, wherein the noisy sound signal includes a low-frequency signal in an uncompressed frequency band and a high-frequency signal after frequency band compression;

[0136] A network structure building unit, used to build the network structure of the speech enhancement model;

[0137] A network training unit is used to train the network parameters of the speech enhancement model according to the training data set.

[0138] Although the present application is disclosed as above with the preferred embodiments, it is not intended to limit the present application. Any person skilled in the art may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

[0139] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0140] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0141] 1. Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include non-transitory media such as modulated data signals and carrier waves.

[0142] 2. Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

Claims

1. A speech enhancement method, characterized in that: include: Constructing a speech enhancement model based on a neural network structure inspired by auditory perception; the neural network structure based on auditory perception is a neural network structure whose input data includes a low-frequency signal in an uncompressed frequency band and a high-frequency signal after frequency band compression; Collect single-channel sound signals; Obtaining a high-frequency signal of the sound signal that has undergone frequency band compression; Obtaining a complex ideal ratio mask in the frequency domain based on the low-frequency signal of the uncompressed frequency band and the high-frequency signal after frequency band compression of the sound signal through the model; A speech enhancement signal is obtained according to the mask and the sound signal.

2. The method according to claim 1, characterized in that The high-frequency signal that has undergone frequency band compression includes: a Mel filter bank, a Bark filter bank, and an equivalent rectangular bandwidth ERB filter bank.

3. The method according to claim 1, characterized in that The step of obtaining a speech enhancement signal according to the masking and the sound signal includes: Acquire a first speech enhancement signal according to the mask and the sound signal; Perform frequency band expansion processing on the high frequency signal included in the first speech enhancement signal.

4. The method according to claim 1, wherein Also includes: Converting the sound signal into a time-frequency domain sound signal; The obtaining of the high-frequency signal of the sound signal after frequency band compression includes: Acquire a high-frequency signal of the time-frequency domain sound signal that has undergone frequency band compression; The method of obtaining a complex ideal ratio mask in the frequency domain based on the low-frequency signal of the uncompressed frequency band and the high-frequency signal after frequency band compression of the sound signal by the model includes: Obtaining a complex ideal ratio mask in the frequency domain based on the low-frequency signal of the uncompressed frequency band and the high-frequency signal after frequency band compression of the time-frequency domain sound signal through the model; The method further comprises: The speech enhancement signal is converted into a time-domain speech enhancement signal.

5. A speech processing device, characterized in that: include: A model construction unit, configured to construct a speech enhancement model based on a neural network structure inspired by auditory perception; the neural network structure based on auditory perception is a neural network structure whose input data includes a low-frequency signal in an uncompressed frequency band and a high-frequency signal after frequency band compression; A sound signal acquisition unit, used for acquiring a single-channel sound signal; A high-frequency signal band compression unit, configured to obtain a band-compressed high-frequency signal of the sound signal; A model prediction unit, configured to obtain a complex ideal ratio mask in the frequency domain based on the low-frequency signal of the uncompressed frequency band and the high-frequency signal after frequency band compression of the sound signal through the model; The speech enhancement signal acquisition unit is configured to acquire a speech enhancement signal according to the mask and the sound signal.

6. The device according to claim 5, characterized in that The speech enhancement signal acquisition unit is specifically configured to acquire a first speech enhancement signal according to the mask and the sound signal; and perform frequency band expansion processing on a high-frequency signal included in the first speech enhancement signal.

7. The device according to claim 5, characterized in that Also includes: A time-frequency domain conversion unit, configured to convert the sound signal into a time-frequency domain sound signal; The high-frequency signal band compression unit is specifically used to obtain the high-frequency signal of the time-frequency domain sound signal after the band compression; The model prediction unit is specifically configured to obtain a complex ideal ratio mask in the frequency domain based on the low-frequency signal of the uncompressed frequency band and the high-frequency signal after frequency band compression of the time-frequency domain sound signal through the model; The inverse time-frequency domain conversion unit is used to convert the speech enhancement signal into a time-domain speech enhancement signal.

8. A conference device, characterized in that: processor; and a memory for storing a program for implementing the method according to any one of claims 1 to 4, wherein the conference device is powered on and runs the program of the method through the processor.

9. A method for constructing a speech enhancement model, characterized in that: include: Acquire a training data set; the training data includes a noisy sound signal and a speech enhancement signal, wherein the noisy sound signal includes a low-frequency signal in an uncompressed frequency band and a high-frequency signal after frequency band compression; Construct the network structure of the speech enhancement model; The network parameters of the speech enhancement model are trained according to the training data set.

Images