Voice noise reduction method and device, medium, product and equipment
Through the combination of pre-trained neural network and linear filter, the problems of speech noise reduction complexity and resource consumption in deep learning methods are solved, and efficient and accurate speech noise reduction is achieved, improving speech quality and response speed.
Patent Information
- Application Number
- CN202411184059.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-27
- Publication Date
- 2024-12-20
AI Technical Summary
In the existing speech noise reduction technology, deep learning methods require the design of complex algorithm structures and a large amount of memory and computing power, which is difficult to effectively reduce the complexity and resource consumption of network models.
The pre-trained neural network is used to determine the speech linear spectrum frequency and noise linear spectrum frequency based on noise band-band speech, and call the linear filter through the filter gain to reduce noise, reducing the complexity of the neural network, and is only used to determine the filter gain in real time, reducing the cost. Memory and computing power requirements.
It realizes efficient and accurate speech noise reduction, reduces network complexity and resource consumption, and improves voice quality and response timeliness.
Smart Images
Figure CN119170035A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of speech processing technology, and in particular to a speech noise reduction method, device, medium, product and equipment. Background Art
[0002] With the development of the Internet and the popularization and enrichment of mobile communication devices, the requirements for voice quality in fields such as speech recognition, voice calls, telephone conferencing, scene recording and hearing assistance are becoming increasingly higher. Among them, speech enhancement is a key technology to improve speech quality and intelligibility, and it has broad application prospects and important research value in the above-mentioned fields.
[0003] Common speech enhancement generally refers to speech noise reduction. In the prior art, speech noise reduction can use deep learning methods to directly reduce the noise of noisy speech. However, in order to accurately extract the features in the speech for direct noise reduction in the deep learning method, the algorithm structure needs to be designed to be very complex and requires a large amount of memory and computing power. Summary of the invention
[0004] In order to solve the above technical problems, the embodiments of the present application propose a speech noise reduction method, device, medium, product and equipment, which can enable the neural network to have the ability to obtain key information for noise reduction, while greatly reducing the complexity of the network model, reducing the requirements for computing power and memory, and completing speech noise reduction efficiently and accurately.
[0005] In a first aspect, an embodiment of the present application provides a method for reducing speech noise, including:
[0006] Determine the speech linear spectrum frequency and the noise linear spectrum frequency based on the noisy speech using a pre-trained neural network, wherein the neural network is trained based on sample speech data;
[0007] Determining a filter gain based on the speech linear spectrum frequency and the noise linear spectrum frequency;
[0008] Based on the filter gain, a linear filter is called to determine the noise-reduced speech matching the noisy speech.
[0009] Furthermore, before using the pre-trained neural network to determine the speech linear spectrum frequency and the noise linear spectrum frequency based on the noisy speech, the method also includes:
[0010] Acquire the noisy speech, and perform a fast Fourier transform on the noisy speech to convert the noisy speech into a logarithmic power spectrum;
[0011] The method of using a pre-trained neural network to determine the speech linear spectrum frequency and the noise linear spectrum frequency based on noisy speech includes:
[0012] Inputting the logarithmic power spectrum into the neural network to obtain the speech linear spectrum frequency and the noise linear spectrum frequency output by the neural network;
[0013] The step of calling a linear filter to determine a noise reduction speech matching the noisy speech based on the filter gain comprises:
[0014] Determining a first amplitude spectrum of the noisy speech;
[0015] Calling the linear filter, using the filter gain as a weighting coefficient, and performing weighted calculation on the values of each frequency in the first amplitude spectrum to obtain a second amplitude spectrum;
[0016] Perform inverse Fourier transform on the second amplitude spectrum to obtain the noise-reduced speech.
[0017] Furthermore, the sample speech data includes sample noisy speech, speech data corresponding to the sample noisy speech, and noise data corresponding to the sample noisy speech, and the training method of the neural network includes:
[0018] Performing a fast Fourier transform on the sample noisy speech to convert the sample noisy speech into a sample logarithmic power spectrum;
[0019] Calculating the linear spectrum frequency of the speech data and calculating the linear spectrum frequency of the noise data;
[0020] The sample logarithmic power spectrum is used as training input data, and the linear spectrum frequency of the speech data and the linear spectrum frequency of the noise data are connected as training output data. The network is trained using a stochastic gradient descent algorithm to obtain the neural network after training.
[0021] Furthermore, the neural network includes 3 hidden layers, each hidden layer includes 512 rectified linear hidden units, and the cost function of the neural network is Euclidean distance.
[0022] Furthermore, the linear filter includes a pre-constructed Wiener adaptive filter, and the construction method of the Wiener adaptive filter includes:
[0023] Converting the speech linear spectrum frequency into speech linear prediction coefficients to calculate the speech spectrum shape according to the speech linear prediction coefficients;
[0024] Converting the noise linear spectrum frequency into noise linear prediction coefficients to calculate the noise spectrum shape according to the noise linear prediction coefficients;
[0025] Based on the speech spectrum shape and the noise spectrum shape, the Wiener adaptive filter is constructed using the Gain update rule.
[0026] Further, the method of calculating the linear spectrum frequency includes:
[0027] Perform Fourier transform on the data to be calculated;
[0028] For the data to be calculated after Fourier transformation, the linear prediction coefficient is calculated, wherein the order of the linear prediction coefficient is the set order;
[0029] Determining a reflection coefficient that matches the linear prediction coefficient, and performing stability processing on the reflection coefficient;
[0030] Based on the reflection coefficient after stability processing, a linear spectrum frequency with an order equal to the set order is calculated.
[0031] In a second aspect, an embodiment of the present application provides a speech noise reduction device, comprising:
[0032] A linear spectrum frequency determination module, used to determine the speech linear spectrum frequency and the noise linear spectrum frequency based on noisy speech using a pre-trained neural network, wherein the neural network is trained based on sample speech data;
[0033] A gain determination module, used for determining a filter gain based on the speech linear spectrum frequency and the noise linear spectrum frequency;
[0034] The noise reduction module is used to call a linear filter to determine a noise reduction speech matching the noisy speech based on the filter gain.
[0035] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any of the above-mentioned speech noise reduction methods are implemented.
[0036] In a fourth aspect, an embodiment of the present application provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of any of the above-mentioned speech noise reduction methods.
[0037] In a fifth aspect, an embodiment of the present application provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the steps of any of the above-mentioned speech noise reduction methods when executing the computer program.
[0038] In summary, the embodiments of the present application have at least the following beneficial effects:
[0039] According to an embodiment of the present application, a pre-trained neural network is used to determine the speech linear spectrum frequency and the noise linear spectrum frequency based on noisy speech, wherein the neural network is trained based on sample speech data; based on the speech linear spectrum frequency and the noise linear spectrum frequency, the filter gain is determined; based on the filter gain, a linear filter is called to determine the noise reduction speech that matches the noisy speech. In this way, since noise reduction is completed through the filter, the neural network only needs to be used to determine the speech linear spectrum frequency and the noise linear spectrum frequency corresponding to the noisy speech in real time to obtain the current filter gain, without the need to extract complex speech features that can directly complete speech noise reduction. Therefore, the network complexity can be greatly reduced, and the timeliness of the speech noise reduction response can be improved, thereby ultimately improving the speech quality in different scenarios. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Figure 1 It is a flowchart of the speech noise reduction method provided in the embodiment of the present application;
[0041] Figure 2 is a schematic diagram of speech noise reduction provided by an embodiment of the present application;
[0042] Figure 3 is a schematic diagram of neural network training provided in an embodiment of the present application;
[0043] Figure 4 is a structural schematic diagram of a speech noise reduction device provided in an embodiment of the present application;
[0044] Figure 5 It is a schematic diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0045] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0046] In the description of the present application, the terms "first", "second", "third", etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, features defined as "first", "second", "third", etc. may explicitly or implicitly include one or more of the features. In the description of the present application, unless otherwise specified, the meaning of "multiple" is two or more. In the description of the present application, the term "including" and its variations are open inclusions, i.e., "including but not limited to". The term "based on" means "at least partially based on". The term "according to" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments".
[0047] In the description of this application, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.
[0048] In the description of this application, it should be noted that, unless otherwise defined, all technical and scientific terms used in this application have the same meaning as those commonly understood by those skilled in the art. The terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood by specific circumstances.
[0049] First, see Figure 1 , shows a flow chart of a speech noise reduction method provided in an embodiment of the present application, the method includes steps S101-S103, which are as follows:
[0050] S101, using a pre-trained neural network to determine a speech linear spectrum frequency and a noise linear spectrum frequency based on noisy speech, wherein the neural network is trained based on sample speech data;
[0051] Exemplarily, the neural network may be DNN (Deep Neural Networks), which is a type of neural network model with multiple hidden layers, which performs well in processing complex nonlinear relationships, especially in the fields of speech recognition, natural language processing, image recognition, etc.
[0052] S102, determining a filter gain based on the speech linear spectrum frequency and the noise linear spectrum frequency;
[0053] S103: Based on the filter gain, call a linear filter to determine a noise-reduced speech that matches the noisy speech.
[0054] In this embodiment, by combining the linear filter with the neural network, the neural network can be used to quickly extract information about non-steady-state noise and timely adjust the filter gain, thereby improving the speech noise reduction capability of the linear filter in different scenarios to improve the quality of speech.
[0055] In an optional implementation, before using the pre-trained neural network to determine the speech linear spectrum frequency and the noise linear spectrum frequency based on the noisy speech, the method further includes:
[0056] Acquire the noisy speech, and perform a fast Fourier transform on the noisy speech to convert the noisy speech into a logarithmic power spectrum;
[0057] See also Figure 2 , the noisy speech can be converted into a log power spectrum (LPS) by performing a fast Fourier transform on the noisy speech, and the converted LPS is used as the input of the DNN.
[0058] In one example, the logarithmic power spectrum of the noisy speech can be obtained as an acoustic feature. In this way, the input of the DNN can be combined with the current LPS frame, 5 previous frames and 5 future frames. Therefore, the input of the DNN can be 11 frames of LPS vectors, which are expressed as:
[0059]
[0060] Among them, m represents the LPS latitude of a frame, and l represents the index of the frame.
[0061] The method of using a pre-trained neural network to determine the speech linear spectrum frequency and the noise linear spectrum frequency based on noisy speech includes:
[0062] Inputting the logarithmic power spectrum into the neural network to obtain the speech linear spectrum frequency and the noise linear spectrum frequency output by the neural network;
[0063] The step of calling a linear filter to determine a noise reduction speech matching the noisy speech based on the filter gain comprises:
[0064] Determining a first amplitude spectrum of the noisy speech;
[0065] Calling the linear filter, using the filter gain as a weighting coefficient, and performing weighted calculation on the values of each frequency in the first amplitude spectrum to obtain a second amplitude spectrum;
[0066] Perform inverse Fourier transform on the second amplitude spectrum to obtain the noise-reduced speech.
[0067] In an optional implementation, the sample speech data includes sample noisy speech, speech data corresponding to the sample noisy speech, and noise data corresponding to the sample noisy speech, and the training method of the neural network includes:
[0068] Performing a fast Fourier transform on the sample noisy speech to convert the sample noisy speech into a sample logarithmic power spectrum;
[0069] Calculating the linear spectrum frequency of the speech data and calculating the linear spectrum frequency of the noise data;
[0070] The sample logarithmic power spectrum is used as training input data, and the linear spectrum frequency of the speech data and the linear spectrum frequency of the noise data are connected as training output data. The network is trained using a stochastic gradient descent algorithm to obtain the neural network after training.
[0071] In one example, see Figure 3 , the sample noisy speech can be converted into the corresponding sample logarithmic power spectrum according to the format of the above 11 frames of LPS vectors, so as to be used as the training input data of the DNN. In this way, the linear spectrum frequencies of the speech data and the noise data calculated separately can be connected to be used as the training output data of the DNN, wherein the training output data can be expressed as
[0072]
[0073] in The LSF (Line Spectral Frequencies) parameter representing the i-th dimension of the speech, represents the LSF parameter of the i-th dimension of the noise, where n is the dimension.
[0074] In an example, the stochastic gradient descent algorithm can be expressed as:
[0075]
[0076] Wherein, k is the training cycle index, η is the learning rate, and α is the momentum. The momentum is 0.2 in the first 20 training cycles and is increased to 0.5 in subsequent training. The learning rate is 0.00003 in the first cycle and decays in subsequent cycles by multiplying 0.98 in the next cycle. In this embodiment, the stochastic gradient descent algorithm with momentum can avoid falling into the local optimal problem, and the neural network is designed with a forgetting rate of 0.2 to prevent the problem of overfitting.
[0077] In an optional implementation, the neural network includes three hidden layers, each hidden layer includes 512 rectified linear hidden units, and the cost function of the neural network is Euclidean distance.
[0078] In one example, the Euclidean distance can be expressed as:
[0079]
[0080] where x (i) is the i-th input vector, y (i) is the vector of the i-th expected output related to the training target, h v,b () indicates that the overall DNN has connection weights v and biases b, and m is the total number of training vectors.
[0081] In an optional implementation, the linear filter includes a pre-constructed Wiener adaptive filter, and the construction method of the Wiener adaptive filter includes:
[0082] Converting the speech linear spectrum frequency into speech linear prediction coefficients to calculate the speech spectrum shape according to the speech linear prediction coefficients;
[0083] Converting the noise linear spectrum frequency into noise linear prediction coefficients to calculate the noise spectrum shape according to the noise linear prediction coefficients;
[0084] Based on the speech spectrum shape and the noise spectrum shape, the Wiener adaptive filter is constructed using the Gain update rule.
[0085] In one example, the spectral shape can be calculated by the following formula: in:
[0086]
[0087] in, is the mth dimension of the estimated speech LPC (Linear Predictive Coding) parameters, K represents the size of the fast Fourier transform, p is the order of the LPC model, and k is the frequency index.
[0088] In one example, the Wiener adaptive filter can be expressed as:
[0089]
[0090] in:
[0091]
[0092]
[0093] in is the power spectrum of the noisy speech. x , H w , H y They are the spectral shapes of speech, noise, and noisy speech, respectively.
[0094] In an optional implementation, the method of calculating the linear spectrum frequency includes:
[0095] Perform Fourier transform on the data to be calculated;
[0096] For the data to be calculated after Fourier transformation, the linear prediction coefficient is calculated, wherein the order of the linear prediction coefficient is the set order;
[0097] In one example, the linear prediction coefficient LPC may be calculated by a Levinson-Durbin recursive algorithm.
[0098] Exemplarily, the set order may be 10 orders.
[0099] Determining a reflection coefficient that matches the linear prediction coefficient, and performing stability processing on the reflection coefficient;
[0100] In one example, the reflection coefficient (RC) is calculated, which can usually be done by using the Levinson-Dubin recursive algorithm. The reflection coefficient reflects the reflection caused by the discontinuity of the boundary conditions at the glottis, and is the last value of the prediction coefficient of each order of LPC. It can be directly recursively obtained in the Levinson-Dubin recursive algorithm, and the absolute value of the reflection coefficient should be kept in the range of [-1,1] (i.e., stability processing). If the reflection coefficient exceeds this range, it needs to be adjusted to return it to the legal range to ensure the stability of the LPC model.
[0101] The stability of the reflection coefficient can be achieved by:
[0102] 1. Truncation method: directly limit the value of the reflection coefficient to the range of [-1,1].
[0103] 2. Smoothing: Reduce instability by smoothing the reflection coefficient, such as using sliding window averaging.
[0104] Based on the reflection coefficient after stability processing, a linear spectrum frequency with an order equal to the set order is calculated.
[0105] In an example, the linear spectrum frequency can be analyzed and derived by using a Chebyshev polynomial root-finding method based on the reflection coefficient after stability processing.
[0106] In some existing technologies, deep learning related methods (time-frequency masking, feature mapping, and end-to-end, etc.) are used to perform speech noise reduction, which requires the network to output complete speech or a large amount of information equivalent to it. It has extremely high complexity and requires a complex network model, and its accuracy and generalization are difficult to guarantee.
[0107] In this embodiment, based on the linear filter, a neural network can be used algorithmically to reduce the dimension of the complex amplitude spectrum to a linear spectrum frequency of a set order (for example, 10th order), retaining key data information. At the same time, the overall design of the network model can be lightweight, the complexity of the network model can be reduced, the consumption of computing power and memory can be low, real-time calculation can be guaranteed at a high sampling rate, and the hardware cost of the terminal can be reduced.
[0108] On the second aspect, accordingly, the embodiment of the present application also provides a speech noise reduction device, which can implement all the processes of the speech noise reduction method provided in the above embodiment.
[0109] See also Figure 4 , shows a schematic diagram of the structure of a speech noise reduction device provided in an embodiment of the present application, including:
[0110] A linear spectrum frequency determination module 401 is used to determine the speech linear spectrum frequency and the noise linear spectrum frequency based on the noisy speech using a pre-trained neural network, wherein the neural network is trained based on sample speech data;
[0111] A gain determination module 402, configured to determine a filter gain based on the speech linear spectrum frequency and the noise linear spectrum frequency;
[0112] The noise reduction module 403 is configured to call a linear filter to determine a noise-reduced speech matching the noisy speech based on the filter gain.
[0113] In an optional embodiment, the device further comprises:
[0114] A speech acquisition and transformation module, used for acquiring the noisy speech and performing a fast Fourier transform on the noisy speech to convert the noisy speech into a logarithmic power spectrum before determining the speech linear spectrum frequency and the noise linear spectrum frequency based on the noisy speech by using the pre-trained neural network;
[0115] The method of using a pre-trained neural network to determine the speech linear spectrum frequency and the noise linear spectrum frequency based on noisy speech includes:
[0116] Inputting the logarithmic power spectrum into the neural network to obtain the speech linear spectrum frequency and the noise linear spectrum frequency output by the neural network;
[0117] The step of calling a linear filter to determine a noise reduction speech matching the noisy speech based on the filter gain comprises:
[0118] Determining a first amplitude spectrum of the noisy speech;
[0119] Calling the linear filter, using the filter gain as a weighting coefficient, and performing weighted calculation on the values of each frequency in the first amplitude spectrum to obtain a second amplitude spectrum;
[0120] Perform inverse Fourier transform on the second amplitude spectrum to obtain the noise-reduced speech.
[0121] In an optional implementation, the sample speech data includes sample noisy speech, speech data corresponding to the sample noisy speech, and noise data corresponding to the sample noisy speech, and the training method of the neural network includes:
[0122] Performing a fast Fourier transform on the sample noisy speech to convert the sample noisy speech into a sample logarithmic power spectrum;
[0123] Calculating the linear spectrum frequency of the speech data and calculating the linear spectrum frequency of the noise data;
[0124] The sample logarithmic power spectrum is used as training input data, and the linear spectrum frequency of the speech data and the linear spectrum frequency of the noise data are connected as training output data. The network is trained using a stochastic gradient descent algorithm to obtain the neural network after training.
[0125] In an optional implementation, the neural network includes three hidden layers, each hidden layer includes 512 rectified linear hidden units, and the cost function of the neural network is Euclidean distance.
[0126] In an optional implementation, the linear filter includes a pre-constructed Wiener adaptive filter, and the construction method of the Wiener adaptive filter includes:
[0127] Converting the speech linear spectrum frequency into speech linear prediction coefficients to calculate the speech spectrum shape according to the speech linear prediction coefficients;
[0128] Converting the noise linear spectrum frequency into noise linear prediction coefficients to calculate the noise spectrum shape according to the noise linear prediction coefficients;
[0129] Based on the speech spectrum shape and the noise spectrum shape, the Wiener adaptive filter is constructed using the Gain update rule.
[0130] In an optional implementation, the method of calculating the linear spectrum frequency includes:
[0131] Perform Fourier transform on the data to be calculated;
[0132] For the data to be calculated after Fourier transformation, the linear prediction coefficient is calculated, wherein the order of the linear prediction coefficient is the set order;
[0133] Determining a reflection coefficient that matches the linear prediction coefficient, and performing stability processing on the reflection coefficient;
[0134] Based on the reflection coefficient after stability processing, a linear spectrum frequency with an order equal to the set order is calculated.
[0135] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any of the above-mentioned speech noise reduction methods are implemented.
[0136] In a fourth aspect, an embodiment of the present application provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of any of the above-mentioned speech noise reduction methods.
[0137] In a fifth aspect, an embodiment of the present application provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the steps of any of the above-mentioned speech noise reduction methods when executing the computer program.
[0138] See also Figure 5 The computer device of this embodiment includes: a processor 501, a memory 502, and a computer program stored in the memory 502 and executable on the processor 501, such as a speech noise reduction program. When the processor 501 executes the computer program, the steps in the above-mentioned speech noise reduction method embodiments are implemented, such as Figure 1 Steps S101-S103 are shown.
[0139] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory 502 and executed by the processor 501 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the computer device.
[0140] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may include, but not limited to, a processor 501 and a memory 502. Those skilled in the art will appreciate that the schematic diagram is merely an example of a computer device and does not constitute a limitation on the computer device. The computer device may include more or fewer components than shown in the diagram, or may combine certain components, or different components. For example, the computer device may also include input and output devices, network access devices, buses, etc.
[0141] The processor 501 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor 501 may also be any conventional processor, etc. The processor 501 is the control center of the computer device, and uses various interfaces and lines to connect various parts of the entire computer device.
[0142] The memory 502 can be used to store the computer program and / or module. The processor 501 implements various functions of the computer device by running or executing the computer program and / or module stored in the memory 502 and calling the data stored in the memory 502. The memory 502 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 502 can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.
[0143] Wherein, if the module / unit integrated in the computer device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor 501. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc.
[0144] In summary, the embodiments of the present application have at least the following beneficial effects:
[0145] According to an embodiment of the present application, a pre-trained neural network is used to determine the speech linear spectrum frequency and the noise linear spectrum frequency based on noisy speech, wherein the neural network is trained based on sample speech data; based on the speech linear spectrum frequency and the noise linear spectrum frequency, the filter gain is determined; based on the filter gain, a linear filter is called to determine the noise reduction speech that matches the noisy speech. In this way, since noise reduction is completed through the filter, the neural network only needs to be used to determine the speech linear spectrum frequency and the noise linear spectrum frequency corresponding to the noisy speech in real time to obtain the current filter gain, without the need to extract complex speech features that can directly complete speech noise reduction. Therefore, the network complexity can be greatly reduced, and the timeliness of the speech noise reduction response can be improved, thereby ultimately improving the speech quality in different scenarios.
[0146] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented by means of software plus the necessary hardware platform, and of course, it can also be implemented entirely by hardware. Based on such an understanding, all or part of the contribution of the technical solution of the present application to the background technology can be embodied in the form of a software product, and the computer software product can be stored in a storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application or some parts of the embodiments.
[0147] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications are also considered to be within the scope of protection of the present application.
Claims
1. A speech noise reduction method, characterized in that: include: Determine the speech linear spectrum frequency and the noise linear spectrum frequency based on the noisy speech using a pre-trained neural network, wherein the neural network is trained based on sample speech data; Determining a filter gain based on the speech linear spectrum frequency and the noise linear spectrum frequency; Based on the filter gain, a linear filter is called to determine the noise-reduced speech matching the noisy speech.
2. The speech noise reduction method according to claim 1, characterized in that: Before determining the speech linear spectrum frequency and the noise linear spectrum frequency based on the noisy speech by using the pre-trained neural network, the method further includes: Acquire the noisy speech, and perform a fast Fourier transform on the noisy speech to convert the noisy speech into a logarithmic power spectrum; The method of using a pre-trained neural network to determine the speech linear spectrum frequency and the noise linear spectrum frequency based on noisy speech includes: Inputting the logarithmic power spectrum into the neural network to obtain the speech linear spectrum frequency and the noise linear spectrum frequency output by the neural network; The step of calling a linear filter to determine a noise reduction speech matching the noisy speech based on the filter gain comprises: Determining a first amplitude spectrum of the noisy speech; Calling the linear filter, using the filter gain as a weighting coefficient, and performing weighted calculation on the values of each frequency in the first amplitude spectrum to obtain a second amplitude spectrum; Perform inverse Fourier transform on the second amplitude spectrum to obtain the noise-reduced speech.
3. The speech noise reduction method according to claim 2, characterized in that: The sample speech data includes sample noisy speech, speech data corresponding to the sample noisy speech, and noise data corresponding to the sample noisy speech, and the training method of the neural network includes: Performing a fast Fourier transform on the sample noisy speech to convert the sample noisy speech into a sample logarithmic power spectrum; Calculating the linear spectrum frequency of the speech data and calculating the linear spectrum frequency of the noise data; The sample logarithmic power spectrum is used as training input data, and the linear spectrum frequency of the speech data and the linear spectrum frequency of the noise data are connected as training output data. The network is trained using a stochastic gradient descent algorithm to obtain the neural network after training.
4. The speech noise reduction method according to claim 3, characterized in that: The neural network includes 3 hidden layers, each hidden layer includes 512 rectified linear hidden units, and the cost function of the neural network is Euclidean distance.
5. The speech noise reduction method according to claim 1, characterized in that: The linear filter includes a pre-constructed Wiener adaptive filter, and the construction method of the Wiener adaptive filter includes: Converting the speech linear spectrum frequency into speech linear prediction coefficients to calculate the speech spectrum shape according to the speech linear prediction coefficients; Converting the noise linear spectrum frequency into noise linear prediction coefficients to calculate the noise spectrum shape according to the noise linear prediction coefficients; Based on the speech spectrum shape and the noise spectrum shape, the Wiener adaptive filter is constructed using the Gain update rule.
6. The speech noise reduction method according to any one of claims 1 to 5, characterized in that: Methods for calculating linear spectrum frequencies include: Perform Fourier transform on the data to be calculated; For the data to be calculated after Fourier transformation, the linear prediction coefficient is calculated, wherein the order of the linear prediction coefficient is the set order; Determining a reflection coefficient that matches the linear prediction coefficient, and performing stability processing on the reflection coefficient; Based on the reflection coefficient after stability processing, a linear spectrum frequency with an order equal to the set order is calculated.
7. A speech noise reduction device, characterized in that: include: A linear spectrum frequency determination module, used to determine the speech linear spectrum frequency and the noise linear spectrum frequency based on noisy speech using a pre-trained neural network, wherein the neural network is trained based on sample speech data; A gain determination module, used for determining a filter gain based on the speech linear spectrum frequency and the noise linear spectrum frequency; The noise reduction module is used to call a linear filter to determine a noise reduction speech matching the noisy speech based on the filter gain.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the speech noise reduction method according to any one of claims 1 to 6 is implemented.
9. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the speech noise reduction method according to any one of claims 1 to 6 is implemented.
10. A computer device, characterized in that: The invention comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the speech noise reduction method according to any one of claims 1 to 6 when executing the computer program.