Voice processing method and device and medium
By using multi-objective deep neural network and dereverberation processing network in a closed space, the problem of reverberation speech reducing speech quality is solved, and the quality of speech information and communication effect are significantly improved.
Patent Information
- Application Number
- CN202411739837.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-29
- Publication Date
- 2025-02-28
AI Technical Summary
Reverb speech in the enclosed space reduces the quality and intelligibility of pure speech information, affecting communication effects, and is especially inconvenient for hearing-impaired customers.
A multi-objective deep neural network is used to reduce the target voice information, remove background noise, and dereverberation of the noise-decreased voice information through the dereverberation processing network to obtain pure voice information.
It significantly improves the quality and intelligibility of pure voice information, improves communication effect, and improves user experience.
Smart Images
Figure CN119541527A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a speech processing method, device and medium. Background Art
[0002] People in closed spaces such as bank branches often use intercoms to communicate. When the speaker is at a certain distance from the microphone, the physical structure of the closed space will cause sound reflection and produce reverberant speech. The speech received by the microphone is a combination of the original speech (also known as the clean speech information) and the reverberant speech on top of the clean speech information. The presence of reverberant speech significantly reduces the quality and intelligibility of the clean speech information. This not only affects the communication effect of ordinary customers, but also brings great inconvenience to customers with hearing impairments. Summary of the invention
[0003] The embodiments of the present application provide a speech processing method, device and medium, which can reduce reverberant speech, improve the quality and intelligibility of original speech, and enhance the communication effect of customers.
[0004] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:
[0005] In a first aspect, the present application provides a speech processing method, the method comprising:
[0006] Acquiring target voice information, wherein the target voice information includes reverberation voice information, background noise and pure voice information;
[0007] Performing noise reduction processing on the target voice information to obtain noise-reduced voice information, wherein the noise-reduced voice information includes the reverberation voice information and the clean voice information;
[0008] The de-noised voice information is subjected to dereverberation processing to obtain the pure voice information.
[0009] Optionally, performing noise reduction processing on the target voice information to obtain noise-reduced voice information includes:
[0010] Performing noise reduction processing on the target speech information using a multi-objective deep neural network of a speech processing model to obtain the noise-reduced speech information;
[0011] The performing dereverberation processing on the noise-reduced voice information to obtain the pure voice information includes:
[0012] Performing dereverberation processing on the noise-reduced speech information using a dereverberation processing network of the speech processing model to obtain the pure speech information;
[0013] The input of the speech processing model is target speech information, and the output is pure speech information. The speech processing model is a model trained based on a training set consisting of background noise information, reverberant speech information, and pure speech information.
[0014] Optionally, the performing noise reduction processing on the target speech information using a multi-objective deep neural network of a speech processing model includes:
[0015] Inputting the target speech information into the multi-target deep neural network to obtain the IRM and amplitude spectrum of the preliminary noise-reduced speech information;
[0016] Convolutionally processing the IRM of the preliminary noise-reduced speech information with the target speech information;
[0017] The information after the convolution processing and the amplitude spectrum of the preliminary noise-reduced speech information are input into a multi-layer perceptron MLP to obtain the noise-reduced speech information.
[0018] Optionally, the cost function of the multi-objective deep neural network is related to the minimum mean square error.
[0019] Optionally, the dereverberation processing network includes a dereverberation preprocessing module, a bidirectional long short-term memory network BLSTM and an inverse Fourier transform IDFT module, and the dereverberation processing network using the speech processing model performs dereverberation processing on the noise-reduced speech information to obtain the pure speech information, including:
[0020] Inputting the amplitude spectrum of the noise-reduced speech information into the dereverberation preprocessing network to obtain a normalized power spectrum of the noise-reduced speech information; the amplitude spectrum of the normalized power spectrum is an amplitude spectrum compressed by a logarithmic function, and the mean value of the amplitude spectrum is 0 and the variance is 1;
[0021] Inputting the normalized power spectrum of the de-noised speech information into the BLSTM to obtain the normalized spectrum of the clean speech information; restoring the normalized spectrum of the clean speech information to obtain the amplitude spectrum of the clean speech information;
[0022] The amplitude spectrum of the clean voice information is input into an inverse Fourier transform IDFT, and the time domain information of the clean voice information is obtained based on the phase information of the clean voice information.
[0023] Optionally, the cost function of the IDFT is:
[0024]
[0025] in, represents element-level multiplication, || ||2 represents L2 regularization, s--original speech information, IFFT is Fourier transform, and R c They represent the enhanced speech amplitude spectrum and the phase information of the pure speech after being processed by the two-stage system respectively, and c represents the parameters of the supervised learning system.
[0026] In a second aspect, an embodiment of the present application provides a speech processing device, the device comprising:
[0027] An acquisition unit, used to acquire target voice information, wherein the target voice information includes reverberation voice information, background noise and pure voice information;
[0028] A noise reduction unit, configured to perform noise reduction processing on the target voice information to obtain noise-reduced voice information, wherein the noise-reduced voice information includes the reverberation voice information and the clean voice information;
[0029] The dereverberation unit is used to perform dereverberation processing on the noise-reduced speech information to obtain the pure speech information.
[0030] Optionally, the noise reduction unit is specifically used for:
[0031] Performing noise reduction processing on the target speech information using a multi-objective deep neural network of a speech processing model to obtain the noise-reduced speech information;
[0032] The dereverberation unit is specifically used to: perform dereverberation processing on the noise-reduced speech information using the dereverberation processing network of the speech processing model to obtain the pure speech information;
[0033] The input of the speech processing model is target speech information, and the output is pure speech information. The speech processing model is a model trained based on a training set consisting of background noise information, reverberant speech information, and pure speech information.
[0034] Optionally, the noise reduction unit is specifically used for:
[0035] Inputting the target speech information into the multi-target deep neural network to obtain the IRM and amplitude spectrum of the preliminary noise-reduced speech information;
[0036] Convolutionally processing the IRM of the preliminary noise-reduced speech information with the target speech information;
[0037] The information after the convolution processing and the amplitude spectrum of the preliminary noise-reduced speech information are input into a multi-layer perceptron MLP to obtain the noise-reduced speech information.
[0038] In a third aspect, an embodiment of the present application provides a computer storage medium for storing a computer program, which, when executed, is used to implement the method as described in any one of the first aspects.
[0039] In a fourth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on at least one computing device, enables the at least one computing device to implement a method as described in any one of the first aspects.
[0040] In summary, the embodiments of the present application provide a speech processing method, device and medium. When the method is specifically implemented, it includes: for speech information containing background noise and reverberant speech information, firstly, a multi-objective deep neural network is used to perform noise reduction processing on the speech information. Then, the speech information after noise reduction processing is subjected to dereverberation processing to obtain pure speech information. That is, the embodiments of the present application remove background noise and reverberant speech information in a two-stage manner, which can significantly improve the quality and intelligibility of the pure speech information obtained, thereby improving the communication effect and enhancing the user experience, compared with the method of directly mapping the spectral characteristics of pure speech from the spectral characteristics of noisy speech. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0042] Figure 1 A system architecture diagram of a speech processing method provided in an embodiment of the present application;
[0043] Figure 2 A flow chart of a speech processing method provided in an embodiment of the present application;
[0044] Figure 3 A schematic diagram of the structure of a speech processing model provided in an embodiment of the present application;
[0045] Figure 4 A schematic diagram of the structure of a speech processing device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0047] As mentioned above, the presence of reverberant speech significantly reduces the quality and intelligibility of the original speech, which not only affects the communication effect of ordinary customers, but also brings great inconvenience to customers with hearing impairments.
[0048] In view of this, an embodiment of the present application provides a speech processing method, which first uses a multi-objective deep neural network to perform noise reduction processing on the speech information containing background noise and reverberant speech information. Then, the speech information after noise reduction processing is subjected to dereverberation processing to obtain pure speech information. That is, the embodiment of the present application removes background noise and reverberant speech information in a two-stage manner. Compared with the method of directly mapping the spectral features of pure speech from the spectral features of noisy speech, it can significantly improve the quality and intelligibility of the pure speech information obtained, thereby improving the communication effect and enhancing the user experience.
[0049] First, the application scenario of the speech processing method provided in the embodiment of the present application is introduced.
[0050] Attached Figure 1 A system architecture diagram of a voice processing method provided in an embodiment of the present application. The system architecture 100 is a diagram of an accounting system architecture, and the system architecture 100 includes an intercom 101, a network 102, and a server 103. The network 102 is used to provide a medium for a communication link between the intercom 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.
[0051] The user can use the intercom 101 to interact with the server 103 via the network 102 to receive or send messages, etc. Various communication client applications can be installed on the intercom 101.
[0052] The server 103 may be a server providing various services, such as a background management server (only an example) providing support for the intercom 101. The background management server may analyze and process the received product information query request and other data, and feed back the processing result (such as voice information - only an example) to the intercom.
[0053] It should be noted that the speech processing method provided in the embodiment of the present invention is generally executed by the server 103, and accordingly, the computing device is generally disposed in the server 103. It should be understood that Figure 1 The numbers of intercoms, networks and servers in the embodiment are merely illustrative. Any number of intercoms, networks and servers may be provided as required.
[0054] The following describes the speech processing method provided in the embodiments of the present application in conjunction with the accompanying drawings.
[0055] Attached Figure 2A flow chart of a speech processing method provided in an embodiment of the present application, the method includes the following contents:
[0056] S210: Acquire target voice information.
[0057] The target voice information is the voice information received by the microphone. After receiving the target voice information, the microphone sends the target voice information to the server for processing.
[0058] In the embodiment of the present application, the target voice information includes reverberant voice information and pure voice information. Among them, pure voice information is also called original voice information, which is voice information without any external interference when the speaker speaks. Reverberant voice information refers to the voice information generated after the physical structure of the closed space reflects the pure voice information, which is attenuated and delayed relative to the pure voice information. It can be understood that the reverberant voice information is related to the pure voice information.
[0059] In addition, the target speech information also includes background noise. Background noise refers to other undesirable sounds other than the speaker's voice. For example, the sound of conversation in a bank branch, the hum of air conditioners, the sound of closing doors, typing, or car horns. Background noise is not correlated with pure speech information.
[0060] S220: Perform noise reduction processing on the target voice information to obtain noise-reduced voice information.
[0061] The background noise in the noise-reduced speech information is much smaller than the background noise in the target speech information.
[0062] The server performs noise reduction processing on the target voice information to remove background noise. The server may perform noise reduction processing on the target voice information based on a noise reduction processing algorithm to remove background noise. For example, the noise reduction processing algorithm may be a Bayesian algorithm, an adaptive filtering algorithm, or a Kalman filtering algorithm.
[0063] Furthermore, in order to remove the background noise of the target voice information and at the same time keep the reverberation as unaffected as possible to avoid affecting the overall voice quality, the server can input the target voice information into a trained Multi-Objective Deep Neural Network (MO-DNN), and use the MO-DNN to perform noise reduction processing on the target voice information to obtain the noise-reduced voice information.
[0064] Among them, the trained MO-DNN refers to the MO-DNN trained using a data set consisting of background noise, reverberant speech information, and pure speech information. The cost function of the MO-DNN can be a minimum mean square error function, that is, the MO-DNN is measured by calculating the average of the square difference between the predicted value and the true value during training. In addition, the cost function of the multi-objective DNN model can also be other functions, which are not specifically limited in the embodiments of the present application.
[0065] S230: performing dereverberation processing on the noise-reduced voice information to obtain pure voice information.
[0066] The server may use a spectrum mapping method to perform dereverberation processing on the noise-reduced speech information to obtain pure speech information. The spectrum mapping method is a method that combines a deep learning model to achieve a mapping relationship between an input speech spectrum and an output speech spectrum.
[0067] Exemplary description: If the speech spectrum of the de-noised speech information is X(f, t), f represents the frequency and t represents the time frame. The spectrum mapping is performed using the trained dereverberation model. The training set of the dereverberation model includes reverberated speech information and pure speech information.
[0068] Since the reverberant speech information and the pure speech information in the denoised speech information are correlated, the spectrum mapping method has a better processing effect than the ideal ratio mask (IRM) separation method.
[0069] In summary, the speech processing method provided in the embodiment of the present application, for speech information containing background noise and reverberant speech information, first uses a multi-objective deep neural network to perform noise reduction processing on the speech information. Then, the speech information after noise reduction processing is subjected to dereverberation processing to obtain pure speech information. That is, the embodiment of the present application removes background noise and reverberant speech information in a two-stage manner, which can significantly improve the quality and intelligibility of the pure speech information obtained, thereby improving the communication effect and enhancing the user experience, compared with the method of directly mapping the spectral characteristics of pure speech from the spectral characteristics of noisy speech.
[0070] Furthermore, the present application embodiment provides a speech processing model, which is used to implement steps S220 and S230. Figure 3 Provide detailed explanation.
[0071] Attached Figure 3 A schematic diagram of the structure of a speech processing model provided in an embodiment of the present application. The speech processing model specifically includes MO-DNN, a multilayer perceptron (MLP) and a dereverberation processing network.
[0072] For MO-DNN, in order to remove the background noise of the target voice information and keep the reverberation as unaffected as possible to avoid affecting the overall voice quality, the input layer of MO-DNN is the voice information. The output layer of MO-DNN includes a first branch and a second branch. Among them, the first branch is used to output the ideal ratio mask (Ideal Ratio Mask, IRM). Among them, IRM is the ratio of the amplitude spectrum of the first voice information and the amplitude spectrum of the target voice information. The second branch is used to output the amplitude spectrum of the noise-reduced voice information in the frequency domain. Exemplarily, for example, the voice spectrum of the target voice information is X(f, t). The server inputs X(f, t) into MO-DNN, the output IRM is M(f, t), and the output amplitude spectrum is S(f, t).
[0073] The cost function of MO-DNN is related to the minimum mean square error. For example, the cost function of MO-DNN can be the minimum mean square error function, that is, by calculating the average of the square difference between the predicted value and the true value during training, MO-DNN is measured to improve the training effect of the MO-DNN model. In addition, the cost function of the multi-objective DNN model can also be other functions, which are not specifically limited in the embodiments of the present application.
[0074] Next, the IRM can perform a convolution operation with the speech spectrum of the target speech information to obtain a speech spectrum of the preliminary noise-reduced speech information that does not contain background noise, or contains background noise but the amplitude of the spectrum of the background noise is less than or equal to the preset amplitude value. Figure 3 As shown in the figure, after X(f, t) is input into MO-DNN, the IRM value M(f, t) and the amplitude spectrum value S(f, t) are output. M(f, t) is convolved with X(f, t) to obtain the speech spectrum Y(f, t) of the preliminary noise-reduced speech information. The specific weighted processing method is shown in formula (1).
[0075] Y(f,t)=M(f,t)·X(f,t) (1)
[0076] The speech spectrum and amplitude spectrum of the preliminary noise-reduced speech information are input into the MLP to obtain the amplitude spectrum of the noise-reduced speech information.
[0077] The dereverberation processing network includes a dereverberation preprocessing module, a bidirectional long short-term memory network (Bidirectional Long Short-Term Memory Network, BLSTM) and an inverse Fourier transform (Inverse Discrete Fourier Transform, IDFT) module.
[0078] The dereverberation preprocessing module is used to receive the amplitude spectrum of the de-noised speech information, and compress the amplitude spectrum of the de-noised speech information using a logarithmic function to obtain a compressed amplitude spectrum. The compressed amplitude spectrum is normalized to ensure that the data is accurately input into the BLSTM network.
[0079] The normalization of the embodiment of the present application is a zero mean and unit variance normalization method. The zero mean and unit variance normalization method means that the normalization result is a mean of 0 and a variance of 1. This normalization method helps to retain more spectrum details relative to percentage normalization (i.e., compressing the value of the spectrum feature to a range of 0 to 1).
[0080] In an embodiment of the present application, the data input to the BLSTM is the normalized features of a preset N frames, where N is a positive integer. Figure 3 Take N=11 as an example for schematic description. That is to say, multiple speech frames need to be processed by scalar method. To ensure the processing effect, for adjacent frames, each frame is first normalized to zero mean and unit variance when splicing.
[0081] Next, the concatenated normalized features are input into the Bidirectional Long Short-Term Memory Network (BLSTM) to output the spectrum W(f,t) of the pure speech information.
[0082] Furthermore, since the target has been normalized before training, the logarithmic power spectrum of the enhanced speech can be restored by using the standard deviation and mean of the logarithm of the pure speech. The logarithmic power spectrum is then exponentially operated to restore the amplitude spectrum P(f,t).
[0083] Then P(f, t) is input into the Inverse Discrete Fourier Transform (IDFT). Combined with the phase information of the pure speech information, P(f, t) can be reconstructed in the time domain to obtain the time domain information of the pure speech information, thereby obtaining the pure speech information.
[0084] The cost function of IDFT is shown in formula (2). In time frame m, where m>0,
[0085]
[0086] in represents element-level multiplication, || ||2 represents L2 regularization, s--original speech information, IFFT is Fourier transform, and R cThey represent the enhanced speech amplitude spectrum and the phase information of the pure speech after being processed by the two-stage system respectively, and c represents the parameters of the supervised learning system.
[0087] In the embodiment of the present application, the server can implicitly pay more attention to the time-frequency units that contribute more to the time-domain signal by calculating the cost function in the time domain, thereby weighting different time-frequency units.
[0088] In the embodiment of the present application, the speech processing model can be jointly trained. Before the joint training, the denoising multi-objective DNN and the dereverberation BLSTM are trained separately using the minimum mean square error as the objective function, and the obtained parameters are used in the two-stage speech enhancement system.
[0089] Figure 3 The speech processing model introduces MO-DNN as a network model in the noise reduction stage, uses ideal ratio masking as a training target combined with a spectrum mapping-based method, and uses a bidirectional long short-term memory network combined with a spectrum mapping-based method in the dereverberation stage, and finally obtains the amplitude spectrum information of the enhanced speech. A new objective function is used, and the phase of pure speech information is added during the model training process to assist in enhancing the speech amplitude spectrum. Next, an iterative phase reconstruction method is used for waveform reconstruction, and finally the two-stage model is jointly trained to optimize the objective function. This further improves the speech clarity.
[0090] In addition, an embodiment of the present application also provides a speech processing device.
[0091] Attached Figure 4 A schematic diagram of the structure of a speech processing device provided in an embodiment of the present application, wherein the device 400 includes:
[0092] An acquisition unit 401 is used to acquire target voice information, where the target voice information includes reverberation voice information, background noise and pure voice information;
[0093] A noise reduction unit 402 is used to perform noise reduction processing on the target voice information to obtain noise-reduced voice information, where the noise-reduced voice information includes reverberation voice information and pure voice information;
[0094] The dereverberation unit 403 is used to perform dereverberation processing on the noise-reduced speech information to obtain pure speech information.
[0095] Optionally, the noise reduction unit is specifically used for:
[0096] The multi-objective deep neural network of the speech processing model is used to perform noise reduction processing on the target speech information to obtain noise-reduced speech information;
[0097] The dereverberation unit is specifically used to: perform dereverberation processing on the noise-reduced speech information using the dereverberation processing network of the speech processing model to obtain pure speech information;
[0098] The input of the speech processing model is the target speech information, and the output is the pure speech information. The speech processing model is a model trained based on a training set consisting of background noise information, reverberation speech information and pure speech information.
[0099] Optionally, the noise reduction unit is specifically used for:
[0100] Input the target speech information into the multi-target deep neural network to obtain the IRM and amplitude spectrum of the preliminary noise-reduced speech information;
[0101] Convolution processing is performed on the IRM of the preliminary noise-reduced speech information and the target speech information;
[0102] The information after convolution processing and the amplitude spectrum of the preliminary noise-reduced speech information are input into the multi-layer perceptron MLP to obtain the noise-reduced speech information.
[0103] Optionally, the cost function of the multi-objective deep neural network is related to the minimum mean square error.
[0104] Optionally, the dereverberation processing network includes a dereverberation preprocessing module, a bidirectional long short-term memory network BLSTM and an inverse Fourier transform IDFT module, and uses the dereverberation processing network of the speech processing model to dereverberate the noise-reduced speech information to obtain pure speech information, including:
[0105] Inputting the amplitude spectrum of the de-noised speech information into the dereverberation preprocessing network to obtain the normalized power spectrum of the de-noised speech information; the amplitude spectrum of the normalized power spectrum is the amplitude spectrum after logarithmic function compression, and the mean value of the amplitude spectrum is 0 and the variance is 1;
[0106] The normalized power spectrum is input into BLSTM to obtain the normalized spectrum of pure speech information;
[0107] Perform restoration processing on the normalized spectrum to obtain the amplitude spectrum of pure speech information;
[0108] The amplitude spectrum of the clean speech information is input into the inverse Fourier transform IDFT, and the time domain information of the clean speech information is obtained based on the phase information of the clean speech information.
[0109] Optionally, the cost function of IDFT is:
[0110]
[0111] in, represents element-level multiplication, || ||2 represents L2 regularization, s--original speech information, IFFT is Fourier transform, and R c They represent the enhanced speech amplitude spectrum and the phase information of the pure speech after being processed by the two-stage system respectively, and c represents the parameters of the supervised learning system.
[0112] The speech processing device provided in the embodiment of the present application first uses a multi-objective deep neural network to perform noise reduction processing on the speech information containing background noise and reverberant speech information. Then, the speech information after noise reduction processing is subjected to dereverberation processing to obtain pure speech information. That is, the embodiment of the present application removes background noise and reverberant speech information in a two-stage manner. Compared with the method of directly mapping the spectral features of pure speech from the spectral features of noisy speech, it can significantly improve the quality and intelligibility of the pure speech information obtained, thereby improving the communication effect and enhancing the user experience.
[0113] It is understandable that in the specific implementation of the present application, related data such as user information and historical user information are involved. When the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.
[0114] For the convenience of description, the above system is described as being divided into various modules according to their functions. Of course, when implementing the present invention, the functions of each module can be implemented in the same or multiple software and / or hardware.
[0115] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.
[0116] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0117] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
[0118] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0119] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A speech processing method, characterized in that: The method comprises: Acquiring target voice information, wherein the target voice information includes reverberation voice information, background noise and pure voice information; Performing noise reduction processing on the target voice information to obtain noise-reduced voice information, wherein the noise-reduced voice information includes the reverberation voice information and the clean voice information; The de-noised voice information is subjected to dereverberation processing to obtain the pure voice information.
2. The method according to claim 1, characterized in that: The performing noise reduction processing on the target voice information to obtain the noise-reduced voice information includes: Performing noise reduction processing on the target speech information using a multi-objective deep neural network of a speech processing model to obtain the noise-reduced speech information; The performing dereverberation processing on the noise-reduced voice information to obtain the pure voice information includes: Performing dereverberation processing on the noise-reduced speech information using a dereverberation processing network of the speech processing model to obtain the pure speech information; The input of the speech processing model is target speech information, and the output is pure speech information. The speech processing model is a model trained based on a training set consisting of background noise information, reverberant speech information, and pure speech information.
3. The method according to claim 2, characterized in that: The noise reduction process of the target speech information using the multi-objective deep neural network of the speech processing model comprises: Inputting the target speech information into the multi-target deep neural network to obtain the IRM and amplitude spectrum of the preliminary noise-reduced speech information; Convolutionally processing the IRM of the preliminary noise-reduced speech information with the target speech information; The information after the convolution processing and the amplitude spectrum of the preliminary noise-reduced speech information are input into a multi-layer perceptron MLP to obtain the noise-reduced speech information.
4. The method according to claim 3, characterized in that: The cost function of the multi-objective deep neural network is related to the minimum mean square error.
5. The method according to claim 2, characterized in that: The dereverberation processing network includes a dereverberation preprocessing module, a bidirectional long short-term memory network BLSTM and an inverse Fourier transform IDFT module. The dereverberation processing network using the speech processing model performs dereverberation processing on the noise-reduced speech information to obtain the pure speech information, including: Inputting the amplitude spectrum of the de-noised speech information into the dereverberation preprocessing network to obtain a normalized power spectrum of the de-noised speech information; the normalized power spectrum is the amplitude spectrum after logarithmic function compression, and the mean of the amplitude spectrum is 0 and the variance is 1; Inputting the normalized power spectrum of the noise-reduced speech information into the BLSTM to obtain a normalized spectrum of the clean speech information; Restoring the normalized frequency spectrum of the clean voice information to obtain the amplitude spectrum of the clean voice information; The amplitude spectrum of the clean voice information is input into an inverse Fourier transform IDFT, and the time domain information of the clean voice information is obtained based on the phase information of the clean voice information.
6. The method according to claim 5, characterized in that The cost function of the IDFT is: in, represents element-wise multiplication, represents L2 normalization, s--original speech information, IFFT is Fourier transform, and R c They represent the enhanced speech amplitude spectrum and the phase information of the pure speech after being processed by the two-stage system respectively, and c represents the parameters of the supervised learning system.
7. A speech processing device, characterized in that: The device comprises: An acquisition unit, used to acquire target voice information, wherein the target voice information includes reverberation voice information, background noise and pure voice information; A noise reduction unit, configured to perform noise reduction processing on the target voice information to obtain noise-reduced voice information, wherein the noise-reduced voice information includes the reverberation voice information and the clean voice information; The dereverberation unit is used to perform dereverberation processing on the noise-reduced speech information to obtain the pure speech information.
8. The device according to claim 7, characterized in that The noise reduction unit is specifically used for: Performing noise reduction processing on the target speech information using a multi-objective deep neural network of a speech processing model to obtain the noise-reduced speech information; The dereverberation unit is specifically used to: perform dereverberation processing on the noise-reduced speech information using the dereverberation processing network of the speech processing model to obtain the pure speech information; The input of the speech processing model is target speech information, and the output is pure speech information. The speech processing model is a model trained based on a training set consisting of background noise information, reverberant speech information, and pure speech information.
9. The device according to claim 8, characterized in that The noise reduction unit is specifically used for: Inputting the target speech information into the multi-target deep neural network to obtain the IRM and amplitude spectrum of the preliminary noise-reduced speech information; Convolutionally processing the IRM of the preliminary noise-reduced speech information with the target speech information; The information after the convolution processing and the amplitude spectrum of the preliminary noise-reduced speech information are input into a multi-layer perceptron MLP to obtain the noise-reduced speech information.
10. A computer storage medium, characterized in that: Used to store a computer program, which, when executed, is used to implement the method according to any one of claims 1 to 6.