Speech processing method, medium, electronic device, and program product
By constructing a pre-defined linear relationship between clean speech and noisy speech in the frequency domain, and using a diffusion model for speech denoising, the problem of misidentification of valid speech in existing speech denoising models is solved, achieving a fast and non-destructive denoising effect for speech data.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HONOR DEVICE CO LTD
- Filing Date
- 2024-04-30
- Publication Date
- 2026-06-09
AI Technical Summary
Existing speech denoising models are prone to misidentification of valid speech as noise during training due to incorrect labeling data, which damages speech data, and the training and inference speed is relatively slow.
Speech denoising is performed using a diffusion model based on a straight-line trajectory. By constructing a pre-defined linear relationship between clean speech and noisy speech in the frequency domain, the diffusion model is used for speech denoising to ensure that the speech data is not damaged during the denoising process. Fast inference is performed using either the Euler algorithm or the Chasen extrapolation RK algorithm.
It achieves fast and effective noise removal without damaging speech data, has strong generalization performance, can effectively suppress noise not seen during the training phase, and ensures the clarity of the denoised speech.
Smart Images
Figure CN120913581B_ABST
Abstract
Claims
1. A speech processing method, characterized in that, The method includes: Obtain the first time-domain data of the speech data to be processed; The first time-domain data is converted from the time domain to the frequency domain to obtain the first frequency-domain data; The first frequency domain data is input into the diffusion model to obtain the second frequency domain data. The diffusion model is a diffusion model based on the ordinary differential equation ODE with a straight trajectory. The diffusion model obtains N correction information corresponding to the first frequency domain data based on N sampling times according to a preset linear relationship. The first frequency domain data is then denoised N times using the N correction information to obtain the second frequency domain data. N is a positive integer, and the first frequency domain data and the second frequency domain data have a preset linear relationship. The preset linear relationship is a straight-line data mapping relationship between noisy speech data and clean speech data. The second frequency domain data is converted from the frequency domain to the time domain to obtain the second time domain data after the denoising process of the speech data to be processed.
2. The method according to claim 1, characterized in that, The correction information is used to indicate the rate at which the first frequency domain data changes into the second frequency domain data.
3. The method according to claim 1, characterized in that, The second frequency domain data is obtained in the following way: Input the N sampling times into the diffusion model; For i < N, for the i-th sampling time, the diffusion model obtains the i-th correction information corresponding to the i-th frequency domain data according to the preset linear relationship, and performs denoising processing on the i-th frequency domain data through the i-th correction information to obtain the (i+1)-th frequency domain data, where the first frequency domain data is the first frequency domain data, the i-th frequency domain data and the (i+1)-th frequency domain data have the preset linear relationship, and i is a positive integer; For i=N, the (i+1)th frequency domain data is used as the second frequency domain data.
4. The method according to claim 3, characterized in that, The (i+1)th frequency domain data is obtained in the following way: For the i-th sampling time, the diffusion model is used to differentiate the i-th frequency domain data with respect to time to obtain the i-th correction information, wherein the i-th correction information is used to indicate the data change rate from the i-th frequency domain data to the (i+1)-th frequency domain data; A correction algorithm is used to denoise the i-th frequency domain data based on the i-th correction information to obtain the (i+1)-th frequency domain data.
5. The method according to claim 4, characterized in that, The correction algorithm includes at least one of Euler's algorithm and Chasen extrapolation RK algorithm.
6. The method according to claim 4, characterized in that, The i-th correction information is the ratio of the i-th frequency domain data to N; and the (i+1)-th frequency domain data is obtained by adding the i-th correction information to the i-th frequency domain data.
7. The method according to any one of claims 1 to 6, characterized in that, The diffusion model is trained in the following way: Acquire first training time-domain data and second training time-domain data of the training speech, wherein the second training time-domain data is obtained by superimposing preset time-domain noise data on the first training time-domain data; The first training time-domain data and the second training time-domain data are converted from the time domain to the frequency domain to obtain the first training frequency-domain data and the second training frequency-domain data, respectively. The diffusion model is trained based on the first training frequency domain data and the second training frequency domain data to obtain the trained diffusion model.
8. The method according to claim 7, characterized in that, The noise indicated by the preset time-domain noise data includes at least one of the following: wind noise, whistle noise, keyboard noise, and human voice noise.
9. The method according to claim 7, characterized in that, The step of training the diffusion model based on the first training frequency domain data and the second training frequency domain data includes: Obtain the training sampling time; According to the training sampling time, the first training frequency domain data and the second training frequency domain data are linearly interpolated based on a preset linear relationship to obtain the third training frequency domain data; The third training frequency domain data is superimposed with preset frequency domain noise data to obtain the fourth training frequency domain data, wherein the dimension of the third training frequency domain data is the same as the dimension of the preset frequency domain noise data. The fourth training frequency domain data is input into the diffusion model, and the diffusion model is used to differentiate the fourth training frequency domain data with respect to time to obtain the first training correction information. The network parameters of the diffusion model are adjusted based on the difference between the first training correction information and the preset correction information.
10. The method according to claim 9, characterized in that, The third training frequency domain data Obtain it using the following formula: ， in, This represents the first training frequency domain data. This represents the second training frequency domain data. This represents the training sampling time, and, In and It has the preset linear relationship.
11. The method according to claim 9 or 10, characterized in that, The fourth training frequency domain data Obtain it through the following methods: ， in, This represents the mean of the third training frequency domain data. This represents the preset frequency domain noise data. The standard deviation of the preset frequency domain noise data is represented.
12. The method according to claim 11, characterized in that, ,and The value of first increases and then decreases as the training sampling time increases.
13. The method according to claim 9, characterized in that, The preset frequency domain noise data is Gaussian noise data with a mean of 0 and a variance of 1.
14. The method according to claim 10, characterized in that, The preset correction information is: .
15. The method according to claim 14, characterized in that, The difference between the first training correction information and the preset correction information is expressed through a loss function. Determine, where mse() represents the mean square error.
16. The method according to claim 7, characterized in that, The first frequency domain data, the first training frequency domain data, and the second training frequency domain data were obtained using the Short Time Fourier Transform (STFT) technique.
17. The method according to claim 3, characterized in that, The value of N is determined based on the location or application of the voice data to be processed, and the value of N is different for the voice data to be processed in different locations and for the voice data to be processed in different applications.
18. A readable medium, characterized in that, The readable medium stores instructions that, when executed on an electronic device, cause the electronic device to perform the method of any one of claims 1 to 17.
19. An electronic device, characterized in that, include: A memory for storing instructions executed by one or more processors of an electronic device, and a processor, one of the processors of the electronic device, for performing the method of any one of claims 1 to 17.
20. A computer program product, characterized in that, When the computer program product is run on an electronic device, it causes the electronic device to perform the method of any one of claims 1 to 17.