Voice processing method and electronic device

Abstract

This application provides a speech processing method and an electronic device. The speech processing method includes: extracting features from acquired speech to be processed to obtain a speech signal-to-noise ratio (SNR); processing the speech to be processed to obtain feature parameters, including at least one of pitch period probability, zero-crossing rate, and energy ratio; and denoising the speech to be processed based on a preset denoising algorithm and the feature parameters to determine the target speech. Thus, during the denoising process of the speech to be processed, at least one of the three parameters—pitch period probability, zero-crossing rate, and energy ratio—can be introduced to assist in the protection of voiced and unvoiced sounds, based on the SNR, and to effectively identify and distinguish noise data, thereby maintaining the original fidelity of the speech as much as possible and improving the speech quality of the target speech.

Description

Technical Field

[0001] This application relates to the field of noise reduction technology, and more specifically, to a speech processing method and electronic device. Background Technology

[0002] Speech denoising algorithms typically sacrifice some of the original speech fidelity during the denoising process. If the signal-to-noise ratio is high and excessive denoising is unnecessary, but the algorithm still treats the speech as strongly noisy, the original speech quality will degrade. Therefore, how to maintain the original speech fidelity as much as possible during denoising to achieve high-quality denoised speech has become an urgent problem to solve. Summary of the Invention

[0003] The embodiments of this application provide a speech processing method and electronic device. In the process of denoising the speech to be processed, at least one of the three parameters of pitch period probability, speech zero-crossing rate and energy ratio can be introduced to assist in the protection of voiced and unvoiced sounds, based on the signal-to-noise ratio, and effectively identify and distinguish noise data, thereby maintaining the original fidelity of the speech as much as possible and improving the speech quality of the target speech.

[0004] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0005] According to one aspect of the embodiments of this application, a speech processing method is provided, comprising: extracting features from acquired speech to be processed to extract speech signal-to-noise ratio; processing the speech to be processed to obtain feature parameters of the speech to be processed, the feature parameters including at least one of pitch period probability, speech zero-crossing rate and energy ratio; and performing noise reduction on the speech to be processed based on a preset noise reduction algorithm according to the feature parameters to determine target speech.

[0006] According to one aspect of the embodiments of this application, an electronic device is provided, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the voice processing method as described in the above embodiments.

[0007] In some embodiments of this application, when denoising the speech to be processed, at least one of the pitch period probability, zero-crossing rate, and energy ratio of the speech to be processed can be obtained, and the obtained parameters can be used as feature parameters of the speech to be processed. Then, the speech to be processed is denoised according to the feature parameters and a preset denoising algorithm to obtain the final target speech. Thus, in the process of denoising the speech to be processed, at least one of the three parameters of pitch period probability, zero-crossing rate, and energy ratio can be introduced to assist in the protection of voiced and unvoiced sounds, based on the signal-to-noise ratio, and effectively identify and distinguish noise data, thereby maintaining the original fidelity of the speech as much as possible and improving the speech quality of the target speech.

[0008] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0009] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings: Figure 1 A schematic flowchart of a speech processing method provided in an embodiment of this application is shown.

[0010] Figure 2 A schematic flowchart of a speech processing method provided in an embodiment of this application is shown.

[0011] Figure 3 A schematic flowchart of a speech processing method provided in an embodiment of this application is shown.

[0012] Figure 4 A schematic flowchart of a speech processing method provided in an embodiment of this application is shown.

[0013] Figure 5 A schematic flowchart of a speech processing method provided in an embodiment of this application is shown.

[0014] Figure 6 A schematic flowchart of a speech processing method provided in an embodiment of this application is shown.

[0015] Figure 7 This diagram illustrates the structure of a voice processing device provided in an embodiment of this application. Detailed Implementation

[0016] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0017] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0018] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0019] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0020] Speech denoising algorithms are mainly divided into two categories: traditional speech signal processing algorithms and neural network-based model algorithms. Traditional speech signal processing algorithms directly process the time or frequency domain of speech data, obtaining speech gain based on the estimated noise spectrum, and finally denoising the speech using the gain. Examples include spectral subtraction, Wiener filtering, and minimum value statistics. Neural network-based model algorithms learn by fitting parameters to speech data on a pre-designed model structure, typically using supervised learning. This type of algorithm can be further divided into two categories: mapping methods and masking methods. Mapping methods directly output clean speech data using the model, while masking methods, similar to traditional speech signal processing algorithms, use the model to output speech denoising gain, denoising the speech using the gain.

[0021] Speech denoising algorithms typically sacrifice some of the original speech fidelity during the denoising process. If the signal-to-noise ratio (SNR) is high and excessive denoising is unnecessary, but the algorithm still treats the speech as strongly noisy, the original speech quality will degrade. Therefore, industry research has proposed adding a denoising intensity control coefficient to the backend of the denoising algorithm to limit the denoising effort. This allows for strong denoising when the speech sample noise is high, and weak or no denoising when the noise is low, achieving both effective denoising and preservation of speech quality. For example, the denoising intensity coefficient can be calculated by predicting the SNR using model methods to control the degree of denoising. Alternatively, speech activity detection methods can be used to control the denoising intensity coefficient.

[0022] While existing techniques can mitigate the impact of strong noise reduction on speech quality, their control coefficients are relatively simple and do not account for the influence of voiced sounds, unvoiced sounds, and noise on the control coefficients in noisy speech. If the characteristics and statistical patterns of voiced sounds, unvoiced sounds, and noise in speech samples are not considered, the control coefficients' ability to control speech segments is limited, directly leading to a decline in speech quality.

[0023] To address the above problems, this application provides a speech processing method.

[0024] The implementation details of the technical solutions in the embodiments of this application are described in detail below: Reference Figure 1 As shown, the speech processing method includes at least the following: Step 01: Extract features from the acquired speech to be processed in order to extract the speech signal-to-noise ratio; The speech signal-to-noise ratio (SNR) is the ratio of the effective speech signal energy to the background noise energy, and is used to characterize the degree to which the speech signal is affected by environmental noise.

[0025] Specifically, the speech data that needs to be estimated for noise reduction intensity coefficients is first acquired, i.e. the speech to be processed. The speech to be processed needs to meet the requirements of real-time speech streaming input. The requirements of real-time speech streaming input include continuous streaming of speech signals frame by frame, low-latency frame-by-frame processing, and the model only having a single frame input (one frame is a 256-point input, corresponding to the 256-point logarithmic amplitude spectrum mentioned above). It does not maintain the historical context state in order to reduce the model complexity used for feature extraction.

[0026] Then, feature extraction is performed on the speech to be processed. For example, the speech to be processed is input into the target neural network for feature extraction or the spectrum and power spectrum features are extracted through Fourier transform to obtain the speech signal-to-noise ratio, thereby providing a basis for subsequent speech denoising.

[0027] Step 02: Process the speech to be processed to obtain the feature parameters of the speech to be processed. The feature parameters include at least one of the following: pitch period probability, speech zero crossing rate and energy ratio. Among them, the pitch period probability is the probability value that the speech frame conforms to the pitch period pattern of human voice. It is used to determine whether the current frame is human voice and to distinguish between human voice and pure noise.

[0028] The zero-crossing rate of speech is the number of times the speech waveform crosses the zero level per unit time. It is used to characterize the fluctuations of the signal waveform and quickly distinguish between silence noise and spoken speech.

[0029] The energy ratio is the ratio of the energy of the current speech frame to the energy of the background noise frame. It directly reflects the signal-to-noise ratio of the current environment and is used to adaptively adjust the noise reduction intensity.

[0030] Specifically, during the noise reduction process, the introduction of pitch period probability, speech zero-crossing rate, and energy ratio can improve the noise reduction effect to some extent. Pitch period probability can be used to help protect voiced speech segments, so that the higher the pitch period probability, the weaker the noise reduction effect, and vice versa. Speech zero-crossing rate can be used to help protect unvoiced speech segments, so that the higher the speech zero-crossing rate, the weaker the noise reduction effect, and vice versa. Energy ratio can be used to help distinguish speech segments from noise segments, so that the higher the energy ratio, the more like a speech segment it is, and the weaker the noise reduction effect, and vice versa.

[0031] Therefore, after acquiring the speech to be processed, the speech can be further processed to obtain the feature parameters of the speech. The feature parameters include at least one of the following: pitch period probability, speech zero-crossing rate, and energy ratio. That is, the feature parameters may include any one or any two of the pitch period probability, speech zero-crossing rate, and energy ratio, or the feature parameters may include the pitch period probability, speech zero-crossing rate, and energy ratio.

[0032] Step 03: Based on the preset noise reduction algorithm, the speech to be processed is denoised according to the feature parameters in order to determine the target speech.

[0033] Specifically, the preset noise reduction algorithm can be a speech noise reduction algorithm, such as traditional speech signal processing methods and deep learning processing methods. Based on the speech noise reduction algorithm and feature parameters, speech frames and noise frames in the speech to be processed can be distinguished, the noise ratio can be adaptively determined, and the noise reduction intensity can be adjusted to determine the target speech, thereby completing the speech noise reduction processing.

[0034] In summary, the speech processing method of this application, when denoising the speech to be processed, can obtain at least one of the pitch period probability, zero-crossing rate, and energy ratio of the speech to be processed, and use the obtained parameters as feature parameters of the speech to be processed. Then, the speech to be processed is denoised according to the feature parameters and a preset denoising algorithm to obtain the final target speech. Thus, in the process of denoising the speech to be processed, at least one of the three parameters of pitch period probability, zero-crossing rate, and energy ratio can be introduced to assist in the protection of voiced and unvoiced sounds, based on the signal-to-noise ratio, and effectively identify and distinguish noise data, thereby maintaining the original fidelity of the speech as much as possible and improving the speech quality of the target speech.

[0035] Please see Figure 2 In some implementations, when the feature parameters include the pitch period probability, step 02: processes the speech to be processed to obtain feature parameters of the speech to be processed, the feature parameters including at least one of the pitch period probability, speech zero-crossing rate, and energy ratio, including: Step 021: Sample the speech to be processed to obtain the first set of sampling points; Step 022: Determine the pitch period probability based on the first sampling point set and the first preset function.

[0036] Specifically, a first set of sampling points is generated from the time-domain sampling points of the speech data to be processed. Then, the pitch period probability estimate γ is calculated based on the first set of sampling points, where the numerical range is: The first preset function includes the following formulas (1) and (2), and the solution process is shown in formulas (1) and (2).

[0037] (1) (2) Where x(n) represents the nth sampling point in the time domain, and k represents the time delay. This represents the autocorrelation value when the time delay is k. The value of k is set to the range of 20≤k≤500, and N is the total number of time-domain sampling points in a single frame, which is usually taken as 320.

[0038] Thus, the pitch period probability corresponding to the speech to be processed can be accurately obtained according to the above formulas (1) and (2).

[0039] Please see Figure 2 In some implementations, when the feature parameters include the speech zero-crossing rate, step 02: processes the speech to be processed to obtain feature parameters of the speech to be processed, the feature parameters including at least one of pitch period probability, speech zero-crossing rate, and energy ratio, including: Step 023: Sample the speech to be processed to obtain a second set of sampling points; Step 024: Determine the speech zero-crossing rate based on the second sampling point set and the second preset function.

[0040] Specifically, a second set of sampling points is generated from the time-domain sampling points of the speech data to be processed. Then, the zero-crossing rate θ of the speech is solved based on the second set of sampling points, where the numerical range is: The second preset function includes the following formula (3), and the solution process is shown in formula (3).

[0041] (3) Here, sign() is the sign function, and x(n) represents the nth sampling point in the time domain. N is the total number of time-domain sampling points in a single frame, usually taken as 320.

[0042] Thus, the zero-crossing rate of the speech to be processed can be accurately obtained according to the above formula (3).

[0043] Please see Figure 2 In some implementations, when the feature parameters include the energy ratio, step 02: processes the speech to be processed to obtain feature parameters of the speech to be processed, the feature parameters including at least one of pitch period probability, speech zero-crossing rate and energy ratio, including: Step 025: Sample the speech to be processed to obtain the third set of sampling points; Step 026: Determine the energy value of the current speech frame based on the third sampling point set, the total number of time-domain sampling points in a single frame, and the third preset function; Step 027: Determine the current noise energy value based on the preset silent voice and preset algorithm; Step 028: Determine the energy ratio based on the current speech frame energy value and the current noise energy value.

[0044] Specifically, the third sampling point set is generated from the time-domain sampling points of the speech data to be processed. Then, the energy value of the current speech frame is determined based on the third sampling point set, the total number of time-domain sampling points in a single frame, and the third preset function. The solution process is shown in formula (4).

[0045] (4) Where x(n) represents the nth sampling point in the time domain. N is the total number of time-domain sampling points in a single frame, usually taken as 320. E is the frame energy value.

[0046] Silent speech refers to quiet segments where no one is speaking. First, the initial ambient noise is measured during these quiet segments to determine the preset silent speech. Then, a preset algorithm processes the preset silent speech, ensuring that the current noise energy value changes in real-time with the ambient noise, thus determining the current noise energy value. For example, the preset algorithm may include a recursive smoothing algorithm, which updates and corrects the noise level of the preset silent speech frame by frame to determine the current noise energy value, ensuring that the current noise energy value changes in real-time with the ambient noise.

[0047] Finally, the energy ratio is determined based on the current speech frame energy value and the current noise energy value. The numerical range is: The solution process is shown in formula (5).

[0048] (5) in, The energy value of the current speech frame. The noise energy value of the current frame is given by Lg(), which is a logarithmic function to the base 10. tanh() is a hyperbolic tangent function that normalizes the energy ratio.

[0049] In this way, the energy value of the current speech frame and the energy value of the current noise can be obtained sequentially through the above method, thereby accurately obtaining the energy ratio corresponding to the speech to be processed.

[0050] In summary, the pitch period probability, speech zero-crossing rate, and energy ratio are all obtained using simple time-domain calculation methods, which can effectively reduce the amount of computation and greatly improve the real-time performance of the algorithm.

[0051] Please see Figure 3 In some implementations, the feature parameters include pitch period probability, speech zero-crossing rate, and energy ratio; Step 03: Based on a preset noise reduction algorithm, noise reduction is performed on the speech to be processed according to the feature parameters to determine the target speech, including: Step 031: Determine the control coefficients based on the feature parameters and the speech signal-to-noise ratio; Step 032: Denoise the speech to be processed according to the control coefficient and the preset noise reduction algorithm to determine the target speech.

[0052] Specifically, control coefficients are widely used in speech denoising algorithms. In speech denoising algorithms, these coefficients can be directly used in the backend of the denoising process to control the denoising intensity. Therefore, the control coefficients can be determined based on feature parameters and the speech signal-to-noise ratio. Then, the speech to be processed is denoised according to the control coefficients and a preset denoising algorithm to determine the target speech.

[0053] The control coefficients are affected by two main parameters: speech signal-to-noise ratio and pitch period probability, speech zero-crossing rate, and energy ratio. Therefore, the characteristic parameters at this time include pitch period probability, speech zero-crossing rate, and energy ratio.

[0054] The process of obtaining the control coefficient is shown in the following formula (6), and the control coefficient is obtained as ψ. The larger the value of ψ, the greater the noise reduction strength, and vice versa. In the formula, (1-δ) is used to control the global speech noise reduction benchmark so that when the signal-to-noise ratio of the speech segment is higher, the overall noise reduction strength is smaller, and vice versa; γ is used to protect voiced speech segments so that the greater the probability of the fundamental period, the smaller the noise reduction effect, and vice versa; (1) θ) is used to assist in protecting unvoiced speech segments, so that the greater the zero-crossing rate of the speech, the smaller the noise reduction effect, and vice versa; 1 ε is used to distinguish between speech segments and noise segments. The higher the energy ratio, the more it resembles a speech segment, and the weaker the noise reduction effect. Conversely, the higher the energy ratio, the stronger the noise reduction effect.

[0055] (6) in, For control coefficients, For speech signal-to-noise ratio. For the periodic probability of the fundamental tone, For the zero-crossing rate of language, λ is the energy ratio, μ is the global smoothing coefficient used to control the ratio of model output parameters to traditional speech signal output parameters, with a default value of 0.7. λ1, λ2, and λ3 are all proportional control coefficients for local traditional speech signal processing output parameters, satisfying: λ1 + λ2 + λ3 = 1.

[0056] The numerical range of the control coefficient is: 0≤ ≤1. When When the value approaches 1, deep denoising is performed on the noisy speech samples; when... When the value approaches 0, weak noise reduction is performed on noisy speech samples to maintain speech fidelity.

[0057] The process of obtaining the target speech is shown in the following formula (7).

[0058] (7) in, This indicates that the speech to be processed is processed using speech denoising algorithms, including traditional speech signal processing methods and deep learning processing methods, where x represents the speech to be processed. This represents the result of speech processing, i.e., the target speech.

[0059] In summary, when introducing control coefficients for denoising the speech to be processed, these coefficients can be determined based on the pitch period probability, the zero-crossing rate, and the energy ratio. Then, denoising is performed on the speech to be processed based on these control coefficients and a preset denoising algorithm. Thus, during the denoising process, at least one of the three parameters—pitch period probability, zero-crossing rate, and energy ratio—can be introduced to assist in the protection of voiced and unvoiced sounds, based on the signal-to-noise ratio, and effectively identify and distinguish noise data, thereby preserving the original fidelity of the speech as much as possible and improving the speech quality of the target speech. In other words, this application can combine multi-dimensional beneficial parameters (i.e., feature parameters) for control coefficient prediction, effectively improving the impact of speech denoising algorithms on speech quality and fidelity, and achieving automated control according to different noise environments.

[0060] In addition, the characteristic parameters and control coefficients are all restricted to a numerical range between [0,1] by corresponding methods, which is beneficial for parameter unification and the application of control coefficients.

[0061] In some implementations, control coefficients are used in conjunction with a preset loss function to optimize the target loss function during the training process of other neural networks, which are not used to perform speech processing methods.

[0062] Specifically, the control coefficients generated when determining the target speech can be used in the training process of other neural networks. These other neural networks refer to other deep learning-based speech denoising models, excluding the neural networks used after the official implementation of this application, because the control coefficients can only be obtained after the official implementation of this application. The control coefficients can be used to assist in optimizing the loss function during the training phase, thereby improving the learning ability of the loss function. For example, when the speech is relatively pure, the control coefficients can weaken the denoising power of the loss function while maintaining speech fidelity; conversely, the control coefficients can enhance the denoising power of the loss function. At this time, the target loss function can be determined based on the control coefficients and the preset loss function. The control method is shown in formula (8).

[0063] L1=ψ•L2(8) Where L1 represents the target loss function and L2 represents the preset loss function, which can be any loss function.

[0064] After generating the target loss function, other neural networks can be trained. During training, the loss value is calculated based on the target loss function, and the parameters of other neural networks are adjusted based on the loss value until training is complete.

[0065] Thus, the control coefficients obtained in this application can be repeatedly applied to assist the training of other neural networks, so as to optimize the learning ability of the loss function of other neural networks and thus ensure the training effect of other neural networks.

[0066] Please see Figure 4 In some implementations, step 01: extracting features from the acquired speech to be processed to extract the speech signal-to-noise ratio, including: Step 011: Obtain the speech to be processed; Step 012: Input the speech to be processed into the target neural network for feature extraction in order to extract the speech signal-to-noise ratio.

[0067] Specifically, the neural network can be pre-trained to obtain a target neural network capable of extracting the speech signal-to-noise ratio. Then, the target neural network can be officially put into use.

[0068] It should be noted that the structure of the target neural network can be set as needed and is not limited here. In one embodiment, the target neural network consists of five fully connected layers, with the dimensions of each fully connected layer being 256, 512, 64, 16, and 1, respectively. The output data of the target neural network is normalized before being output. For example, the final output of the target neural network is normalized using the Sigmoid function to facilitate fast model convergence. In this way, the model only inputs one frame of speech data each time, that is, the model input dimension of 256 corresponds to the 256-point logarithmic amplitude spectrum of one frame of speech. This allows the application to work at the frame level, meeting the real-time requirements and enabling the calculation of the comprehensive speech denoising intensity control coefficients in real time while ensuring low latency.

[0069] When noise reduction is required, the speech to be processed is first acquired. Then, the speech is input into the target neural network. For example, the speech is segmented into frames, windowed, and a Fast Fourier Transform (FFT) is used to obtain the logarithmic magnitude spectrum, acquiring 256 logarithmic magnitude spectrum features. These features are used as the input features of the target neural network. The target neural network then performs feature extraction to obtain the speech signal-to-noise ratio (SNR) of the speech to be processed. For example, the input to the target neural network is the logarithmic magnitude spectrum feature value of the current frame, meaning the input dimension of the target neural network is 256, and the output of the target neural network is the normalized SNR, with an output dimension of 1.

[0070] In this way, neural networks can be used to extract the speech signal-to-noise ratio of the speech to be processed, thereby eliminating the reliance on manual feature engineering and obtaining more discriminative, robust and generalizable feature representations, thus ensuring that the speech signal-to-noise ratio can be accurately obtained.

[0071] Please see Figure 5 In some embodiments, the speech processing method further includes: Step 04: Generate multiple training data based on clean speech samples and noisy speech samples, and determine the target signal-to-noise ratio for each training data; Step 05: Train the neural network to be trained based on each training data and the target signal-to-noise ratio corresponding to each training data to obtain the target neural network.

[0072] A clean speech sample refers to a clean speech signal that contains no background interference and only human speech semantic information.

[0073] Noisy speech samples refer to background noise signals that do not contain valid human voices and consist only of various environmental interference sounds.

[0074] Specifically, training data can be generated from clean language samples and noisy language samples. A clean speech library and a noisy speech library can be pre-set to extract clean language samples and noisy language samples respectively. For example, the clean speech library consists of 1000 hours of clean speech samples from the librispeech dataset, and the noisy speech library consists of all noisy language samples from the NOIZEUS and noise92 noise libraries.

[0075] Clean speech samples and noisy speech samples can be randomly selected from clean speech libraries and noisy speech libraries. These samples are then mixed with a random signal-to-noise ratio (SNR) or a preset SNR to obtain training data. The target SNR for each training data point is determined based on the actual SNR of the clean and noisy samples during training data generation, facilitating its use for model training labels. Specifically, when mixing clean and noisy speech samples with a random SNR, the SNR value is set to a range of -20dB to 40dB, with any value within this range. Alternatively, a preset SNR value can be pre-selected between -20dB and 40dB, and the clean and noisy speech samples are mixed according to this preset SNR during the mixing process.

[0076] Finally, the training data can be input into the neural network to be trained, with the target signal-to-noise ratio used as the model training label during training. After training is complete, the target neural network is obtained. There are several ways to determine if training is complete, such as the loss value being less than a preset value, the validation set metrics stabilizing and no longer improving, or the performance gap between the training and validation sets no longer widening. Taking the loss value being less than a preset value as an example, this can be achieved by using dropout (a neural network regularization technique proposed by Google) in the recurrent neural network layers of the neural network to be trained, using the Adam optimizer, and employing mean squared error (MSE) as the loss function. When training iterates on the training data until the MSE is less than or equal to 10, the training is considered complete. -6 When the time is up, stop the loop and save the model.

[0077] The supervised value is the normalized signal-to-noise ratio value saved when the training data is generated.

[0078] In this way, training data for training the target neural network can be automatically generated, and the accuracy of the target signal-to-noise ratio used during training can be ensured, thereby ensuring that the target neural network can accurately extract features.

[0079] Please see Figure 6 In some implementations, step 04: generating multiple training data based on clean speech samples and noisy speech samples, and determining the target signal-to-noise ratio corresponding to each training data, further includes: Step 041: Generate target training data based on clean speech samples and noisy language samples. The target training data can be any training data. Step 042: Determine the initial signal-to-noise ratio of the target training data based on the clean speech samples and noisy speech samples corresponding to the target training data; Step 043: Normalize the initial signal-to-noise ratio to determine the target signal-to-noise ratio.

[0080] Specifically, clean speech samples and noisy speech samples can be randomly selected from a clean speech database and a noisy speech database, and then mixed to generate target training data. The target training data can be any type of training data.

[0081] Then, based on the true mixing ratio of clean speech samples and noisy language samples when generating the target training data, the initial signal-to-noise ratio (SNR) of the target training data is determined. Finally, the initial SNR is normalized to determine the target SNR.

[0082] The processing procedure is shown in formula (9).

[0083] (9) Where SNR is the initial signal-to-noise ratio, and k is the nonlinear adjustment coefficient of the normalized exponential curve (base e), with the following values: The smaller the k value, the greater the linearity. The target signal-to-noise ratio has a numerical range of: .

[0084] In this way, normalization can be performed when determining the target signal-to-noise ratio to facilitate model convergence and control of noise reduction intensity.

[0085] The following describes an embodiment of the apparatus described in this application, which can be used to execute the speech processing method described in the above embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the speech processing method described in the above embodiments of this application.

[0086] Figure 7 A block diagram of a speech processing apparatus according to an embodiment of this application is shown.

[0087] Reference Figure 7As shown, a speech processing apparatus 500 according to an embodiment of this application includes a feature extraction module 510, a processing module 520, and a noise reduction module 530. The feature extraction module 510 extracts features from the acquired speech to be processed to extract the speech signal-to-noise ratio (SNR). The processing module 520 processes the speech to be processed to obtain feature parameters of the speech, including at least one of pitch period probability, speech zero-crossing rate, and energy ratio. The noise reduction module 530 performs noise reduction on the speech to be processed based on a preset noise reduction algorithm, according to the feature parameters and the speech SNR, to determine the target speech.

[0088] The processing module 520 is specifically used to sample the speech to be processed to obtain a first set of sampling points; and to determine the pitch period probability based on the first set of sampling points and a first preset function.

[0089] The processing module 520 is specifically used to sample the speech to be processed to obtain a second set of sampling points; and to determine the speech zero-crossing rate based on the second set of sampling points and a second preset function.

[0090] The processing module 520 is specifically used to sample the speech to be processed to obtain a third set of sampling points; determine the energy value of the current speech frame based on the third set of sampling points, the total number of time-domain sampling points in a single frame, and a third preset function; determine the current noise energy value based on preset silent speech and preset algorithm; and determine the energy ratio based on the current speech frame energy value and the current noise energy value.

[0091] The noise reduction module 530 is specifically used to determine the control coefficients based on the feature parameters and the speech signal-to-noise ratio; and to reduce the noise of the speech to be processed based on the control coefficients and the preset noise reduction algorithm in order to determine the target speech.

[0092] The feature extraction module 510 is specifically used to acquire the speech to be processed; input the speech to be processed into the target neural network for feature extraction in order to extract the speech signal-to-noise ratio.

[0093] The speech processing device 500 also includes a training module 540, which generates multiple training data based on clean speech samples and noisy speech samples, and determines the target signal-to-noise ratio corresponding to each training data; and trains the neural network to be trained based on each training data and the target signal-to-noise ratio corresponding to each training data to obtain the target neural network.

[0094] The training module 540 is specifically used to generate target training data based on clean speech samples and noisy language samples, where the target training data can be arbitrary; determine the initial signal-to-noise ratio (SNR) of the target training data based on the clean speech samples and noisy language samples corresponding to the target training data; and normalize the initial SNR to determine the target SNR.

[0095] Optionally, this application also provides an electronic device, including a storage device and one or more processors. The storage device stores one or more programs, and the processor executes the stored one or more programs to implement the steps in the above-described method embodiments. The electronic device can be a real-time audio and video embedded device, satisfying streaming real-time voice input processing.

[0096] In particular, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts.

[0097] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. The transmitted data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0098] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0099] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0100] In another aspect, this application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods described in the above embodiments.

[0101] This application also provides a computer program product storing at least one instruction, which is loaded and executed by the processor as described above. Figures 1-6 The method described in the illustrated embodiment can be found in the following document for a detailed execution process. Figures 1-6 The specific details of the illustrated embodiments will not be elaborated here.

[0102] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0103] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the methods according to the embodiments of this application.

[0104] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0105] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A speech processing method, characterized in that, include: Feature extraction is performed on the acquired speech to be processed in order to extract the speech signal-to-noise ratio; The speech to be processed is processed to obtain the feature parameters of the speech to be processed, the feature parameters including at least one of pitch period probability, speech zero crossing rate and energy ratio; Based on a preset noise reduction algorithm, the speech to be processed is denoised according to the feature parameters to determine the target speech.

2. The speech processing method as described in claim 1, characterized in that, When the feature parameters include the pitch period probability, the processing of the speech to be processed to obtain feature parameters of the speech to be processed, wherein the feature parameters include at least one of pitch period probability, speech zero-crossing rate, and energy ratio, including: The speech to be processed is sampled to obtain a first set of sampling points; The pitch period probability is determined based on the first set of sampling points and the first preset function.

3. The speech processing method as described in claim 1, characterized in that, When the feature parameter includes the speech zero-crossing rate, the processing of the speech to be processed to obtain the feature parameters of the speech to be processed, wherein the feature parameters include at least one of pitch period probability, speech zero-crossing rate, and energy ratio, including: The speech to be processed is sampled to obtain a second set of sampling points; The speech zero-crossing rate is determined based on the second set of sampling points and the second preset function.

4. The speech processing method as described in claim 2, characterized in that, When the feature parameter includes the energy ratio, the processing of the speech to be processed to obtain the feature parameters of the speech to be processed, wherein the feature parameters include at least one of pitch period probability, speech zero-crossing rate, and energy ratio, including: The speech to be processed is sampled to obtain a third set of sampling points; The energy value of the current speech frame is determined based on the third set of sampling points, the total number of time-domain sampling points in a single frame, and the third preset function. The current noise energy value is determined based on the preset silent voice and preset algorithm; The energy ratio is determined based on the current speech frame energy value and the current noise energy value.

5. The speech processing method as described in claim 1, characterized in that, The feature parameters include pitch period probability, speech zero-crossing rate, and energy ratio; the step of denoising the speech to be processed based on the feature parameters using a preset noise reduction algorithm to determine the target speech includes: The control coefficients are determined based on the characteristic parameters and the speech signal-to-noise ratio; The speech to be processed is denoised according to the control coefficient and the preset noise reduction algorithm to determine the target speech.

6. The speech processing method as described in claim 1, characterized in that, The control coefficients are used in conjunction with a preset loss function to optimize the target loss function during the training process of other neural networks, which are not used to execute the speech processing method.

7. The speech processing method as described in claim 1, characterized in that, The step of extracting features from the acquired speech to be processed to extract the speech signal-to-noise ratio includes: Obtain the speech to be processed; The speech to be processed is input into the target neural network for feature extraction in order to extract the speech signal-to-noise ratio.

8. The speech processing method as described in claim 7, characterized in that, Also includes: Multiple training data are generated based on clean speech samples and noisy language samples, and the target signal-to-noise ratio corresponding to each training data is determined; The target neural network is trained based on the training data and the target signal-to-noise ratio corresponding to each training data to obtain the target neural network.

9. The speech processing method as described in claim 8, characterized in that, The step of generating multiple training data based on clean speech samples and noisy speech samples, and determining the target signal-to-noise ratio corresponding to each training data, includes: Target training data is generated based on clean speech samples and noisy language samples, wherein the target training data is arbitrary training data; The initial signal-to-noise ratio of the target training data is determined based on the clean speech samples and noisy language samples corresponding to the target training data; The initial signal-to-noise ratio is normalized to determine the target signal-to-noise ratio.

10. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the speech processing method as described in any one of claims 1 to 9.

Images