A speech enhancement model training method and system based on a sub-band loss function, a terminal, and a medium

Abstract

The application discloses a speech enhancement model training method and system based on a subband loss function, a terminal and a medium, and relates to the technical field of speech enhancement. The method comprises the following steps: obtaining noisy speech and clean speech, and determining the logarithmic power spectrum of the enhanced speech and the logarithmic power spectrum of the target speech respectively; based on the mel scale, the logarithmic power spectrum of the enhanced speech and the logarithmic power spectrum of the target speech are segmented respectively to obtain the subband of the enhanced speech and the subband of the target speech; the subband loss value between each enhanced speech subband and the corresponding target speech subband is determined; a perceptual weight is assigned to each subband loss value, and the overall loss value is determined to guide the speech enhancement model training. The application can guide the speech enhancement model to exhibit differentiated learning behavior for different frequencies, so as to make the speech enhancement model output speech that is more in line with the human auditory perception law, and significantly improve the listening comfort after speech enhancement.

Description

Technical Field

[0001] This invention relates to the field of speech enhancement technology, and in particular to a method, system, terminal, and medium for training a speech enhancement model based on a subband loss function. Background Technology

[0002] Current speech enhancement technologies largely rely on spectral domain techniques, training neural network models to extract clean speech representations (such as amplitude spectra or complex masks) from noisy input. These neural network models typically optimize by minimizing the mean square error (MSE) of a given spectral representation. While the MSE loss function is favored due to its simplicity and ease of optimization, it has poor correlation with human auditory perception and suffers from two fundamental drawbacks: First, MSE imposes a uniform penalty on all frequency bands, causing the model to overemphasize low-frequency, high-energy regions while sacrificing high-frequency, low-energy components crucial for speech intelligibility; second, minimizing the spectral Euclidean distance cannot reliably predict or guarantee improvements in perceived speech quality. Although signal-to-noise ratio (SNR) and scale-invariant signal-to-noise ratio (SI-SNR) are frequently used in the time domain, the weights across all frequency ranges remain consistent. Current deep learning-based speech enhancement methods combine these loss functions, resulting in a final model output that still fails to perfectly match the human auditory system.

[0003] Therefore, existing technologies still have shortcomings. Summary of the Invention

[0004] The technical problem to be solved by this invention is to provide a method, system, terminal, and medium for training a speech enhancement model based on a subband loss function, addressing the aforementioned deficiencies of the prior art. The technical solution adopted by this invention is as follows:

[0005] In a first aspect, the present invention provides a method for training a speech enhancement model based on a subband loss function, wherein the method includes:

[0006] Acquire noisy speech and clean speech, and determine the enhanced speech log power spectrum corresponding to the noisy speech and the target speech log power spectrum corresponding to the clean speech, respectively;

[0007] Based on the Mel scale, the logarithmic power spectrum of the enhanced speech and the logarithmic power spectrum of the target speech are segmented to obtain the enhanced speech sub-band and the target speech sub-band.

[0008] Based on the difference between the enhanced speech sub-band and the target speech sub-band, a sub-band loss value between each enhanced speech sub-band and the corresponding target speech sub-band is determined.

[0009] Perceptual weights are assigned to the subband loss values ​​between each enhanced speech subband and the corresponding target speech subband, and an overall loss value is determined. The speech enhancement model is trained based on the overall loss value so that the trained speech enhancement model outputs speech that conforms to the laws of human auditory perception.

[0010] In one implementation, acquiring noisy speech and clean speech, and respectively determining the enhanced speech log power spectrum corresponding to the noisy speech and the target speech log power spectrum corresponding to the clean speech, includes:

[0011] Acquire noisy speech and clean speech, and determine the noisy amplitude spectrum corresponding to the noisy speech and the clean speech amplitude spectrum corresponding to the clean speech, respectively.

[0012] The noisy amplitude spectrum is input into the speech enhancement model, the enhanced amplitude spectrum is output, and the enhanced amplitude spectrum is squared and logarithmically divided to obtain the enhanced speech logarithmic power spectrum.

[0013] The target speech logarithmic power spectrum is obtained by squaring and taking the logarithm of the amplitude spectrum of the clean speech.

[0014] In one implementation, based on the Mel scale, the enhanced speech logarithmic power spectrum and the target speech logarithmic power spectrum are segmented to obtain enhanced speech sub-bands and target speech sub-bands, including:

[0015] Several boundary frequencies are equally spaced on the Mel scale;

[0016] Based on the Mel frequency conversion relationship, the boundary frequency is converted into a frequency index;

[0017] The enhanced speech logarithmic power spectrum and the target speech logarithmic power spectrum are grouped according to the frequency index to generate several enhanced speech sub-bands and several target speech sub-bands.

[0018] In one implementation, based on the difference between the enhanced speech sub-band and the target speech sub-band, a sub-band loss value between each enhanced speech sub-band and its corresponding target speech sub-band is determined, including:

[0019] The difference between the enhanced speech sub-band and the target speech sub-band is determined using the minimum mean square error, and the sub-band loss function is determined accordingly.

[0020] Based on the subband loss function, the subband loss value between each enhanced speech subband and the corresponding target speech subband is determined.

[0021] In one implementation, the subband loss function is expressed as:

[0022]

[0023] in, It indicates the first Subband loss values ​​between each enhanced speech subband and its corresponding target speech subband. It indicates the first One enhanced speech subband or the first Number of frequency points in each target speech subband Indicates the number of time frames. The first One target speech sub-band It indicates the first One enhanced speech subband, Indexed by frequency dimension, Indexed for the time dimension.

[0024] In one implementation, perceptual weights are assigned to the subband loss values ​​between each enhanced speech subband and the corresponding target speech subband, and the overall loss value is determined, including:

[0025] The perceptual weights of each enhanced speech sub-band and the corresponding target speech sub-band are set based on the 40-phon equal loudness curve;

[0026] Based on the perceived weights and the sub-band loss values, a weighted sub-band loss function is determined, and the overall loss value is determined based on the weighted sub-band loss function.

[0027] In one implementation, the weighted subband loss function is expressed as:

[0028]

[0029] in, Indicates the first Perceptual weights of each enhanced speech subband and its corresponding target speech subband Indicates the number of subbands.

[0030] Secondly, embodiments of the present invention also provide a speech enhancement model training system based on a subband loss function, wherein the system is used to implement the steps of the speech enhancement model training method based on a subband loss function described above, and the system includes:

[0031] The logarithmic power spectrum determination module is used to acquire noisy speech and clean speech, and to determine the enhanced speech logarithmic power spectrum corresponding to the noisy speech and the target speech logarithmic power spectrum corresponding to the clean speech, respectively.

[0032] The sub-band segmentation module is used to segment the enhanced speech log power spectrum and the target speech log power spectrum based on the Mel scale to obtain the enhanced speech sub-band and the target speech sub-band.

[0033] The subband loss determination module is used to determine the subband loss value between each enhanced speech subband and the corresponding target speech subband based on the difference between the enhanced speech subband and the target speech subband;

[0034] The total loss determination module is used to assign perceptual weights to the sub-band loss values ​​between each enhanced speech sub-band and the corresponding target speech sub-band, and to determine the total loss value. Based on the total loss value, the speech enhancement model is trained so that the trained speech enhancement model outputs speech that conforms to the laws of human auditory perception.

[0035] Thirdly, embodiments of the present invention also provide a terminal, wherein the terminal includes a memory, a processor, and a speech enhancement model training program based on a subband loss function stored in the memory and executable on the processor. When the processor executes the speech enhancement model training program based on a subband loss function, it implements the steps of the speech enhancement model training method based on a subband loss function in any of the above schemes.

[0036] Fourthly, embodiments of the present invention also provide a computer-readable storage medium, wherein a speech enhancement model training program based on a subband loss function is stored on the computer-readable storage medium, and the speech enhancement model training program based on a subband loss function implements the steps of the speech enhancement model training method based on a subband loss function as described in any of the above schemes on the computer-readable storage medium.

[0037] Beneficial Effects: Compared with existing technologies, this invention provides a speech enhancement model training method based on subband loss function. First, noisy and clean speech are acquired, and the logarithmic power spectrum of the enhanced speech corresponding to the noisy speech and the logarithmic power spectrum of the target speech corresponding to the clean speech are determined respectively. Then, based on the Mel scale, the logarithmic power spectra of the enhanced and target speech are segmented to obtain enhanced and target speech subbands. Next, based on the difference between the enhanced and target speech subbands, a subband loss value is determined between each enhanced subband and its corresponding target speech subband. Finally, perceptual weights are assigned to each subband loss value between the enhanced and target speech subbands, and an overall loss value is determined. The speech enhancement model is trained based on this overall loss value, so that the trained speech enhancement model outputs speech that conforms to the laws of human auditory perception. This invention can guide the speech enhancement model to exhibit differentiated learning behaviors for different frequencies, strengthen detail optimization in sensitive frequency bands, and rationally allocate computing resources in non-sensitive frequency bands, thereby making the output of the speech enhancement model more in line with the human ear's perception rules and significantly improving the listening comfort after speech enhancement. Attached Figure Description

[0038] Figure 1This is a flowchart of a preferred embodiment of the speech enhancement model training method based on subband loss function provided in this invention.

[0039] Figure 2 This is a schematic diagram illustrating the application process of the speech enhancement model training method based on subband loss function provided in an embodiment of the present invention.

[0040] Figure 3 This is a block diagram illustrating the principle of a speech enhancement model training system based on subband loss function, as provided in an embodiment of the present invention.

[0041] Figure 4 A schematic diagram of a terminal provided in an embodiment of the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0043] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content, operations, or steps, nor does it require execution in the described order. For example, some operations or steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0044] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0045] It should be understood that, in order to clearly describe the technical solutions of the embodiments of the present invention, the terms "first" and "second" are used in the embodiments of the present invention to distinguish identical or similar items with essentially the same function and effect. For example, "first control information" and "second control information" are only used to distinguish different control information and do not limit their order.

[0046] Those skilled in the art will understand that the words "first" and "second" do not limit the quantity or the order of execution, and that the words "first" and "second" do not necessarily imply that they are different.

[0047] It should also be understood that the term “and / or” as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0048] The core innovation of this embodiment lies in its precise alignment with the frequency perception characteristics of the human auditory system—the human ear is most sensitive to the 2kHz-5kHz frequency band, while its sensitivity to high and low frequencies is significantly reduced. Traditional loss functions often fail to reflect this non-uniform sensitivity difference. Therefore, this embodiment provides a speech enhancement model training method based on a subband loss function. This method guides the speech enhancement model to exhibit differentiated learning behavior for different frequencies, strengthening detail optimization in sensitive frequency bands and rationally allocating computational resources in non-sensitive frequency bands. This allows the speech enhancement model output to better align with human auditory perception patterns, significantly improving the listening comfort after speech enhancement. In specific applications, this embodiment first acquires noisy and clean speech, and determines the logarithmic power spectrum of the enhanced speech corresponding to the noisy speech and the logarithmic power spectrum of the target speech corresponding to the clean speech, respectively. Then, based on the Mel scale, the logarithmic power spectra of the enhanced and target speech are segmented to obtain enhanced and target speech subbands. Next, based on the differences between the enhanced and target speech subbands, the subband loss value between each enhanced speech subband and its corresponding target speech subband is determined. Finally, perceptual weights are assigned to the subband loss values ​​between each enhanced speech subband and the corresponding target speech subband, and an overall loss value is determined. The speech enhancement model is then trained based on the overall loss value so that the trained speech enhancement model outputs speech that conforms to the laws of human auditory perception.

[0049] The speech enhancement model training method based on subband loss function in this embodiment can be applied to a terminal, such as a computer or other intelligent product terminal. Figure 1 As shown, the speech enhancement model training method based on the subband loss function includes the following steps:

[0050] Step S100: Obtain noisy speech and clean speech, and determine the enhanced speech log power spectrum corresponding to the noisy speech and the target speech log power spectrum corresponding to the clean speech, respectively.

[0051] In practical applications, combined with Figure 2 As shown in the figure, this embodiment first obtains noisy speech and clean speech. Then, based on short-time fast Fourier transform and modulus taking, the noisy amplitude spectrum corresponding to the noisy speech is determined. and the clean speech amplitude spectrum corresponding to the clean speech. The noisy amplitude spectrum Input speech enhancement model, output enhancement amplitude spectrum and the enhanced amplitude spectrum Performing squaring and logarithmic operations yields the enhanced speech logarithmic power spectrum. The specific formula is expressed as follows: Similarly, for the clean speech amplitude spectrum By performing squaring and logarithmic operations, the logarithmic power spectrum of the target speech is obtained. The specific formula is expressed as follows: .

[0052] Because human hearing perceives sound intensity on a logarithmic scale (dB), rather than a nonlinear physical sound pressure level, this characteristic is determined by equal-loudness curves. These curves describe the relationship between sound pressure levels perceived as equally loud at different frequencies. Therefore, converting the original physical amplitude spectrum into the logarithmic power domain corresponding to human hearing provides a fundamental adaptation space for all subsequent calculations (banding, weighting) based on human hearing characteristics. If a linear amplitude spectrum is used directly, subsequent steps cannot match the human ear's perception of intensity, leading to a disconnect between loss calculations and auditory perception.

[0053] Step S200: Based on the Mel scale, the enhanced speech log power spectrum and the target speech log power spectrum are segmented to obtain the enhanced speech sub-band and the target speech sub-band.

[0054] Human frequency perception is non-linear; the auditory system exhibits high frequency resolution at lower frequencies and low frequency resolution at higher frequencies. The Mel scale effectively simulates this psychoacoustic characteristic. The Mel scale is a psychoacoustic scale used to simulate the non-linearity of human frequency perception; it is approximately linear below 1 kHz and logarithmic above 1 kHz. Therefore, this embodiment divides the logarithmic power spectrum of the enhanced speech and the logarithmic power spectrum of the target speech into perceptually meaningful sub-bands based on the Mel scale.

[0055] To simulate the non-uniform frequency resolution of the human auditory system, such as Figure 2 As shown, this embodiment describes the enhanced speech logarithmic power spectrum. Logarithmic power spectrum of target speech Perform Mel-scale subband segmentation, dividing into Several overlapping sub-bands. In specific operation, this embodiment first divides the area at equal intervals on the Mel scale (specifically, several...). (Number) boundary frequencies, boundary frequencies Represented as ,at this time Indicates the first There are several boundary frequencies. Next, in this embodiment, based on the Mel frequency transformation relationship, the boundary frequencies are converted into frequency indices. The Mel frequency transformation relationship includes a linear frequency to Mel scale formula and a Mel scale to linear frequency formula. The linear frequency to Mel scale formula is: ,in, For the Mel scale, The frequency is linear. The formula for converting the Mel-scale to a linear frequency is: Based on this Mel frequency conversion relationship, boundary frequencies can be mapped to their corresponding frequency indices. Above, represented as Then, in this embodiment, the enhanced speech logarithmic power spectrum and the target speech logarithmic power spectrum are grouped according to the frequency index to generate several enhanced speech sub-bands and several target speech sub-bands. In specific applications, in order to create spectrally smooth sub-bands with 50% overlap, this embodiment further refines the target speech logarithmic power spectrum... The target speech sub-band obtained after segmentation is represented as follows: ,in Indicates the number of time frames. Representing real numbers, the frequency is indexed from the frequency index. arrive Grouping is formed. Represented as the first The number of frequency points in each subband is Similarly, for enhanced speech logarithmic power spectrum... Enhanced speech subbands obtained after segmentation This implementation separates the frequency bands that the human ear is sensitive to (such as 2kHz-5kHz) from the high and low frequency bands that are not sensitive to them through subband segmentation, providing a basis for analyzing subband loss and overall loss in subsequent steps.

[0056] Step S300: Based on the difference between the enhanced speech sub-band and the target speech sub-band, determine the sub-band loss value between each enhanced speech sub-band and the corresponding target speech sub-band.

[0057] After dividing into enhanced speech subbands and target speech subband Subsequently, this embodiment can use any distance metric to measure the enhanced speech subband. and target speech subband The distance between them. Specifically, combining Figure 2 As shown in the illustration, this embodiment uses the minimum mean square error (MSE) to determine the difference between the enhanced speech sub-band and the target speech sub-band, and determines the sub-band loss function. Then, based on the sub-band loss function, each enhanced speech sub-band is determined. With the corresponding target speech sub-band Subband loss values ​​between Specifically, the subband loss function is expressed as:

[0058]

[0059] In the above subband loss function, It indicates the first Subband loss values ​​between each enhanced speech subband and its corresponding target speech subband. It indicates the first One enhanced speech subband or the first Number of frequency points in each target speech subband Indicates the number of time frames. The first One target speech sub-band It indicates the first One enhanced speech subband, Indexed by frequency dimension, This is an index for the time dimension. This embodiment uses minimum mean square error to calculate subband loss, which can stably quantize and enhance speech subbands. and target speech subband It measures the overall deviation in two dimensions: frequency and time, and is simple to calculate and easy to backpropagate.

[0060] Step S400: Assign perceptual weights to the subband loss values ​​between each enhanced speech subband and the corresponding target speech subband, and determine the overall loss value. Guide the speech enhancement model training based on the overall loss value so that the trained speech enhancement model outputs speech that conforms to the laws of human auditory perception.

[0061] This embodiment assigns perceptual weights to the subband loss values ​​between each enhanced speech subband and the corresponding target speech subband to impose a more severe penalty on errors in perceptually sensitive frequency bands. Specifically, a 40-phon equal-loudness curve (an equal-loudness curve simulating the loudness level of typical dialogue) can be used as the weighting basis because it accurately reflects the characteristics of typical conversational speech. This 40-phon equal-loudness curve shows that human auditory sensitivity changes with frequency; specifically, lower sound pressure levels (SPL) on the curve correspond to lower hearing thresholds, meaning the ear is more sensitive in that frequency band.

[0062] Since 1000 Hz is a specific reference frequency used to measure the loudness curve of a 40-phon equal loudness band, this embodiment assigns a perceptual weight to each sub-band. Perceptual weight is defined as the reciprocal of the 40-phon hearing threshold relative to that specific reference frequency. Specifically:

[0063]

[0064] in, It is a function that returns a given frequency. The sound pressure level of the absolute hearing threshold. In specific implementation, this embodiment can achieve this through nearest neighbor search using the frequency and sound pressure level comparison table provided in Table 1 below. This perceptual weight... The formula ensures that frequency bands with lower hearing thresholds (i.e. higher sensitivity) receive greater weight.

[0065] Table 1

[0066]

[0067] Next, as Figure 2 As shown in the figure, this embodiment is based on the perceptual weights and the sub-band loss values. Determine the weighted subband loss function, and determine the overall loss value based on the weighted subband loss function. The overall loss is defined as the average of the perceived weighted subband losses; therefore, the weighted subband loss function is expressed as:

[0068]

[0069] in, Indicates the first Perceptual weights of each enhanced speech subband and its corresponding target speech subband Indicates the number of subbands.

[0070] After determining the overall loss value, it can be used to guide the training of the speech enhancement model. This guides the model to exhibit differentiated learning behaviors for different frequencies: enhancing detail optimization in sensitive frequency bands and rationally allocating computing resources in non-sensitive frequency bands. This allows the speech enhancement model to output speech that more closely resembles human auditory perception, significantly improving the listening comfort of the enhanced speech. This invention has wide applications, such as in mobile terminals, wearable devices, in-vehicle voice systems, voice conferencing systems, and intelligent voice assistants. All devices requiring AI speech noise reduction can use this invention to further optimize the speech enhancement model, making its expressive behavior more aligned with human auditory perception.

[0071] This embodiment further provides an analysis of the advantages of the loss function used in this embodiment, as shown in Table 2. Loud-loss is used to represent the loss function in this embodiment, and MSE is the minimum mean squared error loss function. Exp.ID refers to the experiment number. WB-PESQ (Wideband-Perceptual Evaluation of Speech Quality) is a wideband perceptual evaluation of speech quality, NB-PESQ (Narrowband-Perceptual Evaluation of Speech Quality) is a narrowband perceptual evaluation of speech quality, ESTOI (Extended Short-Time Objective Intelligibility) is an extended objective intelligibility, STOI (Short-Time Objective Intelligibility) is a short-time objective intelligibility, SNR (Signal to Noise Ratio) is a signal-to-noise ratio, and SI-SNR is a scale-invariant signal-to-noise ratio. The GTCRN model and ICCRN model are specific neural network structures (such as speech enhancement models based on gated recurrent, convolution, etc.). The † and ‡ symbols in Table 2 are used to identify different improved versions of the GTCRN model. Therefore, GTCRN† and GTCRN‡ are different improved versions of the GTCRN model. GTCRN†+PCS represents the GTCRN† model incorporating the Perceptual Compensation Strategy module. Unprocessed refers to the noisy speech without processing. Compressed represents the compression coefficient.

[0072] Table 2

[0073]

[0074] Specifically, as can be seen from Table 2, this embodiment has at least the following advantages:

[0075] 1. Superior to MSE: Loud-loss demonstrates a significant advantage over the standard MSE baseline across all test architectures. For the mapping-based GTCRN† (Experiment 1 vs. 3), the method in this embodiment improves WB-PESQ by 0.76 and ESOI by 0.024. This trend also applies to the masking-based GTCRN‡ (Experiment 4 vs. 5) and ICCRN models (Experiment 6 vs. 7). The significant improvements in perceptual metrics (such as PESQ) and intelligibility metrics (such as ESOI) highlight the importance of incorporating psychoacoustic principles into the loss function. MSE loss, by uniformly weighting all frequency bands, forces the model to prioritize high-energy low-frequency regions. This leads to higher-order harmonic blurring and inaccurate estimation of key high-frequency components. Notably, although loudness loss performs well in perceptual metrics, its scores in signal-to-noise ratio (SNR) and spectral information content-to-signal-to-noise ratio (SI-SNR) are still lower than the MSE baseline. This trade-off is expected, not limiting, and instead validates the method in this embodiment. Signal-to-noise ratio (SNR) and spectral information content (SNR) are essentially correlated with MSE distance. The lower scores on these metrics, coupled with excellent performance in PESQ and ESOI, strongly demonstrate that loudness loss successfully guides the model optimization focus from preserving the original energy fidelity to preserving human perception fidelity, which is the core argument of this invention.

[0076] 2. Generality and Flexibility: Loud-loss exhibits strong generality. First, this loss function performs excellently in both mapping-based and mask-based augmentation frameworks, providing significant improvements over mean squared error (MSE) in both scenarios. Second, this loss function is model-independent, continuing to provide improvements even when integrated into GTCRN and ICCRN models with vastly different architectures.

[0077] 3. Comparison with State-of-the-art Benchmark Methods: The method in this embodiment also outperforms other state-of-the-art training objectives, such as spectral compression (Experiments 9 and 10) and the perception-driven PCS method (Experiment 2). While the spectral compression method using the optimal compression factor α = 0.3 achieves competitive results by reducing the spectral dynamic range, the method in this embodiment still achieves a higher WB-PESQ score (2.93 in Experiment 3 compared to 2.85 in Experiment 9). More importantly, the method in this embodiment has a significant advantage in interpretability: its weight parameters are strictly derived from the psychoacoustic model, and the optimal compression factor α needs empirical validation and is highly sensitive, as evidenced by the significant performance drop when α is adjusted to 0.7. Furthermore, compared to the perception-driven PCS method (Experiment 2), which introduces input-target mismatch, the method in this embodiment demonstrates a comprehensive advantage across all reporting metrics, exhibiting superior overall performance.

[0078] 4. Compatibility with other loss functions: The Loud-loss loss function is highly compatible with other loss functions. To verify this, in Experiment 8, this embodiment combines it with the SI-SNR loss term. This composite loss function achieved the best SI-SNR score of 18.44 (a significant improvement over 14.41 in Experiment 3), while the WB-PESQ only decreased by 0.01 points, which is almost negligible. This result highlights the flexibility of the method in this embodiment, enabling the construction of a balanced multi-objective loss function that can be customized according to specific application needs.

[0079] To verify the effectiveness of different noise reduction designs, this embodiment conducted a series of ablation experiments using the mapping-based GTCRN† as the baseline model. The experimental results are detailed in Table 3. The baseline experiment (Experiment 3) in this embodiment employed the Mel band segmentation method with a 50% overlap parameter. First, the impact of the overlap parameter was investigated: removing this parameter (Experiment 11) resulted in a slight decrease in the PESQ index, which may be due to the discontinuity at the sub-band edges during optimization. Subsequently, the Mel band was replaced with a uniform Hz band for band segmentation (Experiment 12), and the WB-PESQ value significantly decreased from 2.93 to 2.40, confirming that the perception-driven Mel band is crucial for allocating model capacity in different frequency regions. Finally, completely discarding the sub-bands and directly applying interpolation weights to each band (Experiment 13) also produced poor results, indicating that the sub-band averaging mechanism is beneficial.

[0080] Table 3

[0081]

[0082] To verify the psychoacoustic weighting scheme proposed in this embodiment, it is compared with two alternative schemes. First, in Experiment 14, this embodiment applies uniform weights to all sub-bands, ensuring each band contributes equally to the total loss. This approach, equivalent to mean squared error loss in the logarithmic power domain, leads to a precipitous drop in performance, with the WB-PESQ score plummeting from 2.93 to 2.60. In Experiment 15, this embodiment replaces the weights based on equal loudness profiles with PCS method weights based on the frequency band importance function, resulting in a significant decrease in WB-PESQ (2.93 vs. 2.70). These results confirm that the weighting scheme proposed in this embodiment, which is more directly related to the psychoacoustic loudness perception model, is more advantageous in improving overall perceived quality.

[0083] Applying the loss function in the logarithmic power domain is a critical design choice. This embodiment validates its importance in Experiment 16, where decibel-based perceived importance weights are directly applied to the loss function used in linear amplitude spectrum calculations. This domain mismatch leads to a severe performance degradation, with the WB-PESQ score plummeting from 2.93 to 2.05. This result confirms that for perceptual weighting to be effective, the spectral characteristics must be represented in a domain mathematically consistent with the weights themselves (the logarithmic power domain).

[0084] Based on the above embodiments, the present invention also provides a speech enhancement model training system based on a sub-band loss function, the system being used to implement the steps of the above method embodiments. Specifically, as follows... Figure 3 As shown, the system includes: a logarithmic power spectrum determination module 10, a sub-band segmentation module 20, a sub-band loss determination module 30, and a total loss determination module 40. Specifically, the logarithmic power spectrum determination module 10 is used to acquire noisy speech and clean speech, and determine the logarithmic power spectrum of the enhanced speech corresponding to the noisy speech and the logarithmic power spectrum of the target speech corresponding to the clean speech, respectively. The sub-band segmentation module 20 is used to segment the logarithmic power spectrum of the enhanced speech and the logarithmic power spectrum of the target speech based on the Mel scale, respectively, to obtain enhanced speech sub-bands and target speech sub-bands. The sub-band loss determination module 30 is used to determine the sub-band loss value between each enhanced speech sub-band and its corresponding target speech sub-band based on the difference between the enhanced speech sub-band and the target speech sub-band. The total loss determination module 40 is used to assign perceptual weights to the sub-band loss value between each enhanced speech sub-band and its corresponding target speech sub-band, determine the overall loss value, and guide the training of the speech enhancement model based on the overall loss value, so that the trained speech enhancement model outputs speech that conforms to the laws of human auditory perception.

[0085] The speech enhancement model training system based on subband loss function in this embodiment is based on the same principle as the steps in the above method embodiments, and will not be repeated here.

[0086] Based on the above embodiments, the present invention also provides a terminal, the principle block diagram of which can be as follows: Figure 4 As shown. The terminal may include one or more processors 100 ( Figure 4(Only one is shown in the image), memory 101, and computer program 102 stored in memory 101 and executable on one or more processors 100. For example, a speech enhancement model training program based on a subband loss function. When one or more processors 100 execute computer program 102, they can implement the various steps in the embodiment of the speech enhancement model training method based on a subband loss function. Alternatively, when one or more processors 100 execute computer program 102, they can implement the functions of various modules / units in the embodiment of the speech enhancement model training system based on a subband loss function, which is not limited here.

[0087] In one embodiment, the processor 100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0088] In one embodiment, memory 101 may be an internal storage unit of an electronic device, such as a hard drive or RAM. Memory 101 may also be an external storage device of the electronic device, such as a plug-in hard drive, smart media card (SMC), secure digital card (SD), flash card, etc. Furthermore, memory 101 may include both internal and external storage units. Memory 101 is used to store computer programs and other programs and data required by the terminal. Memory 101 can also be used to temporarily store data that has been output or will be output.

[0089] Those skilled in the art will understand that Figure 4 The block diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the terminal to which the present invention is applied. A specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0090] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, operational databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual operating data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for training a speech enhancement model based on a subband loss function, characterized in that, The method comprises: obtaining noisy speech and clean speech, and determining the corresponding enhanced speech log power spectrum of the noisy speech and the corresponding target speech log power spectrum of the clean speech respectively; based on the mel scale, the enhanced speech log power spectrum and the target speech log power spectrum are segmented respectively to obtain the enhanced speech subband and the target speech subband; based on the difference between the enhanced speech subband and the target speech subband, the subband loss value between each enhanced speech subband and the corresponding target speech subband is determined; assign a perceptual weight to the subband loss value between each enhanced speech subband and the corresponding target speech subband, and determine the overall loss value, based on the overall loss value to guide the speech enhancement model training, so that the trained speech enhancement model outputs the speech that conforms to the human auditory perception law; based on the mel scale, the enhanced speech log power spectrum and the target speech log power spectrum are segmented respectively to obtain the enhanced speech subband and the target speech subband, comprising: equidistantly divide a plurality of boundary frequencies on the mel scale; based on the mel frequency conversion relationship, the boundary frequency is converted into a frequency index; group the enhanced speech log power spectrum and the target speech log power spectrum according to the frequency index to generate a plurality of enhanced speech subbands and a plurality of target speech subbands; based on the difference between the enhanced speech subband and the target speech subband, the subband loss value between each enhanced speech subband and the corresponding target speech subband is determined, comprising: determine the difference between the enhanced speech subband and the target speech subband by using the least mean square error, and determine the subband loss function; based on the subband loss function, the subband loss value between each enhanced speech subband and the corresponding target speech subband is determined; assign a perceptual weight to the subband loss value between each enhanced speech subband and the corresponding target speech subband, and determine the overall loss value, comprising: based on the 40 phon equal response curve, the perceptual weight of each enhanced speech subband and the corresponding target speech subband is set; based on the perceptual weight and the subband loss value, the weighted subband loss function is determined, and the overall loss value is determined based on the weighted subband loss function; wherein the perceptual weight is represented as: wherein, is a function for returning a sound pressure level of an absolute hearing threshold for a given frequency , which can be implemented by a nearest neighbor lookup according to a preset frequency and sound pressure level table, denotes a boundary frequency, denotes the boundary frequency, denotes a number of overlapping subbands obtained by performing Mel-scale subband division on the enhanced speech log power spectrum and the target speech log power spectrum.

2. The method of claim 1, wherein the sub-band loss function-based speech enhancement model training method is characterized by, obtaining noisy speech and clean speech, and determining the corresponding enhanced speech log power spectrum of the noisy speech and the corresponding target speech log power spectrum of the clean speech, comprising: obtaining noisy speech and clean speech, and determining the corresponding noisy amplitude spectrum of the noisy speech and the corresponding clean speech amplitude spectrum of the clean speech; input the noisy amplitude spectrum into the speech enhancement model, output the enhanced amplitude spectrum, and perform square and logarithm operation on the enhanced amplitude spectrum to obtain the enhanced speech log power spectrum; perform square and logarithm operation on the clean speech amplitude spectrum to obtain the target speech log power spectrum. 3.The method of claim 1, wherein, The subband loss function is represented as: wherein, represents a subband loss value between the th enhanced speech subband and the corresponding target speech subband, represents a frequency point number in the th enhanced speech subband or the th target speech subband, represents a time frame number, represents the th target speech subband, represents the th enhanced speech subband, is a frequency dimension index, is a time dimension index.

4. The method of claim 1, wherein the sub-band loss function-based speech enhancement model training method is characterized by, The weighted subband loss function is represented as: wherein, represents the number of enhanced speech subbands, represents the perceptual weight of the represents the number of subbands. 5.A speech enhancement model training system based on a subband loss function, characterized in that, The system is used to realize the steps of the subband loss function based speech enhancement model training method according to any one of claims 1-4, and the system comprises: The log power spectrum determination module is configured to obtain noisy speech and clean speech, and determine an enhanced speech log power spectrum corresponding to the noisy speech and a target speech log power spectrum corresponding to the clean speech, respectively; The sub-band division module is configured to divide the enhanced speech log power spectrum and the target speech log power spectrum based on a mel scale, to obtain an enhanced speech sub-band and a target speech sub-band, respectively. The sub-band loss determination module is configured to determine a sub-band loss value between each enhanced speech sub-band and a corresponding target speech sub-band based on a difference between the enhanced speech sub-band and the target speech sub-band. The total loss determination module is configured to assign a perceptual weight to the sub-band loss value between each enhanced speech sub-band and a corresponding target speech sub-band, and determine a total loss value, and guide training of a speech enhancement model based on the total loss value, so that the trained speech enhancement model outputs speech that conforms to human auditory perception rules.

6. A terminal, characterized by comprising: The terminal includes a memory, a processor, and a sub-band loss function-based speech enhancement model training program stored in the memory and executable on the processor. When the processor executes the sub-band loss function-based speech enhancement model training program, the steps of the sub-band loss function-based speech enhancement model training method according to any one of claims 1-4 are implemented.

7. A computer readable storage medium characterized in that, The computer-readable storage medium stores a sub-band loss function-based speech enhancement model training program, which implements the steps of the sub-band loss function-based speech enhancement model training method according to any one of claims 1-4 on the computer-readable storage medium.

Images