Segmented energy weighted noisy speech generation method and system, medium and equipment

Abstract

The invention discloses a segmented energy weighted noisy voice generation method and system, a medium and equipment, and belongs to the field of voice communication and audio processing, the method comprises the following steps: acquiring original human sound and noise audio which are equal in length, and calculating energy weight of each window of the original human sound and the noise audio according to a preset signal window; combining the energy weight of each window and the energy of the sampling point, constructing a weighted energy equation, and obtaining first weighted energy of human voice and second weighted energy of noise; and according to the signal amplitude scaling coefficients of the original noise and the noise to be added, third weighted energy of the noise to be added is constructed and calculated. And utilizing the target signal-to-noise ratio, the first weighted energy and the third weighted energy to solve an amplitude scaling coefficient through a signal-to-noise ratio equation, adjusting the original noise to generate noise to be added, and mixing the noise with the original human sound to obtain noisy voice conforming to the target signal-to-noise ratio. Therefore, by implementing the method and the device, the problem that the noisy voice generated in the prior art does not conform to the auditory perception characteristic of the human ear to the unsteady noise can be solved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of speech communication and audio processing, and relates to a segment energy weighted noisy speech generation method, system, medium and device. BACKGROUND

[0002] In the field of speech communication and audio processing, the synthesis of noisy speech is a key link for evaluating the performance of noise reduction algorithms and training speech models. The core requirement is to generate noisy speech data consistent with the real environment listening. In the prior art, the traditional noisy speech generation method determines the noise scaling ratio by calculating the average power ratio of the signal to the noise (signal-to-noise ratio), but this method assumes that the noise is a stationary signal and cannot adapt to non-stationary noise with varying intensity over time in actual scenarios. At the same time, some improved schemes, such as the robust signal-to-noise ratio estimation technology based on the derivative envelope of the speech signal proposed by Shome et al., use a frequency domain segment weighting strategy, which requires adjusting the weight coefficients for different scenarios, resulting in high computational complexity and poor universality. The listening of the generated noisy speech is significantly deviated from the theoretical signal-to-noise ratio result. SUMMARY

[0003] The present application provides a segment energy weighted noisy speech generation method, system, medium and device, which can solve the problem that the generated noisy speech in the prior art does not conform to the auditory perception characteristics of non-stationary noise.

[0004] To achieve the above-mentioned purpose, in a first aspect, the present application provides a segment energy weighted noisy speech generation method, comprising:

[0005] obtaining original human voice audio and original noise audio with equal time length;

[0006] based on a preset signal window, calculating the energy weight of the audio signal of each signal window on the original human voice audio and the original noise audio, respectively;

[0007] constructing a weighted energy calculation equation according to the energy weight and the sampling point energy of the audio signal corresponding to each signal window, and calculating the first weighted speech energy of the original human voice audio and the second weighted speech energy of the original noise audio, respectively;

[0008] based on the preset signal amplitude scaling coefficient between the original noise audio and the noise to be added, combining the weighted energy calculation equation, constructing a third weighted energy calculation equation of the noise to be added, and calculating the third weighted speech energy of the noise to be added;

[0009] combining the preset target signal-to-noise ratio, the first weighted speech energy and the third weighted speech energy, solving the signal amplitude scaling coefficient in the preset signal-to-noise ratio calculation equation;

[0010] According to the signal amplitude scaling coefficient, the original noise audio is adjusted to generate a to-be-noised audio, and the original human voice audio is combined to generate a noise speech in line with the target signal-to-noise ratio.

[0011] Compared with the prior art, the embodiments of the present application have the following beneficial effects: by acquiring original human voice and noise audios of equal time length, the synchronization of subsequent energy calculation and noise adding processing is ensured, and energy deviation caused by time length difference is avoided; based on a preset signal window, energy weights of each window are calculated, and by energy proportion quantization, differential weighting of a speech active segment (high energy) and a silent segment (low energy) is realized, so that noise energy adjustment is focused on a human ear sensitive speech active period; a weighted energy equation is constructed to obtain first and second weighted speech energies, by segmented energy representation, the ability to capture the energy distribution of time is refined, and the rapid energy change characteristics of non-stationary noise (such as burstiness and volatility) are adapted; a third weighted energy equation of the to-be-noised noise is derived by combining the amplitude scaling coefficient, the adjustment of noise and the segmented weighted energy are directly related, and it is ensured that the adjustment proportion of noise energy in each time window is dynamically matched with the speech energy; by solving the scaling coefficient according to the target signal-to-noise ratio and mixing the speech, the noise speech is finally generated, a closed loop of "perception demand-energy calculation-intensity regulation" is formed, the noise energy distribution is accurately matched with the time-varying perception characteristics of the human ear to non-stationary noise, and the technical contradiction of the traditional method that "the calculated signal-to-noise ratio meets the standard but the listening feeling is distorted" is solved.

[0012] In some embodiments of the first aspect of the present application, the energy weight of the audio signal of each signal window is calculated based on a preset signal window on the original human voice audio and the original noise audio, respectively, and the calculation includes:

[0013] In the time domain, the original human voice audio and the original noise audio are windowed respectively, and the energy weight of the corresponding audio signal in each signal window is calculated; wherein the calculation formula is as follows:

[0014] wherein E speech (k) and E noise (k) represent the energy of the original human voice signal and the original noise signal in the kth window, respectively, E speech_total and E noise_total represent the total energy of all windows of the original human voice audio and the original noise audio, respectively, w speech (k) and w noise (k) represent the energy weight of the original human voice signal and the original noise signal in the kth window, respectively.

[0015] Compared with the prior art, the above-mentioned embodiments have the beneficial effects that: by windowing the audio in the time domain and calculating the energy weight, the weight value is quantified by the window energy proportion, the dynamic focusing of the voice activity segment (high energy window) is realized, the interference of the mute segment (low energy window) on the overall energy calculation is reduced, the time domain pertinence of the energy weight distribution is improved, and accurate weight parameter support is provided for the subsequent weighted energy equation.

[0016] In some embodiments of the first aspect of the application, the construction of the weighted energy calculation equation according to the energy weight and the sample point energy corresponding to the audio signal in each signal window, and the calculation of the first weighted voice energy of the original human voice audio and the second weighted voice energy of the original noise audio respectively, comprises:

[0017] On the original human voice audio, a first weighted energy calculation equation is constructed based on the energy weight and the sample point energy corresponding to each signal window, and the first weighted voice energy is solved; wherein the first weighted energy calculation equation is expressed as follows: wherein E speech represents the first weighted voice energy of the original human voice audio, E speech_max_energy (k) represents the sample point energy of the original human voice signal at the maximum absolute value in the window k, w speech (k) represents the energy weight of the original human voice signal in the kth window.

[0018] On the original noise audio, a second weighted energy calculation equation is constructed based on the energy weight and the sample point energy corresponding to each signal window, and the second weighted voice energy is solved; wherein the second weighted energy calculation equation is expressed as follows: wherein E noise represents the second weighted voice energy of the original noise audio, E noise_max_energy (k) represents the sample point energy of the original noise signal at the maximum absolute value in the window k, w noise (k) represents the energy weight of the original noise signal in the kth window, and N represents the total number of signal windows of the original human voice audio and the total number of signal windows of the original noise audio.

[0019] Compared with the prior art, the above-mentioned embodiments have the beneficial effects that: based on the energy weight and the sample point energy, the weighted energy calculation equation is constructed, the proportion of the high energy voice period in the overall energy calculation is strengthened by summing the products of the maximum energy and the corresponding weight, the sensitivity of the energy representation to the time domain change of the voice signal is improved, and the problem of loss of non-stationary noise energy characteristics caused by traditional full time domain average energy calculation is avoided.

[0020] In some embodiments of the first aspect of the application, the third weighted energy calculation equation of the to-be-added noise is constructed based on the preset signal amplitude scaling coefficient between the original noise signal and the to-be-added noise, combined with the weighted energy calculation equation, and the third weighted speech energy of the to-be-added noise is calculated, including:

[0021] The numerical relationship between the sample point energy at the maximum amplitude absolute value of the original noise signal in the signal window and the maximum signal amplitude is represented as follows:

[0022] E noise_max_energy (k) = noise_max_amp(k) 2 ; wherein noise_max_amp(k) represents

[0023] The maximum signal amplitude of the original noise signal in the signal window k;

[0024] The relationship of the maximum signal amplitude between the original noise signal and the to-be-added noise is represented as follows: add_noise_max_amp(k) = m x noise_max_amp(k); wherein m represents the signal amplitude scaling coefficient between the original noise signal and the to-be-added noise, and add_noise_max_amp(k) represents the maximum signal amplitude of the to-be-added noise in the signal window k,

[0025] The energy weight between the original noise signal and the to-be-added noise is represented as follows:

[0026] w add_noise (k) = w noise (k); wherein w add_noise (k) represents the energy weight of the to-be-added noise signal in the kth window;

[0027] Combined with the above three equations, the third weighted energy calculation equation of the to-be-added noise is constructed and the third weighted speech energy is solved, represented as follows:

[0028]

[0029]

[0030] wherein E add_noise represents the third weighted speech energy of the to-be-added noise.

[0031] Compared with the prior art, the above-mentioned embodiment has the following beneficial effects: by establishing the amplitude scaling relationship between the original noise and the to-be-added noise, the square term of the amplitude scaling coefficient is directly introduced into the noise energy calculation, the amplitude to energy dimensional conversion is realized without introducing additional parameters, the mathematical correlation between the noise energy adjustment process and the amplitude scaling coefficient is ensured, the consistency of the weight parameter in the noise energy calculation is maintained, the mapping logic from amplitude adjustment to energy regulation is simplified, and a rigorous mathematical foundation is provided for subsequent solving of the scaling coefficient based on the target signal-to-noise ratio.

[0032] In some embodiments of the first aspect of the application, the signal amplitude scaling coefficient is solved in a preset signal-to-noise ratio calculation equation by combining the preset target signal-to-noise ratio, the first weighted voice energy and the third weighted voice energy, and the signal-to-noise ratio calculation equation comprises:

[0033] The signal-to-noise ratio calculation equation is as follows: Wherein, SNR represents the signal-to-noise ratio, E speech represents the first weighted voice energy, E add_noise represents the third weighted voice energy.

[0034] In the signal-to-noise ratio calculation equation, the target signal-to-noise ratio is given, and the expression of the third weighted voice energy is substituted to obtain the signal amplitude scaling coefficient, which is as follows:

[0035] Compared with the prior art, the above-mentioned embodiment has the following beneficial effects: the target signal-to-noise ratio is substituted into the weighted energy equation to solve the scaling coefficient, the nonlinear relationship between the signal-to-noise ratio and the energy ratio is converted into a linear equation through logarithmic conversion, the analytical solution of the scaling coefficient is realized, and the noise intensity regulation accuracy is ensured to meet the target signal-to-noise ratio requirement.

[0036] In some embodiments of the first aspect of the application, the to-be-added noise audio is generated by adjusting the original noise audio according to the signal amplitude scaling coefficient, and the noise voice meeting the target signal-to-noise ratio is generated by combining the original voice audio, and the method comprises:

[0037] The signal relationship between the original noise audio and the to-be-added noise is as follows:

[0038] add_noise = m x noise; wherein, add_noise represents the to-be-added noise, m represents the signal amplitude scaling coefficient, and noise represents the original noise audio.

[0039] The expression of the signal amplitude scaling coefficient is substituted to generate the to-be-added noise audio, which is as follows:

[0040]

[0041] Based on the to-be-noised audio, combined with the original human voice audio, a noisy speech conforming to the target signal-to-noise ratio is generated, and is expressed as follows:

[0042] Noisy=speech+add_noise; wherein, noisy represents the noisy speech, and speech represents the original human voice audio.

[0043] Compared with the prior art, the above embodiment has the following beneficial effects: the to-be-noised amplitude is adjusted according to the scaling coefficient and mixed with the pure speech, the process of noise adjustment and speech synthesis is quantified directly through a mathematical formula, and the consistency between the theoretical calculation value of the signal-to-noise ratio of the noisy speech and the actual auditory perception is ensured.

[0044] In some embodiments of the first aspect of the application, the size of the signal window is 10-30 ms.

[0045] Compared with the prior art, the above embodiment has the following beneficial effects: in view of the diversity of different audio signals and the complexity of noise energy distribution, the size of the signal window is limited to 10-30 ms, which can balance the time domain resolution and calculation stability: too small window size will cause violent energy fluctuation, and too large window size will mask the short-time energy change of the non-stationary noise; the perception time scale of the human ear to the non-stationary noise can be effectively adapted.

[0046] Secondly, the application further provides a segmented energy weighted noisy speech generation system, comprising: a data acquisition module, a weight calculation module, a weighted energy calculation module, a third weighted energy calculation module, a solving module and a result output module.

[0047] The data acquisition module is configured to acquire original human voice audio and original noise audio with equal time length.

[0048] The weight calculation module is configured to calculate the energy weight of the audio signal of each signal window based on a preset signal window on the original human voice audio and the original noise audio, respectively.

[0049] The weighted energy calculation module is configured to construct a weighted energy calculation equation according to the energy weight and the sampling point energy of the audio signal in each signal window, and calculate a first weighted speech energy of the original human voice audio and a second weighted speech energy of the original noise audio, respectively.

[0050] The third weighted energy calculation module is configured to construct a third weighted energy calculation equation of the to-be-noised based on the preset signal amplitude scaling coefficient between the original noise audio and the to-be-noised, combined with the weighted energy calculation equation, and calculate a third weighted speech energy of the to-be-noised.

[0051] The solving module is configured to solve the signal amplitude scaling coefficient in a preset signal-to-noise ratio calculation equation in combination with the preset target signal-to-noise ratio, the first weighted voice energy and the third weighted voice energy.

[0052] The result output module is configured to adjust the original noise audio to generate a to-be-noised audio and combine the original human voice audio to generate a noisy voice meeting the target signal-to-noise ratio according to the signal amplitude scaling coefficient.

[0053] Compared with the prior art, the above embodiments have the following beneficial effects: by acquiring original human voice and noise audio of equal time length, the synchronization of subsequent energy calculation and noise processing is ensured, and energy deviation caused by time length difference is avoided; based on a preset signal window, each window energy weight is calculated, and the energy weight is quantified to realize differential weighting of a voice active segment (high energy) and a mute segment (low energy), so that noise energy adjustment is focused on a voice active period sensitive to human ears; a weighted energy equation is constructed to obtain first and second weighted voice energies, and the segmented energy is used to refine time domain energy distribution capture capability, so as to adapt to the rapid energy change characteristics of non-stationary noise (such as burst noise and fluctuating noise); the third weighted energy equation of the to-be-noised noise is derived in combination with the amplitude scaling coefficient, so that noise adjustment is directly associated with the segmented weighted energy, and the adjustment proportion of noise energy in each time window is dynamically matched with voice energy; the scaling coefficient is solved by the target signal-to-noise ratio, and the voice is mixed to finally generate a noisy voice, thereby forming a closed loop of "perception demand-energy calculation-intensity regulation", so that noise energy distribution is accurately matched with the time-varying perception characteristics of human ears to non-stationary noise; the steps are cooperated to form a complete technical chain of time domain alignment-dynamic weighting-resolution improvement-quantitative regulation-closed loop control, so that the distribution rule of noise energy of the generated noisy voice over time is consistent with the perception rule of the human auditory system to non-stationary noise, and the technical contradiction of the traditional method that "the calculated signal-to-noise ratio meets the standard but the hearing sensation is distorted" is solved.

[0054] In a third aspect, the present application further provides a device for generating a noisy voice based on segmented energy weighting, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the computer program, when loaded into the processor, implements the steps of any of the methods for generating a noisy voice based on segmented energy weighting.

[0055] In a fourth aspect, the embodiments of the present application further provide a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program, when executed by a processor, implements the steps of any of the methods for generating a noisy voice based on segmented energy weighting. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1A flowchart of a segment energy weighted noisy speech generation method provided in some embodiments of the present application.

[0057] Figure 2 A structural diagram of a segment energy weighted noisy speech generation system provided in some embodiments of the present application.

[0058] Figure 3 A structural diagram of a segment energy weighted noisy speech generation device provided in some embodiments of the present application.

[0059] Figure 4 A window size and level relationship analysis diagram provided in some embodiments of the present application. DETAILED DESCRIPTION

[0060] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0061] Embodiment one:

[0062] Please refer to Figure 1 To solve the problem that the generated noisy speech in the prior art does not conform to the auditory perception characteristics of human ears to non-steady-state noise, an embodiment of the present application provides a segment energy weighted noisy speech generation method, which comprises the following steps S1 to S6:

[0063] Step S1: obtaining original human voice audio and original noise audio with equal time length.

[0064] In specific implementation, to ensure the effect of the finally generated noisy speech, the original human voice audio and the original noise audio should be as pure human voice audio and noise audio as possible.

[0065] Step S2: based on a preset signal window, calculating the energy weight of the audio signal of each signal window on the original human voice audio and the original noise audio, respectively.

[0066] Further, the step S2 can be implemented through the following preferred embodiment, which comprises the following step S21 and is specifically as follows:

[0067] S21: performing window division on the original human voice audio and the original noise audio in the time domain, respectively, and calculating the energy weight of the corresponding audio signal on each signal window; wherein the calculation formula is as follows:

[0068] wherein, Espeech (k) and E noise (k) and E speech_total and E noise_total and E speech (k) and w noise (k) and w

[0069] In the preferred embodiment, by windowing the audio in the time domain and calculating the energy weight, the weight value is quantified by the window energy proportion, the dynamic focusing of the speech activity segment (high energy window) is realized, the interference of the silent segment (low energy window) on the overall energy calculation is reduced, the time domain pertinence of the energy weight distribution is improved, and accurate weight parameter support is provided for the subsequent weighted energy equation.

[0070] Further, the size of the signal window is 10-30 ms.

[0071] In the preferred embodiment, in view of the diversity of different audio signals and the complexity of noise energy distribution, the size of the signal window is limited to 10-30 ms, which can balance the time domain resolution and calculation stability: too small window size will cause energy fluctuation, and too large window size will mask the short-time energy change of non-stationary noise; it can effectively adapt to the perception time scale of human ear to non-stationary noise.

[0072] Step S3: constructing a weighted energy calculation equation according to the energy weight and sample point energy corresponding to the audio signal in each signal window, and respectively calculating the first weighted speech energy of the original speech audio and the second weighted speech energy of the original noise audio.

[0073] Further, the step S3 can be realized by the following preferred embodiment, including steps S31-S32, as follows:

[0074] S31: constructing a first weighted energy calculation equation based on the energy weight and sample point energy corresponding to each signal window on the original speech audio, and solving the first weighted speech energy; wherein the first weighted energy calculation equation is expressed as follows: wherein E speech represents the first weighted speech energy of the original speech audio, E speech_max_energy (k) represents the sample point energy of the original speech signal in window k at the maximum absolute value (i.e. the square of the amplitude of the point), w speech (k) represents the energy weight of the original speech signal in the kth window;

[0075] S32: Based on the energy weights and sampling point energy corresponding to each signal window, a second weighted energy calculation equation is constructed and solved to obtain the second weighted speech energy on the original noisy audio; wherein, the second weighted energy calculation equation is expressed as follows: Among them, E noise E represents the second-weighted speech energy of the original noisy audio. noise_max_energy (k) represents the energy of the original noise signal in window k at the sampling point where the absolute value of the amplitude is the largest, w noise (k) represents the energy weight of the original noise signal in the k-th window, and N represents the total number of signal windows of the original human voice audio and the total number of signal windows of the original noise audio.

[0076] In this preferred embodiment, a weighted energy calculation equation is constructed based on energy weights and sampling point energy. By summing the product of the maximum energy of the window and the corresponding weight, the proportion of high-energy speech periods in the overall energy calculation is strengthened, the sensitivity of energy representation to temporal changes in speech signals is improved, and the problem of loss of non-steady-state noise energy features caused by traditional full-time-domain average energy calculation is avoided.

[0077] In practice, the original audio is first divided into N short-time analysis windows in the time domain. The corresponding weight is calculated in each window. Then, in order to more accurately describe the salient region of the speech signal, the concept of weighted speech energy is introduced to represent the intensity characteristics of the entire speech signal. The calculation method is as described in step S3 above.

[0078] In order to study the impact of window size on the subsequent calculation results of this application, this application designs a level calculation formula. If the amplitude of the audio sampling points is normalized to [-1.0, 1.0], according to the mathematical characteristics of the energy weight in step S2 and the weighted energy formula proposed in step S3:

[0079]

[0080] From the two formulas above, E speech The maximum value is 1.

[0081] The formula for calculating the level is:

[0082] Where Amplitude represents the amplitude of the speech signal, and Base_Amplitude represents the maximum amplitude of the audio sample (the maximum energy amplitude of the sampled value may be different for different digital sampling precisions; for example, the maximum amplitude of 16-bit sampling is (2^16) / 2. Here, due to normalization, the maximum amplitude is 1).

[0083] E speech The maximum value of 1 is used as Base_Amplitude 2Amplitude 2 Replace with E speech The formula for calculating the voltage level can be obtained as follows:

[0084] Level (dB) = 10 × log 10 E speech ; where E speech The value is affected by the window size (because E speech_max_energy and w speech (Both are affected by window size).

[0085] During the experiment, different types of audio files were selected (various steady-state and non-steady-state noises, including white noise, machine noise, bin Laden's voice, coughing sounds, keyboard sounds, human voices, etc.). Graphics software was used to analyze the relationship between window size and audio level (dB). Figure 4 The diagram illustrates the relationship between window size and audio level. For various sounds, when the window size is greater than approximately 80ms, the audio level changes significantly with the window size. However, if the window size is too small, less than approximately 2ms, a jagged pattern of level variation appears. Overall, the calculated audio level for various sounds is relatively stable within a window size range of 10ms to 50ms.

[0086] In addition, there is a concept called auditory masking, which refers to the temporary inability of the human ear to hear the subsequent weaker sound after a loud sound. This phenomenon includes two situations:

[0087] (1) Forward masking: The stronger sound will mask the weaker sound within about 50ms to 200ms after it ends.

[0088] (2) Backward masking: A stronger sound will mask a weaker sound about 10 to 20 ms before it appears.

[0089] Based on the above experimental analysis and the auditory masking phenomenon, and considering the computational load, a window size of 10ms to 30ms can be selected as an optimal value, such as 20ms.

[0090] Step S4: Based on the preset signal amplitude scaling factor between the original noise audio and the noise to be added, and combined with the weighted energy calculation equation, construct the third weighted energy calculation equation for the noise to be added, and calculate the third weighted speech energy of the noise to be added.

[0091] Furthermore, step S4 can be implemented through the following preferred embodiment, including step S41, as follows:

[0092] S41: The numerical relationship between the energy of the sampling point where the absolute value of the amplitude of the original noise signal is the maximum in the signal window and the maximum signal amplitude is expressed as follows:

[0093] E noise_max_energy (k) = noise_max_amp(k) 2 Where noise_max_amp(k) represents

[0094] The maximum signal amplitude of the original noise signal within the signal window k;

[0095] The relationship between the maximum signal amplitude of the original noise audio and the noise to be added is expressed as follows: add_noise_max_amp(k) = m × noise_max_amp(k); where m represents the signal amplitude scaling factor between the original noise audio and the noise to be added, and add_noise_max_amp(k) represents the maximum signal amplitude of the noise to be added within the signal window k.

[0096] The energy weighting between the original noise audio and the noise to be added is represented as follows:

[0097] w add_noise (k)=w noise (k); where w add_noise (k) represents the energy weight of the noise signal to be added in the k-th window;

[0098] Combining the above three equations, the third weighted energy calculation equation for the noise to be added is constructed and solved to obtain the third weighted speech energy, as shown below:

[0099]

[0100] Among them, E add_noise This represents the third weighted speech energy of the noise to be added.

[0101] In this preferred embodiment, by establishing the amplitude scaling relationship between the original noise and the noise to be added, the square term of the amplitude scaling factor is directly introduced into the noise energy calculation. This achieves the physical dimension conversion from amplitude to energy without introducing additional parameters, ensuring the mathematical correlation between the noise energy adjustment process and the amplitude scaling factor. At the same time, it maintains the consistency of the weighting parameters in the noise energy calculation, simplifies the mapping logic from amplitude adjustment to energy control, and provides a rigorous mathematical basis for the subsequent solution of the scaling factor based on the target signal-to-noise ratio.

[0102] Step S5: Combine the preset target signal-to-noise ratio, the first weighted speech energy, and the third weighted speech energy to solve for the signal amplitude scaling factor in the preset signal-to-noise ratio calculation equation.

[0103] Furthermore, step S5 can be implemented through the following preferred embodiment, including step S51, as follows:

[0104] S51: The signal-to-noise ratio calculation equation is expressed as follows: Where SNR represents the signal-to-noise ratio, E speech E represents the first weighted speech energy. add_noise This represents the third-weighted speech energy;

[0105] In the signal-to-noise ratio calculation equation, given the target signal-to-noise ratio, and substituting it into the expression for the third weighted speech energy, the signal amplitude scaling factor is obtained, as follows:

[0106] In this preferred embodiment, the target signal-to-noise ratio is substituted into the weighted energy equation to solve for the scaling factor. The nonlinear relationship between the signal-to-noise ratio and the energy ratio is transformed into a linear equation through logarithmic transformation, thereby realizing the analytical solution of the scaling factor and ensuring that the noise intensity control accuracy meets the target signal-to-noise ratio requirements.

[0107] Furthermore, compared to other signal-to-noise ratio (SNR) calculation methods, this application's weighted speech energy substitution method is adaptable to both steady-state and non-steady-state noise, is more sensitive to the sound-generating region, and has stronger suppression of silent regions. Specifically, for steady-state noise, since its energy is relatively stable in the time dimension, the SNR after segmented weighting is basically consistent with the traditional average SNR result. This application does not introduce additional bias under steady-state noise conditions and can accurately reflect the noise level. For non-steady-state noise, since this method pays more attention to the speech activity segment during the weighting process, the SNR calculation result can highlight the degree of noise influence during speech, enhance the adaptability to non-stationary background noise, and demonstrate good versatility and practicality. Moreover, the calculation process is simple and efficient, making it easy to apply in practice. Furthermore, based on this SNR calculation method, the synthesis formula for the audio to be noised is derived, which can generate noisy speech data that better conforms to the characteristics of human auditory perception, providing a more realistic and effective simulation environment for training and evaluating speech denoising models. The specific method is as follows:

[0108] Step S6: Adjust the original noise audio according to the signal amplitude scaling factor to generate noise audio to be added, and combine it with the original human voice audio to generate noisy speech that meets the target signal-to-noise ratio.

[0109] Furthermore, step S6 can be implemented through the following preferred embodiments, including steps S61-S62, as follows:

[0110] S61: The signal relationship between the original noise audio and the noise to be added is expressed as follows:

[0111] add_noise = m × noise; where add_noise represents the noise to be added, m represents the signal amplitude scaling factor, and noise represents the original noise audio;

[0112] Substituting the expression for the signal amplitude scaling factor, the generated audio with added noise is shown below:

[0113]

[0114] S62: Based on the audio to be noised and combined with the original human voice audio, generate noisy speech that meets the target signal-to-noise ratio, as follows:

[0115] noisy = speech + add_noise; where noisy represents noisy speech and speech represents the original human voice audio.

[0116] In this preferred embodiment, the amplitude of the noise to be added is adjusted according to the scaling factor and mixed with the clean speech. The noise adjustment and speech synthesis process is directly quantified by mathematical formulas to ensure the consistency between the theoretically calculated value of the signal-to-noise ratio of the noisy speech and the actual auditory perception.

[0117] In summary, compared with the prior art, the above embodiments of this application have the following beneficial effects: by acquiring original human voice and noise audio of equal duration, the synchronicity of subsequent energy calculation and noise addition processing is ensured, avoiding energy deviation caused by duration differences; based on a preset signal window, the energy weight of each window is calculated, and the energy proportion quantification is used to achieve differentiated weighting of speech activity segments (high energy) and silent segments (low energy), so that noise energy adjustment focuses on the speech activity periods that are sensitive to the human ear; a weighted energy equation is constructed to obtain the first and second weighted speech energy, and the ability to capture temporal energy distribution is refined through segmented energy characterization, adapting to the rapid energy change characteristics of non-steady-state noise (such as sudden and fluctuating noise); the first and second weighted speech energy are derived by combining the amplitude scaling coefficient. The three-weighted energy equation directly links noise adjustment with piecewise weighted energy, ensuring that the adjustment ratio of noise energy in each time window dynamically matches the speech energy. By solving for the scaling factor through the target signal-to-noise ratio and mixing the speech, noisy speech is finally generated, forming a closed loop of "perceptual needs - energy calculation - intensity control". This makes the noise energy distribution accurately match the time-varying perception characteristics of the human ear to non-steady-state noise. The synergistic effect of each step forms a complete technical chain of time-domain alignment → dynamic weighting → resolution improvement → quantization control → closed-loop control. Ultimately, the distribution law of noise energy in the generated noisy speech over time is consistent with the perception law of the human auditory system to non-steady-state noise, solving the technical contradiction of "achieving the target signal-to-noise ratio but hearing distortion" in traditional methods.

[0118] Example 2:

[0119] Please refer to Figure 2Based on the same inventive concept, the present invention discloses a segmented energy weighted noisy speech generation system, comprising: a data acquisition module M1, a weight calculation module M2, a weighted energy calculation module M3, a third weighted energy calculation module M4, a solution module M5, and a result output module M6.

[0120] The data acquisition module M1 is used to acquire original human voice audio and original noise audio of equal duration.

[0121] The weight calculation module M2 is used to calculate the energy weight of the audio signal of each signal window based on the preset signal window, respectively, on the original human voice audio and the original noise audio.

[0122] Furthermore, the weight calculation module M2 includes: a partitioning calculation unit;

[0123] The partitioning calculation unit is used to partition the original human voice audio and the original noise audio into windows in the time domain, and calculate the energy weight of the corresponding audio signal in each signal window; the calculation formula is as follows:

[0124] Among them, E speech (k) and E noise (k) represents the energy of the original human voice signal and the original noise signal in the k-th window, respectively, E speech_total and E noise_total w represents the total energy of all windows of the original human voice audio and the original noise audio, respectively. speech (k) and w noise (k) represents the energy weights of the original human voice signal and the original noise signal in the k-th window, respectively.

[0125] In this preferred embodiment, by dividing the audio into windows in the time domain and calculating energy weights, the weight values ​​are quantified using the window energy ratio, thereby achieving dynamic focusing on the speech activity segment (high-energy window), reducing the interference of the silent segment (low-energy window) on the overall energy calculation, improving the time-domain specificity of energy weight allocation, and providing accurate weight parameter support for the subsequent weighted energy equation.

[0126] Furthermore, the size of the signal window is 10–30 ms.

[0127] In this preferred embodiment, considering the diversity of different audio signals and the complexity of noise energy distribution, the signal window size is limited to 10-30ms. This range can balance time domain resolution and computational stability: too small a window will cause drastic energy fluctuations, while too large a window will mask the short-term energy changes of non-steady-state noise; it can effectively adapt to the human ear's perception time scale of non-steady-state noise.

[0128] The weighted energy calculation module M3 is used to construct a weighted energy calculation equation based on the energy weights and sampling point energy of the audio signals in each signal window, and to calculate the first weighted speech energy of the original human voice audio and the second weighted speech energy of the original noise audio respectively.

[0129] Furthermore, the weighted energy calculation module M3 includes: a first weighted energy calculation unit and a second weighted energy calculation unit;

[0130] The first weighted energy calculation unit is used to construct a first weighted energy calculation equation and solve it to obtain the first weighted speech energy based on the energy weights and sampling point energy corresponding to each signal window on the original human voice audio. The first weighted energy calculation equation is expressed as follows:

[0131] Among them, E speech E represents the first weighted speech energy of the original human voice audio. speech_max_energy (k) represents the energy of the original human voice signal in window k at the sampling point where the absolute value of the amplitude is the maximum, w speech (k) represents the energy weight of the original human voice signal in the k-th window;

[0132] The second weighted energy calculation unit is used to construct a second weighted energy calculation equation and solve it to obtain the second weighted speech energy based on the energy weights corresponding to each signal window and the energy of the sampling points on the original noisy audio; wherein, the second weighted energy calculation equation is expressed as follows:

[0133] Among them, E noise E represents the second-weighted speech energy of the original noisy audio. noise_max_energy (k) represents the energy of the original noise signal in window k at the sampling point where the absolute value of the amplitude is the largest, w noise (k) represents the energy weight of the original noise signal in the k-th window, and N represents the total number of signal windows of the original human voice audio and the total number of signal windows of the original noise audio.

[0134] In this preferred embodiment, a weighted energy calculation equation is constructed based on energy weights and sampling point energy. By summing the product of the maximum energy of the window and the corresponding weight, the proportion of high-energy speech periods in the overall energy calculation is strengthened, the sensitivity of energy representation to temporal changes in speech signals is improved, and the problem of loss of non-steady-state noise energy features caused by traditional full-time-domain average energy calculation is avoided.

[0135] The third weighted energy calculation module M4 is used to construct the third weighted energy calculation equation of the noise to be added based on the preset signal amplitude scaling coefficient between the original noise audio and the noise to be added, combined with the weighted energy calculation equation, and to calculate the third weighted speech energy of the noise to be added.

[0136] Furthermore, the third weighted energy calculation module M4 includes: a joint solution unit;

[0137] The numerical relationship between the energy of the sampling point with the maximum absolute value of the amplitude of the original noise signal within the signal window and the maximum signal amplitude is expressed as follows:

[0138] E noise_max_energy (k) = noise_max_amp(k) 2 Where noise_max_amp(k) represents

[0139] The maximum signal amplitude of the original noise signal within the signal window k;

[0140] The relationship between the maximum signal amplitude of the original noise audio and the noise to be added is expressed as follows: add_noise_max_amp(k) = m × noise_max_amp(k); where m represents the signal amplitude scaling factor between the original noise audio and the noise to be added, and add_noise_max_amp(k) represents the maximum signal amplitude of the noise to be added within the signal window k.

[0141] The energy weighting between the original noise audio and the noise to be added is represented as follows:

[0142] w add_noise (k)=w noise (k); where w add_noise (k) represents the energy weight of the noise signal to be added in the k-th window;

[0143] The joint solving unit is used to combine the above three equations to construct the third weighted energy calculation equation for the noise to be added and solve for the third weighted speech energy, as shown below:

[0144]

[0145] Among them, E add_noise This represents the third weighted speech energy to which noise is to be added.

[0146] In this preferred embodiment, by establishing the amplitude scaling relationship between the original noise and the noise to be added, the square term of the amplitude scaling factor is directly introduced into the noise energy calculation. This achieves the physical dimension conversion from amplitude to energy without introducing additional parameters, ensuring the mathematical correlation between the noise energy adjustment process and the amplitude scaling factor. At the same time, it maintains the consistency of the weighting parameters in the noise energy calculation, simplifies the mapping logic from amplitude adjustment to energy control, and provides a rigorous mathematical basis for the subsequent solution of the scaling factor based on the target signal-to-noise ratio.

[0147] The solution module M5 is used to solve the signal amplitude scaling factor in a preset signal-to-noise ratio calculation equation by combining the preset target signal-to-noise ratio, the first weighted speech energy, and the third weighted speech energy.

[0148] Furthermore, the solution module M5 includes: a coefficient solution unit;

[0149] The signal-to-noise ratio calculation equation is expressed as follows: Where SNR represents the signal-to-noise ratio, E speech E represents the first weighted speech energy. add_noise This represents the third-weighted speech energy;

[0150] The coefficient solving unit is used to solve for the signal amplitude scaling coefficient in the signal-to-noise ratio calculation equation, given the target signal-to-noise ratio and substituting it into the expression for the third weighted speech energy, as follows:

[0151]

[0152] In this preferred embodiment, the target signal-to-noise ratio is substituted into the weighted energy equation to solve for the scaling factor. The nonlinear relationship between the signal-to-noise ratio and the energy ratio is transformed into a linear equation through logarithmic transformation, thereby realizing the analytical solution of the scaling factor and ensuring that the noise intensity control accuracy meets the target signal-to-noise ratio requirements.

[0153] The result output module M6 is used to adjust the original noise audio according to the signal amplitude scaling factor, generate noise audio to be added, and combine it with the original human voice audio to generate noisy speech that meets the target signal-to-noise ratio.

[0154] Furthermore, the result output module M6 includes: a substitution solution unit and a combination generation unit;

[0155] The signal relationship between the original noise audio and the noise to be added is expressed as follows:

[0156] add_noise = m × noise; where add_noise represents the noise to be added, m represents the signal amplitude scaling factor, and noise represents the original noise audio;

[0157] The substitution and solving unit is used to substitute the expression for the signal amplitude scaling factor to generate the audio to be noised, as shown below:

[0158]

[0159] The combined generation unit is used to generate noisy speech that meets the target signal-to-noise ratio based on the audio to be noised and the original human voice audio, as shown below:

[0160] noisy = speech + add_noise; where noisy represents noisy speech and speech represents the original human voice audio.

[0161] In this preferred embodiment, the amplitude of the noise to be added is adjusted according to the scaling factor and mixed with the clean speech. The noise adjustment and speech synthesis process is directly quantified by mathematical formulas to ensure the consistency between the theoretically calculated value of the signal-to-noise ratio of the noisy speech and the actual auditory perception.

[0162] In summary, compared with the prior art, the embodiments of this application have the following beneficial effects: by acquiring original human voice and noise audio of equal duration, the synchronicity of subsequent energy calculation and noise addition processing is ensured, avoiding energy deviation caused by duration differences; based on a preset signal window, the energy weight of each window is calculated, and the energy proportion quantification is used to achieve differentiated weighting of speech activity segments (high energy) and silent segments (low energy), so that noise energy adjustment focuses on the speech activity periods that are sensitive to the human ear; a weighted energy equation is constructed to obtain the first and second weighted speech energy, and the ability to capture temporal energy distribution is refined through segmented energy characterization, adapting to the rapid energy change characteristics of non-steady-state noise (such as sudden and fluctuating noise); the third weighted energy of the noise to be added is derived by combining the amplitude scaling coefficient. The weighted energy equation directly links noise adjustment with piecewise weighted energy, ensuring that the adjustment ratio of noise energy in each time window dynamically matches the speech energy. By solving for the scaling factor through the target signal-to-noise ratio and mixing the speech, noisy speech is finally generated, forming a closed loop of "perceptual needs - energy calculation - intensity control". This makes the noise energy distribution accurately match the time-varying perception characteristics of the human ear to non-steady-state noise. The synergistic effect of each step forms a complete technical chain of time-domain alignment → dynamic weighting → resolution improvement → quantization control → closed-loop control. Ultimately, the distribution law of noise energy in the generated noisy speech over time is consistent with the perception law of the human auditory system to non-steady-state noise, solving the technical contradiction of "achieving the target signal-to-noise ratio but hearing distortion" in traditional methods.

[0163] Example 3:

[0164] Figure 3 A structural diagram of a segmented energy-weighted noisy speech generation device according to this application is presented. Figure 3As shown, the segmented energy-weighted noisy speech generation device may include: a processor N1, a memory N2, a data interface N3, and a communication bus N4.

[0165] Wherein: processor N1, memory N2, and data interface N3 communicate with each other through communication bus N4; data interface N3 is used for data communication with other devices such as input devices or output devices; processor N1 is used to execute program N5, specifically it can execute the relevant steps in any of the above-mentioned segmented energy weighted noisy speech generation method embodiments.

[0166] Specifically, program N5 may include program code, which includes computer-executable instructions.

[0167] The processor N1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The segmented energy-weighted noisy speech generation device includes one or more processors, which may be processors of the same type, such as one or more CPUs, or processors of different types, such as one or more CPUs and one or more ASICs.

[0168] Memory N2 is used to store program N5. Memory N2 may include high-speed RAM memory and may also include non-volatile memory, such as at least one disk storage device.

[0169] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Furthermore, the embodiments in this application are not directed to any particular programming language.

[0170] Example 4:

[0171] This invention also provides a computer-readable storage medium storing at least one executable instruction that, when executed on a segmented energy-weighted noisy speech generation device / system, causes the segmented energy-weighted noisy speech generation device / system to perform one of the segmented energy-weighted noisy speech generation methods described in any of the above method embodiments.

[0172] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. Similarly, for the purpose of simplification and aiding understanding of one or more aspects of the invention, in the above description of exemplary embodiments of this application, various features of the embodiments are sometimes grouped together in a single embodiment, figure, or description thereof. The claims, which follow the detailed description, are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0173] Those skilled in the art will understand that the modules in the device of the embodiment can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiment can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components, except that at least some of such features and / or processes or units are mutually exclusive.

Claims

1. A segmented energy-weighted method for generating noisy speech, characterized in that, include: Obtain original human voice audio and original noise audio of equal duration; Based on a preset signal window, the energy weight of the audio signal in each signal window is calculated on the original human voice audio and the original noise audio respectively. Based on the energy weights and sampling point energy of the audio signals in each signal window, a weighted energy calculation equation is constructed, and the first weighted speech energy of the original human voice audio and the second weighted speech energy of the original noise audio are calculated respectively. Based on the preset signal amplitude scaling factor between the original noise audio and the noise to be added, and combined with the weighted energy calculation equation, a third weighted energy calculation equation for the noise to be added is constructed, and the third weighted speech energy of the noise to be added is calculated. The signal amplitude scaling factor is solved in the preset signal-to-noise ratio calculation equation by combining the preset target signal-to-noise ratio, the first weighted speech energy, and the third weighted speech energy. Based on the signal amplitude scaling factor, the original noise audio is adjusted to generate noise audio to be added, and combined with the original human voice audio to generate noisy speech that meets the target signal-to-noise ratio.

2. The segmented energy-weighted noisy speech generation method as described in claim 1, characterized in that, The method, based on a preset signal window, calculates the energy weight of the audio signal for each signal window on both the original human voice audio and the original noise audio, including: In the time domain, the original human voice audio and the original noise audio are divided into windows, and the energy weight of the corresponding audio signal is calculated for each signal window; the calculation formula is as follows: Among them, E speech (k) and E noise (k) represents the energy of the original human voice signal and the original noise signal in the k-th window, respectively, E speech_total and E noise_total w represents the total energy of all windows of the original human voice audio and the original noise audio, respectively. speech (k) and w noise (k) represents the energy weights of the original human voice signal and the original noise signal in the k-th window, respectively.

3. The segmented energy-weighted noisy speech generation method as described in claim 2, characterized in that, The step involves constructing a weighted energy calculation equation based on the energy weights and sampling point energy of the audio signal in each signal window, and calculating the first weighted speech energy of the original human voice audio and the second weighted speech energy of the original noise audio, including: Based on the energy weights and sampling point energy corresponding to each signal window, a first weighted energy calculation equation is constructed and solved to obtain the first weighted speech energy. The first weighted energy calculation equation is expressed as follows: Among them, E speech E represents the first weighted speech energy of the original human voice audio. speech_max_energy (k) represents the energy of the original human voice signal in window k at the sampling point where the absolute value of the amplitude is the maximum, w speech (k) represents the energy weight of the original human voice signal in the k-th window; Based on the energy weights and sampling point energy corresponding to each signal window, a second weighted energy calculation equation is constructed and solved to obtain the second weighted speech energy. The second weighted energy calculation equation is expressed as follows: Among them, E noise E represents the second-weighted speech energy of the original noisy audio. noise_max_energy (k) represents the energy of the original noise signal in window k at the sampling point where the absolute value of the amplitude is the largest, w noise (k) represents the energy weight of the original noise signal in the k-th window, and N represents the total number of signal windows of the original human voice audio and the total number of signal windows of the original noise audio.

4. The segmented energy-weighted noisy speech generation method as described in claim 3, characterized in that, The third weighted energy calculation equation for the noise to be added is constructed based on the preset signal amplitude scaling factor between the original noise audio and the noise to be added, combined with the weighted energy calculation equation, and the third weighted speech energy of the noise to be added is calculated, including: The numerical relationship between the energy of the sampling point with the maximum absolute value of the amplitude of the original noise signal within the signal window and the maximum signal amplitude is expressed as follows: E noise_max_energy (k) = noise_max_amp(k) 2 Where noise_max_amp(k) represents The maximum signal amplitude of the original noise signal within the signal window k; The relationship between the maximum signal amplitude of the original noise audio and the noise to be added is expressed as follows: add_noise_max_amp(k) = m × noise_max_amp(k); where m represents the signal amplitude scaling factor between the original noise audio and the noise to be added, and add_noise_max_amp(k) represents the maximum signal amplitude of the noise to be added within the signal window k. The energy weighting between the original noise audio and the noise to be added is represented as follows: w add_noise (k)=w noise (k); where w add_noise (k) represents the energy weight of the noise signal to be added in the k-th window; Combining the above three equations, the third weighted energy calculation equation for the noise to be added is constructed and solved to obtain the third weighted speech energy, as shown below: Among them, E add_noise This represents the third weighted speech energy to which noise is to be added.

5. The segmented energy-weighted noisy speech generation method as described in claim 4, characterized in that, The step of combining a preset target signal-to-noise ratio, the first weighted speech energy, and the third weighted speech energy to solve for the signal amplitude scaling factor in a preset signal-to-noise ratio calculation equation includes: The signal-to-noise ratio calculation equation is expressed as follows: Where SNR represents the signal-to-noise ratio, E speech E represents the first weighted speech energy. add_noise This represents the third-weighted speech energy; In the signal-to-noise ratio calculation equation, given the target signal-to-noise ratio, and substituting it into the expression for the third weighted speech energy, the signal amplitude scaling factor is obtained, as follows:

6. The segmented energy-weighted noisy speech generation method as described in claim 5, characterized in that, The step of adjusting the original noise audio according to the signal amplitude scaling factor to generate noise audio to be added, and combining it with the original human voice audio to generate noisy speech that meets the target signal-to-noise ratio, includes: The signal relationship between the original noise audio and the noise to be added is expressed as follows: add_noise = m × noise; where add_noise represents the noise to be added, m represents the signal amplitude scaling factor, and noise represents the original noise audio; Substituting the expression for the signal amplitude scaling factor, the generated audio with added noise is shown below: Based on the audio to be noised, and combined with the original human voice audio, noisy speech that meets the target signal-to-noise ratio is generated, as follows: noisy = speech + add_noise; where noisy represents noisy speech and speech represents the original human voice audio.

7. A segmented energy-weighted noisy speech generation method as described in any one of claims 1-6, characterized in that, The size of the signal window is 10–30 ms.

8. A segmented energy-weighted noisy speech generation system, characterized in that, include: The system includes a data acquisition module, a weight calculation module, a weighted energy calculation module, a third weighted energy calculation module, a solution module, and a result output module. The data acquisition module is used to acquire original human voice audio and original noise audio of equal duration. The weight calculation module is used to calculate the energy weight of the audio signal of each signal window based on the preset signal window, respectively, on the original human voice audio and the original noise audio. The weighted energy calculation module is used to construct a weighted energy calculation equation based on the energy weight and sampling point energy of the audio signal in each signal window, and to calculate the first weighted speech energy of the original human voice audio and the second weighted speech energy of the original noise audio respectively. The third weighted energy calculation module is used to construct a third weighted energy calculation equation for the noise to be added based on a preset signal amplitude scaling factor between the original noise audio and the noise to be added, combined with the weighted energy calculation equation, and to calculate the third weighted speech energy of the noise to be added. The solution module is used to solve the signal amplitude scaling factor in a preset signal-to-noise ratio calculation equation by combining the preset target signal-to-noise ratio, the first weighted speech energy and the third weighted speech energy. The result output module is used to adjust the original noise audio according to the signal amplitude scaling factor, generate noise audio to be added, and combine it with the original human voice audio to generate noisy speech that meets the target signal-to-noise ratio.

9. A segmented energy-weighted noisy speech generation device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is loaded into the processor, it implements the steps of a segmented energy-weighted noisy speech generation method according to any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of a segmented energy-weighted noisy speech generation method according to any one of claims 1-7.

Images