A method for sound cloning in translation headphones

Abstract

This invention discloses a voice cloning method for translation headsets, relating to the field of speech signal processing technology. The method includes the following steps: In the initialization phase before the translation headset is activated, multiple segments of speech data from the wearer in a natural context are collected using a high-fidelity microphone to comprehensively extract their personalized voiceprint features. This invention effectively avoids the damage to the speech spectrum structure and voiceprint features caused by high-frequency interference by introducing a mechanism for identifying and dynamically suppressing implicit high-frequency noise. The solution combines feature quantization analysis, deep model evaluation, and multi-dimensional spectrum suppression techniques to construct a closed-loop control process of identification-judgment-response, significantly improving the modeling accuracy of the voice cloning model, reducing the risk of harsh feedback and distortion in speech synthesis, and enhancing the naturalness and consistency of the translation headset's speech output.

Description

Technical Field

[0001] This invention relates to the field of speech signal processing technology, and more specifically to a method for sound cloning for translation headphones. Background Technology

[0002] "Voice cloning for translation headphones" refers to the technical process of collecting the wearer's original voice features during the voice translation task, constructing a personalized voice model using artificial intelligence modeling technology, and then synthesizing the translation result in the wearer's original voice style after semantic translation is completed. This method not only preserves the accuracy of the translated content but also achieves a high degree of restoration of personalized features such as voice emotion, intonation, and rhythm, making the voice output by the translation headphones sound more natural and more individually recognizable. This enhances the immersiveness and approachability of multilingual communication, and is especially suitable for cross-language communication scenarios that require maintaining consistent voice identity, such as international conferences, business negotiations, or educational lectures.

[0003] Existing technologies have the following shortcomings: In the speech modeling stage of applying sound cloning technology to translation headphones, it is necessary to collect the user's original speech samples through a high-sensitivity microphone to extract their personalized speech features. However, in specific application scenarios such as smart office environments or medical facilities, there are often several high-frequency excitation signal sources that are beyond the range of human hearing perception, such as pulse modulation noise generated by LED lighting driver power supplies, high-frequency harmonic interference from air purification equipment during motor operation, and feedback howling generated by hearing aids under the formation of audio loops. Although these signals are difficult to detect subjectively, they are easily detected by high-sensitivity microphones and mixed into the speech samples during actual speech acquisition, becoming "hidden high-frequency noise pollution sources."

[0004] The intrusion of such high-frequency interference signals will directly disrupt the spectral structure of speech data, causing the neural network to mistakenly learn abnormal frequency components unrelated to the user's actual pronunciation when modeling user speech features. This leads to problems such as abrupt changes in tone, harsh whistling, or sound quality distortion in the subsequent speech synthesis process, seriously affecting the naturalness and intelligibility of the translation headset in the semantic output stage, reducing the user's communication experience and system reliability.

[0005] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this invention is to provide a sound cloning method for translation headphones. By introducing a mechanism for identifying and dynamically suppressing implicit high-frequency noise, it effectively avoids the damage of high-frequency interference to the speech spectrum structure and voiceprint features. The solution combines feature quantization analysis, deep model evaluation, and multi-dimensional spectrum suppression techniques to construct a closed-loop control process of identification-judgment-response. This significantly improves the modeling accuracy of the sound cloning model, reduces the risk of harsh feedback and distortion in speech synthesis, and enhances the naturalness and consistency of the speech output from the translation headphones, thereby solving the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a sound cloning method for translation headphones, comprising the following steps:

[0008] During the initialization phase before the translation headset is activated, multiple segments of voice data of the wearer in a natural context are collected through a high-fidelity microphone to comprehensively extract their personalized voiceprint features.

[0009] The collected speech data is preprocessed, and the core indicators that characterize the presence of hidden high-frequency pollution in the speech data are extracted from the preprocessed data through feature engineering methods. The extracted core indicators are comprehensively analyzed to quantify the degree of pollution in the current speech data.

[0010] The core indicators after comprehensive analysis are constructed into feature vectors and input into a pre-trained deep learning model. The model then performs intelligent real-time evaluation of the current data to determine whether there is any hidden high-frequency pollution in the speech samples.

[0011] Once implicit high-frequency noise pollution is identified in the collected speech data, a high-frequency dynamic suppression mechanism is activated to adaptively adjust the spectral boundaries of the high-frequency part of the speech signal to limit the frequency range participating in subsequent feature modeling. Subsequently, a nonlinear spectral compression method is applied to suppress the amplitude of the identified abnormal frequency components, and an adaptive window filter with corresponding parameters is constructed based on the dynamic positioning results of the polluted frequency band to perform targeted filtering of the identified high-frequency pollution region and suppress the influence of abnormal frequency components.

[0012] Preferably, during the initialization phase before the translation headset is activated, the specific steps for personalized voiceprint acquisition are as follows:

[0013] Wearers complete multiple rounds of voice-guided recording in natural contexts, including daily greetings, emotional phrases, and commonly used conversational phrases, ensuring the diversity of language data and the coverage of language contexts;

[0014] Each segment of speech is captured in real time using a high-fidelity microphone, fully recording the time and frequency domain signal characteristics of the speech.

[0015] Sentence-level segmentation and metadata tagging are performed on the collected speech samples;

[0016] All tagged speech samples are aggregated and stored in a local cache or cloud database.

[0017] Preferably, feature engineering methods are used to extract core indicators characterizing the presence of latent high-frequency pollution in speech data from the preprocessed data. The extracted indicators include the degree of deviation of the high-frequency spectral envelope shape from the normal template and the synchronicity between the speech intonation trend and high-frequency energy changes. The degree of deviation of the high-frequency spectral envelope shape from the normal template and the synchronicity between the speech intonation trend and high-frequency energy changes are comprehensively analyzed under the detection window to generate high-frequency distortion index and intonation-high-frequency mismatch index respectively. The degree of pollution of the current speech data is quantified by the high-frequency distortion index and the intonation-high-frequency mismatch index.

[0018] Preferably, the specific steps for generating a high-frequency distortion index by comprehensively analyzing the deviation of the high-frequency spectrum envelope shape from the normal template within the detection window are as follows:

[0019] After the speech signal undergoes short-time Fourier transform processing, the high-frequency spectrum corresponding to the detection window is extracted from the spectrogram. The spectral envelope curve of each frame within the frequency band is sampled and normalized to a unit interval [0, 1]. Simultaneously, the built-in standard speech high-frequency template envelope is invoked, and the normalized structure deviation is calculated for each frequency point. The calculation expression is as follows:

[0020]

[0021] In the formula, S(f i S(f) is the spectral amplitude of the actual speech sample at frequency point i, and max(S(f)) is the maximum spectral amplitude of the actual speech sample among all high-frequency points within the current detection window. ref (f i ) is the spectral amplitude value of the standard speech template at frequency point i, max(S ref (f) is the maximum amplitude value of the reference template in the high-frequency band, Δ i It is the deviation of the normalized structure of the actual speech sample at frequency point i from the spectrum of the reference template;

[0022] After obtaining the envelope deviation at each frequency point, a nonlinear enhancement mechanism is introduced to enhance the response sensitivity to severe high-frequency anomalies. The differences of all frequency points are superimposed through a nonlinear mapping function to generate a high-frequency distortion index, the calculation expression of which is as follows:

[0023]

[0024] In the formula, N is the total number of sampling frequency points included in the high-frequency band, and e is the natural base. It is the deviation enhancement coefficient, and HFSWI is the high spectral distortion index.

[0025] Preferably, the specific steps for generating a pitch-high frequency mismatch index by comprehensively analyzing the synchronicity between the intonation trend and high-frequency energy changes within a detection window are as follows:

[0026] Within the detection window, the fundamental frequency sequence F0(j) of the speech signal is first extracted, and a normalized dynamic curve of intonation change is constructed using a weighted rate of change to highlight the subtle trends in intonation. Simultaneously, the total energy E of the target high-frequency band in each frame is extracted from the spectrogram. HF (j) Construct a normalized high-frequency energy variation curve and perform amplitude rounding to align the scale, as shown in the following formula:

[0027]

[0028] In the formula, ΔF0(j) is the rate of change of the fundamental frequency of the speech signal between the j-th frame and the previous frame, calculated as follows: ΔF0(j)=F0(j)-F0(j-1), where F0(j) is the fundamental frequency value of the speech signal in the current j-th frame, F0(j-1) is the fundamental frequency value of the speech signal in the (j-1)-th frame (i.e., the fundamental frequency of the previous frame), λ1 is the pitch change response coefficient, P(j) is the pitch change dynamic curve, and E HF (j) is the total energy value of the high-frequency band of the speech signal in the j-th frame, and λ2 is the high-frequency energy response coefficient, ln(1+E HF H(j) is a logarithmic compression function, and H(j) is a high-frequency energy variation curve, representing the high-frequency energy normalization result in the j-th frame.

[0029] Based on the acquired dynamic curve of intonation change P(j) and high-frequency energy change curve H(j), the intonation-high frequency mismatch index is calculated by nonlinearly accumulating the point-to-point differences between the two within the detection window. The calculation expression is as follows:

[0030]

[0031] In the formula, PHFMI is the tone-high frequency mismatch index, M is the total number of frames in the detection window, δ is the asynchronous mutation amplification coefficient, γ is the deviation amplitude enhancement index, and e is the natural base.

[0032] Preferably, the high-frequency distortion index and the tone-high frequency mismatch index after comprehensive analysis are constructed into feature vectors and input into a pre-trained deep learning model. The model generates a latent high-frequency pollution risk coefficient, and the current data is intelligently evaluated in real time based on the latent high-frequency pollution risk coefficient to determine whether there is latent high-frequency pollution in the speech sample.

[0033] Preferably, the implicit high-frequency pollution risk coefficient generated during the intelligent real-time evaluation of the current data by a pre-trained deep learning model is compared with a pre-set reference threshold of the implicit high-frequency pollution risk coefficient to determine whether there is implicit high-frequency pollution in the speech sample. The judgment logic is as follows:

[0034] If the implicit high-frequency pollution risk coefficient is greater than the pre-set reference threshold of the implicit high-frequency pollution risk coefficient, it is determined that there is implicit high-frequency pollution in the speech sample; if the implicit high-frequency pollution risk coefficient is less than or equal to the pre-set reference threshold of the implicit high-frequency pollution risk coefficient, it is determined that there is no implicit high-frequency pollution in the speech sample.

[0035] Preferably, when implicit high-frequency noise pollution is identified in the collected speech data, a high-frequency band dynamic suppression mechanism is activated to perform adaptive adjustment of the spectral boundary on the high-frequency part of the speech signal; subsequently, a non-linear spectral compression method is applied to suppress the amplitude of the identified abnormal frequency components, and an adaptive window filter with corresponding parameters is constructed based on the dynamic positioning result of the polluted frequency band to perform directional filtering on the identified high-frequency pollution area. The specific steps are as follows:

[0036] When implicit high-frequency noise pollution is identified in the collected speech data, a high-frequency band dynamic suppression mechanism is activated to perform adaptive adjustment of the spectral boundary on the high-frequency part of the speech signal, and the highest frequency upper limit for subsequent modeling of the current speech sample is dynamically determined. The calculation formula is as follows:

[0037]

[0038] , where f max is the modeling frequency upper limit, f0 is the default highest frequency, θ1 is the frequency boundary contraction coefficient, R p is the implicit high-frequency pollution risk coefficient, R th is the reference threshold of the implicit high-frequency pollution risk coefficient;

[0039] After determining the modeling frequency upper limit f max , for all spectral components within the frequency range f max < f ≤ f0, a risk coefficient-driven non-linear compression mechanism is adopted to weaken the weight of the amplitude values within the region. The calculation formula for the compressed spectral amplitude is as follows:

[0040]

[0041] where A(f) is the amplitude of the original speech at frequency f, θ2 is the amplitude compression intensity coefficient, ω is the power exponent controlling the compression non-linearity, and A'(f) is the compressed spectral amplitude;

[0042] After amplitude compression, an adaptive band-stop filter with adjustable parameters is constructed based on the spectral location and range of the contaminated region to perform targeted filtering of the high-frequency contaminated region. The transfer ratio of the filter at each frequency point is calculated using the following formula:

[0043]

[0044] In the formula, θ3 is the filter suppression strength coefficient. It is the normalized window function, and H(f) is the proportion of the spectrum signal transmitted by the filter at frequency f;

[0045] The final output spectrum calculation formula is as follows: A out (f)=A'(f)·H(f), where A out (f) is the final output spectral amplitude, which is the amplitude value of the speech signal at frequency f after filtering.

[0046] The technical effects and advantages provided by the present invention in the above technical solution are as follows:

[0047] This invention effectively ensures the integrity of the spectral structure of speech data and the purity of personalized voiceprint features by introducing a precise identification and dynamic suppression mechanism for implicit high-frequency noise pollution before speech modeling. Compared with the lack of identification and intervention mechanisms for high-frequency interference in traditional speech acquisition processes, this method not only introduces quantitative analysis and deep model evaluation of pollution feature indicators, but also constructs a closed-loop mechanism of "identification-judgment-response" for high-frequency anomalies through multiple control methods such as adaptive adjustment of spectral boundaries, nonlinear spectral compression, and adaptive window filtering. This significantly improves the modeling accuracy of the voice cloning model for intrinsic speech features, reduces the risk of harsh whistling and spectral distortion in synthesized speech, and thus enhances the naturalness, consistency, and user experience of translation headsets in multilingual speech output. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0049] Figure 1 This is a flowchart of a method for cloning the sound of a translation headset according to the present invention. Detailed Implementation

[0050] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that the description of this disclosure will be more complete and fully convey the concept of the exemplary embodiments to those skilled in the art.

[0051] This invention provides, for example Figure 1 The method shown here for sound cloning of translation headphones includes the following steps:

[0052] During the initialization phase before the translation headset is activated, multiple segments of voice data of the wearer in a natural context are collected through a high-fidelity microphone to comprehensively extract their personalized voiceprint features, in order to support the training and optimization of the subsequent voice cloning model.

[0053] During the initialization phase before the translation headset is activated, the system uses a high-fidelity microphone to collect multiple segments of speech data from the wearer in natural contexts (such as daily conversations, declarative sentences, question-and-answer sentences, emotional intonation, and other scenarios), constructing a diverse and continuous set of original language samples. This language data typically includes multi-dimensional acoustic features such as pitch, timbre, speech rate, intonation, formants, tone variations, and rhythm, used to characterize the wearer's personalized voiceprint. The core purpose of collecting this data is to provide accurate and comprehensive training samples for the voice cloning model, enabling the model to synthesize target language speech with the wearer's unique timbre and tone style after semantic translation, achieving a natural cross-language output experience with "speaker identity consistency."

[0054] In the initialization phase before the translation headset is activated, personalized voiceprint acquisition can be divided into four specific steps: First, the wearer is guided to complete multiple rounds of voice recording in a natural context, including daily greetings, emotional phrases, and commonly used conversational phrases, to ensure the diversity of the corpus and the coverage of language contexts; second, the system uses a high-fidelity microphone to collect each voice segment in real time, completely recording the time-domain and frequency-domain signal characteristics of the voice; third, the system performs sentence-level segmentation and meta-information labeling on the collected voice samples, such as labeling voice attribute tags such as language, speech rate, gender, and emotional state; finally, all labeled voice samples are aggregated and stored in a local cache or cloud database, providing a high-quality and structurally complete voice data foundation for the subsequent training and parameter initialization of the personalized voice cloning model.

[0055] During the data acquisition process, the equipment should have a high dynamic range (e.g., above 96dB) and wide frequency response (>24kHz) to ensure that potential pollution sources can be recorded, thereby providing original evidence for subsequent pollution detection.

[0056] The collected speech data is preprocessed, and the core indicators that characterize the presence of hidden high-frequency pollution in the speech data are extracted from the preprocessed data through feature engineering methods. The extracted core indicators are comprehensively analyzed to quantify the degree of pollution in the current speech data.

[0057] The preprocessing of the acquired speech data mainly includes the following key steps: First, silence segments before and after the speech data are removed, and non-vocalized parts are eliminated to reduce interference from invalid information in subsequent analysis. Second, the speech signal is amplitude normalized to ensure that different speech samples have a uniform dynamic range in amplitude, eliminating the impact of differences in speech volume on model training. Third, the time-domain signal is converted into a time-frequency domain representation using Short-Time Fourier Transform (STFT) to generate a two-dimensional image reflecting the speech spectrum structure, which is used to reveal the energy distribution characteristics of speech at different frequencies and times. Finally, the speech signal is segmented into frames and windowed, dividing continuous speech into short frames of fixed length, and a window function (such as a Hamming window) is applied to suppress boundary effects, ensuring the smoothness and stability of the spectrum analysis. The core function of this series of operations is to provide a clear, scale-uniform, and information-dense spectral foundation for subsequent feature extraction and speech modeling, improving the model's learning effect and generalization ability for speech patterns.

[0058] Feature engineering methods are used to extract core indicators characterizing the presence of latent high-frequency pollution in speech data from preprocessed data. The extracted indicators include the degree of deviation of the high-frequency spectral envelope shape from the normal template and the synchronicity between the speech intonation trend (low-frequency component) and high-frequency energy changes. The degree of deviation of the high-frequency spectral envelope shape from the normal template and the synchronicity between the speech intonation trend (low-frequency component) and high-frequency energy changes are comprehensively analyzed under the detection window to generate high-frequency distortion index and intonation-high-frequency mismatch index respectively. The degree of pollution of the current speech data is quantified by the high-frequency distortion index and the intonation-high-frequency mismatch index.

[0059] A significant deviation in the shape of the high-frequency spectral envelope from the normal template usually indicates the presence of latent high-frequency contamination in the acquired speech data. This is because natural human voices exhibit relatively stable energy attenuation characteristics and resonance modes in the high-frequency range (e.g., 18–24 kHz), resulting in a smooth, gradually decreasing spectral envelope. However, when latent contamination sources such as LED drive pulses, electromagnetic interference, hearing aid whistling, or motor harmonics are present, these non-verbal signals can be mixed into the speech as abnormal frequency components or sudden energy peaks, significantly altering the structural characteristics of the original high-frequency envelope. This manifests as abrupt changes, spikes, and local bounces, resulting in morphological distortions. These changes cannot be explained by normal vocalization and are often unrelated to semantics or intonation, constituting a systemic high-frequency contamination characteristic. Therefore, when the system detects a significant morphological deviation between the high-frequency envelope of the speech and the standard template, it serves as a crucial criterion for judging whether the speech data has been contaminated by high-frequency contamination, posing a substantial threat to the accuracy and usability of subsequent speech modeling.

[0060] The specific steps for generating a high-frequency distortion index by comprehensively analyzing the deviation of the high-frequency spectrum envelope shape from the normal template within the detection window are as follows:

[0061] After the speech signal undergoes Short Time Fourier Transform (STFT) processing, the high-frequency spectrum corresponding to the detection window is extracted from the spectrogram. The spectral envelope curve of each frame within the frequency band is sampled and normalized to a unit interval [0, 1]. Simultaneously, the built-in standard speech high-frequency template envelope is invoked, which is the reference envelope obtained by training multiple speech segments under uncontaminated conditions. For each frequency point, the normalized structure deviation is calculated, and the calculation expression is as follows:

[0062]

[0063] In the formula, S(f i S(f) represents the spectral amplitude of the actual speech sample at frequency i (i.e., the magnitude after Fourier transform), reflecting the energy distribution intensity of the speech signal at that frequency, with the unit being amplitude (which can be further logarithmically converted to dB). max(S(f)) is the maximum spectral amplitude of the actual speech sample among all frequency points in the high-frequency band (e.g., 18kHz–24kHz) within the current detection window. The current speech spectrum is normalized to a maximum value of 1 to avoid errors caused by different speech volumes in different speech segments. ref (f i ) is the spectral amplitude value of the standard speech template at frequency point i, serving as the structural benchmark for "clean speech" and used to determine whether the current speech deviates from the normal state at this frequency point, max(S ref (f) represents the maximum amplitude value of the reference template in the high-frequency band. The template spectrum is also normalized to ensure that the difference calculation is performed on a uniform amplitude scale. Δ iIt is the normalized structural deviation of the actual speech sample at frequency point i from the spectrum of the reference template, with a value range of [0, 1], where 0 indicates complete consistency and 1 indicates maximum deviation;

[0064] The above steps are used to measure the relative deviation magnitude between the actual speech envelope and the reference template at each frequency point. Δ i The larger the value, the more severe the deviation in the spectral structure, reflecting potential local high-frequency pollution.

[0065] After obtaining the envelope deviation at each frequency point, a nonlinear enhancement mechanism is introduced to enhance the response sensitivity to severe high-frequency anomalies. The differences of all frequency points are superimposed through a nonlinear mapping function to generate a high-frequency distortion index, the calculation expression of which is as follows:

[0066]

[0067] In the formula, N is the total number of sampling frequency points included in the high-frequency band, and e is the natural base. It is the deviation enhancement coefficient (nonlinear amplification factor), which is used in the control function to measure the degree of deviation Δ. i The amplification sensitivity ranges from [3, 10] and can be adaptively adjusted according to the application scenario. HFSWI is a high-frequency distortion index that quantifies the degree of structural deviation of the high-frequency spectral envelope in a speech signal relative to a normal speech template. The theoretical value range is: HFSWI∈[0, +∞). When HFSWI≈0, it means that the high-frequency structure of the speech is very close to the normal template and there is no obvious pollution. When HFSWI>0.3~0.5, there may be a slight structural abnormality. When HFSWI>1.0, it means that the high-frequency spectral morphology is significantly distorted and is very likely to be affected by high-frequency pollution. In the case of extreme pollution or spectrum out of control, HFSWI can exceed 2.0 or even higher, depending on the λ coefficient and the intensity of pollution characteristics.

[0068] In the calculation of the high-spectral distortion exponent, the nonlinear mapping function refers to the exponential function form. Its function is to measure the spectral structure deviation Δ at each frequency point. iNonlinear enhancement processing suppresses small structural deviations while significantly amplifying large anomalous deviations, thereby highlighting potential high-frequency contamination features in speech. The reason for using this nonlinear mapping function is that high-frequency latent contamination typically presents only peaks or structural abrupt changes at certain frequency points. If linear functions are used, these local outliers are easily masked by averaging effects in the overall score, failing to effectively reflect the severity of contamination. Exponential functions, as a common convex growth function, can exponentially amplify the influence of outliers, improving detection sensitivity, and are one of the widely used enhancement functions in signal detection and spectral anomaly recognition. This function belongs to the standard family of mathematical exponential functions and is used as a nonlinear activation or weighting mechanism in many existing image enhancement, anomaly detection, and speech recognition models, possessing a clear engineering basis and mathematical stability.

[0069] The higher the high-frequency distortion index generated by comprehensively analyzing the deviation of the high-frequency spectral envelope shape from the normal template within the detection window, the greater the deviation of the energy distribution in the high-frequency part of the current speech spectrum from normal speech, such as envelope abrupt changes, local surges, and asymmetric structures. These characteristics often originate from high-frequency excitation noise, harmonic interference, or implicit pollution signals introduced by electronic devices. Conversely, if the index value is small, it indicates that the current high-frequency envelope is consistent with the standard shape and there is no obvious structural abnormality, and the high-frequency part of the speech data can be judged as clean.

[0070] An abnormal mismatch between the intonation trend (i.e., the fundamental frequency F0 and intonation fluctuations reflected in the low-frequency components) and high-frequency energy changes is often an important indicator of latent high-frequency pollution in speech data. This is because in natural speech, the distribution of high-frequency energy usually exhibits a synchronous fluctuation relationship with low-frequency driving factors such as intonation, emotion, and vocal intensity. For example, when the intonation rises, vocal energy increases, and the high-frequency components also rise accordingly. However, if the high-frequency energy shows an abnormal increase, abrupt change, or continuous distribution on the time axis independent of intonation changes, it often indicates that the high-frequency signal is not driven by the human voice itself, but rather originates from electromagnetic coupling of equipment, harmonic interference, or the injection of inaudible high-frequency sources in the environment. This "decoupling" phenomenon of the low-frequency-high-frequency relationship reflects an imbalance in the spectral structure and is a typical characteristic of latent high-frequency pollution. Therefore, by detecting the degree of mismatch between intonation trend and high-frequency energy changes, the presence of pollution can be indirectly quantitatively identified, becoming one of the key indicators in intelligent assessment of speech data quality.

[0071] The specific steps for generating a pitch-high frequency mismatch index by comprehensively analyzing the synchronicity between the intonation trend (low-frequency component) and high-frequency energy changes within a detection window are as follows:

[0072] Within the detection window, the fundamental frequency sequence F0(j) of the speech signal is first extracted, and a normalized dynamic curve of intonation change is constructed using a weighted rate of change to highlight the subtle trends in intonation. Simultaneously, the total energy E of the target high-frequency band (e.g., 18-24kHz) in each frame is extracted from the spectrogram. HF (j) Construct a normalized high-frequency energy variation curve and perform amplitude rounding to align the scale, as shown in the following formula:

[0073]

[0074] In the formula, ΔF0(j) is the rate of change of the fundamental frequency of the speech signal between the j-th frame and the previous frame, calculated as follows: ΔF0(j)=F0(j)-F0(j-1), where F0(j) is the fundamental frequency value of the speech signal in the current j-th frame. The fundamental frequency represents the dominant frequency of vocal cord vibration and is the basis of the "pitch" of speech, directly determining the trend of intonation (rising, falling, or flat), F0(j-1) is the fundamental frequency value of the speech signal in the (j-1)-th frame, i.e., the fundamental frequency of the previous frame, and λ1 is the intonation change response coefficient, used to control intonation. The rate of change in the normalization mapping process is "amplified / compressed" in intensity, ranging from 0.5 to 5. If λ1 is large, small intonation fluctuations will be amplified, enhancing the perception of subtle intonation changes. If the value is small, the system is not sensitive to intonation changes and only responds to obvious intonation fluctuations. P(j) is the intonation change dynamic curve, representing the normalized expression of intonation change in frame j. It is a continuous value between [-1, 1], used to reflect the intonation dynamic change trend of the current speech frame, i.e., "intensity of intonation fluctuation". E HF (j) is the total energy value of the speech signal in the high-frequency band (e.g., 18–24kHz) in the j-th frame. It is obtained by summing the values ​​in the target frequency band after spectral decomposition. It is an important basic data for detecting hidden high-frequency pollution, especially for identifying abnormal energy accumulation in the high-frequency band. λ2 is the high-frequency energy response coefficient, which controls the sensitivity to high-frequency energy. Its value is adjusted within the range of 0.1-3 to balance the recognition sensitivity and anti-interference ability. ln(1+E HF (j) is a logarithmic compression function that performs amplitude compression on high-frequency energy to avoid abnormal peaks dominating the curve shape and to prevent extreme high-frequency values ​​from distorting the overall trend. The addition operation ensures that even in E HF When (j) = 0, the function also has a numerical definition. H(j) is a high-frequency energy change curve, which represents the normalized result of high-frequency energy in the j-th frame. It is a value between [-1, 1], used to reflect the energy change trend of the high-frequency band (such as 18–24kHz) in the current speech frame.

[0075] Logarithmic compression is a nonlinear amplitude modulation function commonly used in signal processing. Its main function is to compress the range of values ​​with excessively large amplitude differences in a signal, thereby enhancing the expressive power of weak signals and suppressing the dominant influence of extremely high amplitude values ​​on the overall model. Wherein ln(1+E) HF (j) is a typical logarithmic compression function. It is an existing and widely used standard nonlinear function, a fundamental tool in fields such as speech signal processing, image dynamic range compression, and neural network input normalization. Its main characteristics are: it maintains a delicate response (approximately linear) to small-amplitude inputs, while gradually "flattening" the response to large-amplitude inputs, thus preventing occasional strong noise points in the high-frequency band (such as howling and harmonic peaks) from excessively dominating the entire characteristic curve, and helping to construct a more stable and representative high-frequency energy change index. The increment operation is to avoid... HF The problem of the logarithmic function being undefined when (j) = 0 falls under conventional mathematical treatment. This function, combined with the λ² response coefficient and the tanh activation operation, constitutes a high-frequency sensing mechanism with nonlinear suppression, smooth response, and amplitude normalization.

[0076] In the two formulas in step 1, the tanh function (hyperbolic tangent function) serves as a nonlinear mapping tool, used to normalize the rate of change in intonation and the amplitude of changes in high-frequency energy, respectively. Its main function is to compress the input values ​​to the interval [-1, 1], thereby limiting the influence of extreme values ​​and enhancing the resolution of the middle interval, thus more effectively capturing the changing trend. In the formula for calculating the dynamic curve P(j) of intonation change, tanh smooths and normalizes the rate of change in the fundamental frequency, preventing sudden intonation jumps from causing structural distortions in subsequent feature trends, making the intonation curve more stable and comparable.

[0077] In the formula for calculating the high-frequency energy change curve H(j), tanh is used to suppress extreme values ​​in the high-frequency energy, ensuring that energy peaks caused by strong interference do not dominate the overall trend, thereby improving the detection sensitivity for mild but persistent contamination. The tanh function is chosen instead of simple linear normalization because its central symmetry and asymptotic edge characteristics are more suitable for expressing the "mid-segment sensitivity, extreme value suppression" pattern in speech signals, which is beneficial for the nonlinear modeling of differences in the subsequent mismatch index.

[0078] The core of this step is to construct two dynamic curve pairs that are synchronously correlated but will deviate when contamination is present, providing a basis for subsequent mismatch calculations.

[0079] Based on the acquired dynamic curve P(j) of intonation change and curve H(j) of high-frequency energy change, an intonation-high-frequency mismatch index is calculated by nonlinearly accumulating the point-to-point differences between the two within the detection window. This index is used to quantify the degree of synchronization imbalance between the two. The calculation expression is as follows:

[0080]

[0081] In the formula, PHFMI is the intonation-high frequency mismatch index. The larger the value, the more serious the synchronicity disconnect between the intonation trend and the high frequency energy change, which may be caused by implicit high frequency pollution from non-human voice excitation. M is the total number of frames in the detection window. δ is the asynchronous mutation amplification coefficient, which is a parameter that adjusts the "sensitivity" of the synchronization mismatch to the exponential response curve. Its value is in the range of [1,10] and is used to control the steepness of the exponential function. The larger the value, the more obvious the response of the formula to small synchronization mismatches (easier to identify weak pollution). The smaller the value, the more significantly only large mismatches are amplified. γ is the deviation amplitude enhancement index, which is a power parameter for nonlinearly enhancing the deviation amplitude. It is between 1.5 and 2 and is used to amplify the weight of the synchronization deviation, so that the score of high difference frames improves faster and contributes more to the overall index. It nonlinearly amplifies the "absolute amplitude" of the deviation, which helps to identify sudden high-energy shifts in the pollution area. e is the natural base.

[0082] By quantifying the degree of synchronization deviation between intonation changes and high-frequency energy changes in speech, we can identify whether there is a structural disconnect between the two. By constructing a nonlinear deviation accumulation mechanism, we can highlight the impact of local abrupt change pairs and accurately reveal potentially hidden high-frequency contamination behaviors in speech.

[0083] This study comprehensively analyzes the synchronicity between the intonation trend (low-frequency components) and high-frequency energy changes within a detection window to generate an intonation-high-frequency mismatch index. This index assesses the consistency between the intonation trend (mainly composed of low-frequency components such as the fundamental frequency F0) and high-frequency energy changes by quantifying the degree of synchronicity. In normal speech, these two usually have a strong temporal correlation; that is, when the intonation rises or the tone intensifies, the high-frequency energy also increases accordingly, exhibiting high consistency, and the intonation-high-frequency mismatch index value is small. However, a larger intonation-high-frequency mismatch index value indicates that the changes in high-frequency energy and intonation do not have a clear synchronous trend, resulting in a "disconnection" phenomenon. This is usually caused by high-frequency interference introduced from non-human voice sources (such as harmonics from electronic devices, microphone cavity resonance, etc.), reflecting the possible presence of latent high-frequency pollution in the speech. Therefore, a higher intonation-high-frequency mismatch index indicates a greater possibility of pollution; conversely, a smaller intonation-high-frequency mismatch index indicates good overall speech structure coordination, natural coordination between the high-frequency components and intonation, and a lower risk of pollution.

[0084] The core indicators after comprehensive analysis are constructed into feature vectors and input into a pre-trained deep learning model (such as a convolutional neural network CNN or a Transformer attention model). The model intelligently evaluates the current data in real time to determine whether there is hidden high-frequency pollution in the speech sample.

[0085] The high-frequency distortion index and the intonation-high frequency mismatch index, after comprehensive analysis, are constructed into feature vectors and input into a pre-trained deep learning model (such as a convolutional neural network CNN or a Transformer attention model). The model generates a latent high-frequency pollution risk coefficient, and the current data is intelligently evaluated in real time based on the latent high-frequency pollution risk coefficient to determine whether there is latent high-frequency pollution in the speech sample.

[0086] The pre-trained deep learning model refers to a model that has been thoroughly trained by the R&D team using a large amount of labeled sample data before the deployment of the translation headphone sound cloning system. This enables the model to identify and judge the risk of latent high-frequency contamination from the input feature vector. The training phase of this model typically involves constructing a standard dataset based on a massive amount of known "clean speech" and "high-frequency contaminated speech." The contamination labels corresponding to each sample (e.g., no contamination, slight contamination, moderate contamination, heavy contamination) are used as supervisory input signals. Deep neural networks (such as CNN or Transformer structures) are used for parameter learning and feature weight allocation, thereby establishing a mapping relationship between input features (such as high-spectral distortion indicators and pitch-high-frequency mismatch indicators) and the contamination risk level. After training, the model has formed a generalizable feature discrimination mechanism. When faced with new, unseen speech samples, it can quickly determine whether they have signs of high-frequency contamination based on the structured indicator features of the input.

[0087] In this approach, the core advantage of the pre-trained deep model lies in its ability to model complex nonlinear feature relationships. For example, Convolutional Neural Networks (CNNs) can effectively capture the manifestations of local pattern variations and high-frequency perturbations in the spectrogram, making them suitable for extracting fixed-structure features; while Transformer models utilize their self-attention mechanism to establish long-range dependencies between feature dimensions, making them suitable for handling non-local behavioral patterns such as cross-frame mismatches between pitch and high-frequency energy. When the extracted high-spectral distortion index and pitch-high-frequency mismatch index are input into the model, the model automatically extracts the correlation between these features and outputs a "latent high-frequency contamination risk coefficient," which can be set with a threshold for contamination judgment (e.g., a coefficient > 0.7 indicates contamination). Since the model has been optimized on a large amount of real or synthetic contamination data, it possesses real-time performance, stability, and adaptability in the actual deployment phase. It can serve as the core decision-making tool in the translation headset system for determining whether speech data can be used for sound cloning modeling, ensuring the stability of the quality of the system-generated speech and the consistency of user experience.

[0088] The implicit high-frequency contamination risk coefficient generated by the pre-trained deep learning model during the intelligent real-time evaluation of the current data is compared and analyzed with a pre-set reference threshold for the implicit high-frequency contamination risk coefficient to determine whether implicit high-frequency contamination exists in the speech sample. The judgment logic is as follows:

[0089] If the risk coefficient of latent high-frequency contamination is greater than the preset reference threshold for latent high-frequency contamination risk coefficient, then latent high-frequency contamination is determined to exist in the speech sample; if the risk coefficient of latent high-frequency contamination is less than or equal to the preset reference threshold for latent high-frequency contamination risk coefficient, then latent high-frequency contamination is determined not to exist in the speech sample.

[0090] Once implicit high-frequency noise pollution is detected in the collected speech data, a high-frequency dynamic suppression mechanism is activated to adaptively adjust the spectral boundaries of the high-frequency part of the speech signal to limit the frequency range participating in subsequent feature modeling. Subsequently, a nonlinear spectral compression method is applied to suppress the amplitude of the identified abnormal frequency components, and an adaptive window filter with corresponding parameters is constructed based on the dynamic positioning results of the polluted frequency band to perform targeted filtering of the identified high-frequency pollution region and suppress the influence of abnormal frequency components.

[0091] The core function of this step is to accurately isolate and intervene in the energy of contaminated frequency bands after identifying implicit high-frequency noise pollution in the collected speech data. This is achieved through a series of dynamic regulation and targeted removal mechanisms targeting high-frequency pollution, preventing it from entering the subsequent feature modeling process and thus ensuring the quality and integrity of the training data for the voice cloning system and the stability of speech synthesis. This step is not a simple filtering and noise reduction, but a highly structured and hierarchical pollution control strategy, encompassing four dimensions: spectrum perception, modeling control, energy suppression, and spatial filtering.

[0092] First, adaptive adjustment of the spectral boundaries is the primary step in this mechanism. This refers to dynamically adjusting the upper limit of speech frequencies that can participate in subsequent speaker modeling, based on the actual distribution range of the contaminated frequencies after pollution is identified. Unlike traditional static high-pass or low-pass filtering, this boundary adjustment mechanism does not abruptly eliminate high-frequency bands. Instead, it sets a dynamic spectral gating range through a frequency window function (such as an adjustable-bandwidth Gaussian window), actively isolating the contaminated region from the modeling range in the frequency domain. For example, when pollution is located between 19.5–22kHz, the system sets this frequency band as a no-go zone for feature modeling, retaining only clean frequency data below 18kHz as model input. The core purpose of this is "isolation first, processing later," that is, prioritizing blocking the interference of contaminated frequencies on the modeling process from the data source, avoiding leaving structural "noise memory" during model training.

[0093] Secondly, after completing the spectral boundary constraints, the system performs nonlinear spectral compression to further weaken the dominance of residual energy in the polluted frequency bands. Since high-frequency pollution typically exists in the form of sharp resonant peaks, periodic excitations, or narrowband interference, without control, they can still affect feature modeling in the spectrogram as "high intensity, low diffusion." Nonlinear spectral compression technology applies nonlinear amplitude mapping functions such as logarithmic transformation, power functions, and sigmoid suppression to perform asymmetric compression of the energy in the polluted frequency bands, significantly reducing their relative influence in the overall spectral energy map. For example, the energy value of the 19.8kHz frequency point in the original signal is 0.9 (normalized scale), which is adjusted to below 0.3 after nonlinear compression, effectively preventing it from being misjudged as part of the speaker's features in subsequent modeling. This method has the characteristics of "preserving structure and weakening energy," achieving energy control of pollution sources while avoiding damage to speech integrity.

[0094] Furthermore, after energy compression, the system constructs a set of window-type band-stop filters with adaptive parameters based on the real-time location results of the contaminated frequency band, performing directional filtering of the contaminated frequency. The key technical point in this step lies in the dynamic adaptability of the filter parameters (center frequency, bandwidth, order, etc.). Unlike traditional fixed-bandwidth band-stop filters (notch filters) that can only handle noise of known frequencies, this system determines the main peak and fluctuation range of the contaminated frequency through real-time spectrum analysis, automatically setting the filter bandwidth and attenuation curve to achieve "customized" filter window generation based on the contaminated location. For example, after detecting the core contaminated frequency band between 19.2–20.1kHz, the system will generate a high-order filter with a center frequency of 19.65kHz and a bandwidth of 0.9kHz, accurately covering the contaminated range and avoiding "trapping effects" on spectral components above 20kHz or below 18kHz. In addition, if the contamination is intermittent in time, the system can also achieve "intermittent filtering" in the time domain based on a short-time analysis window, that is, activating the filter only when contamination occurs, enhancing the intelligence and high fidelity of the processing.

[0095] Through the combined effect of the above three strategies, the system not only achieves step-by-step suppression of high-frequency contamination signals—from spectral boundary setting to energy interference reduction to precise filtering and elimination—but also avoids common problems in traditional noise processing schemes such as over-filtering, loss of speech details, and frequency shifting. The greatest advantage of this processing flow lies in its ability to respond to both contamination intensity and protect against modeling tasks, achieving separate control of contamination and speech features. This provides clean and stable speech input data for personalized voice cloning, effectively supporting the natural restoration of synthesized speech in terms of emotion, timbre, and rhythm, and ensuring consistent identity output. The technical value of this step in the entire system lies not only in the effectiveness of contamination processing but also in its establishment of a reliable data foundation for intelligent speech synthesis tasks, forming a crucial barrier to ensure the quality of speech output from the translation headset.

[0096] Once implicit high-frequency noise pollution is identified in the collected speech data, a high-frequency dynamic suppression mechanism is activated to adaptively adjust the spectral boundaries of the high-frequency portion of the speech signal. Subsequently, a nonlinear spectral compression method is applied to suppress the amplitude of the identified abnormal frequency components. Based on the dynamic localization results of the polluted frequency band, an adaptive window filter with corresponding parameters is constructed. The specific steps for targeted filtering of the identified high-frequency pollution regions are as follows:

[0097] When implicit high-frequency noise pollution is detected in the collected speech data (the risk coefficient of implicit high-frequency pollution is greater than a pre-set reference threshold), a high-frequency dynamic suppression mechanism is activated. This mechanism adaptively adjusts the spectral boundaries of the high-frequency portion of the speech signal, dynamically determining the highest frequency limit for the current speech sample to be used for subsequent modeling. This upper limit is narrowed based on the current pollution risk level, and the calculation formula is as follows:

[0098]

[0099] In the formula, f max This is the upper limit of the modeling frequency, representing the highest upper limit of speech frequencies that the system allows to participate in subsequent feature modeling. When high-frequency pollution is detected in the speech, the frequency is lowered by adjusting f. max High-contamination frequency bands are removed from the modeling range to avoid negative impacts on the speech cloning model. These frequencies dynamically change with the degree of contamination, generally ranging from 16kHz to 22kHz. f0 is the default maximum frequency, referring to the upper limit of the default modeling frequency for the speech signal under uncontaminated conditions; it is the upper boundary of the original complete spectrum of the system, serving as a reference value for the standard spectrum upper limit. When there is no high-frequency contamination, f... max= f0, where all frequency bands can participate in modeling and is usually set to 22050Hz (corresponding to the Nyquist frequency at a sampling rate of 44.1kHz). θ1 is the frequency boundary shrinkage coefficient, which controls the adjustment intensity of the upper frequency limit compression and is a proportional coefficient between 0 and 1. This coefficient determines the sensitivity and intensity of frequency compression when the pollution risk value is higher than the reference threshold. The larger the value, the more obvious the frequency compression, R p is the implicit high-frequency pollution risk coefficient, which is the pollution risk score generated by the deep learning model based on the high-frequency abnormal features in the current speech data, reflecting the intensity and credibility of the implicit high-frequency pollution in the current speech sample. As the core variable for pollution discrimination and dynamic regulation, it directly determines whether to initiate boundary adjustment and the adjustment amplitude, and is normalized to between [0,1] or [0,100]. The higher the value, the more serious the pollution, R th is the reference threshold for the implicit high-frequency pollution risk coefficient;

[0100] Through the above adjustments, dynamic compression of the spectral range can be achieved to prevent high-pollution frequency bands from participating in feature modeling.

[0101] After determining the upper limit of the modeling frequency f max for all spectral components within the frequency range f max < f ≤ f0, a non-linear compression mechanism driven by the risk coefficient is adopted to weaken the weight of the amplitude values in the region. The formula for the compressed spectral amplitude is as follows:

[0102]

[0103] In the formula, A(f) is the amplitude of the original speech at frequency f, reflecting the intensity of the speech energy corresponding to the current frequency point and being the original input for compression processing. θ2 is the amplitude compression intensity coefficient, which is a global weight factor controlling the compression attenuation amplitude and has a value range between 0.5 - 5 and is adjustable, used to adjust the steepness of the overall compression curve. The larger the value, the more obvious the compression effect and the stronger the weakening of the spectral energy. ω is the power exponent controlling the compression non-linearity, used to control the growth curve shape of the frequency normalization distance in the overall amplitude compression, adjusting the frequency sensitivity of the compression amplitude in the high-frequency band, with a value range of (0.5, 3.0), selected according to the tolerance of the target system to high-frequency suppression. A'(f) is the compressed spectral amplitude, representing the amplitude value at frequency f after pollution-aware driven compression, reducing the energy expression weight of high-frequency noise in a "soft processing" manner to avoid damage to the speech structure caused by hard filtering;

[0104] This step dynamically adjusts the energy weakening level according to the frequency distance and pollution degree, thereby effectively suppressing the disturbance caused by abnormal frequency components to subsequent analysis.

[0105] "Employing a risk-based nonlinear compression mechanism to weight and weaken the amplitude values ​​within this region" means that after detecting latent contamination risks in the high-frequency region of the speech signal, the system does not directly delete the data in these frequency bands. Instead, it nonlinearly reduces the energy of these frequency bands based on the severity of the contamination (i.e., the latent high-frequency contamination risk coefficient). This compression process uses nonlinear transformations such as exponential or power functions, so that the more severe the contamination and the closer the frequency is to the core contamination region, the smaller its corresponding amplitude value is compressed. The purpose of this method is to avoid the destruction of speech integrity caused by direct filtering by "retaining and weakening" the energy influence of high-risk frequencies, while effectively suppressing the interference of contaminated frequencies on the subsequent speech modeling process, thus achieving a balance between contamination control and speech fidelity.

[0106] After amplitude compression, an adaptive band-stop filter with adjustable parameters is constructed based on the spectral location and range of the contaminated region to perform targeted filtering of the high-frequency contaminated region. The transfer ratio of the filter at each frequency point (i.e., how much energy is retained) is calculated using the following formula:

[0107]

[0108] In the formula, θ3 is the filter suppression strength coefficient, which controls the maximum degree of suppression of contaminated frequency energy by the band-stop filter. Its value ranges from 0 to 1. The larger the value, the stronger the suppression; the smaller the value, the more energy is retained. It is a normalized window function, a window function defined on the interval [0, 1]. Commonly used windows include Gaussian window, beta window, and Hanning window. H(f) is the proportion of the spectrum signal transmitted by the filter at frequency f (i.e. how much energy is retained). It determines whether the energy of a certain frequency component is fully retained (=1), partially suppressed (between 0 and 1), or severely weakened (approaching 0). Its value range is 0-1.

[0109] A normalized window function is a mathematical function defined within the interval [0, 1] for weighted control of a specific signal region. Its core function is to apply a point-by-point varying weight distribution to the signal amplitude, filter intensity, or other processing parameters within a certain interval to achieve a smooth transition or targeted suppression effect. In high-frequency contamination filtering scenarios, normalized window functions are often used to adjust the filter's suppression level according to the frequency position, ensuring that the filtering starts from the contamination region's starting frequency f. maxThe frequency range from the highest system frequency f0 is continuously varied, avoiding abrupt changes at the boundaries. This function can be normalized using existing classic window functions, such as Gaussian windows, beta windows, and Raised Cosine windows (e.g., Hanning windows, Blackman windows). By linearly mapping the input domain of these window functions to [0, 1], a "position weight function" suitable for polluted frequency bands can be constructed, achieving flexible suppression control of the filtering intensity from "weak at the beginning → strong in the middle → weakening at the end." This mechanism is widely used in signal processing for operations such as spectrum trimming, band-stop filtering, and power spectrum modulation, and is particularly suitable for high-quality, low-damage targeted intervention in polluted frequency bands.

[0110] The final output spectrum calculation formula is as follows: A out (f)=A'(f)·H(f), where A out (f) is the final output spectral amplitude, which is the amplitude value of the speech signal at frequency f after filtering. It represents the actual intensity of the speech signal at frequency f after pollution suppression processing and is the spectral amplitude data that the system ultimately uses for speech modeling, reconstruction or speech synthesis.

[0111] The above steps achieve targeted suppression and energy removal of high-risk frequencies, preventing them from interfering with the modeling structure of the speech cloning model.

[0112] By constructing an adaptive band-stop filter, targeted filtering is performed on high-frequency contamination regions, further reducing the residual energy of anomalous frequency components in the compressed spectrum. This step enables frequency domain response modulation based on the intensity of contamination risk, ensuring that contaminated frequency bands are accurately suppressed without affecting normal speech characteristics.

[0113] This invention effectively ensures the integrity of the spectral structure of speech data and the purity of personalized voiceprint features by introducing a precise identification and dynamic suppression mechanism for implicit high-frequency noise pollution before speech modeling. Compared with the lack of identification and intervention mechanisms for high-frequency interference in traditional speech acquisition processes, this method not only introduces quantitative analysis and deep model evaluation of pollution feature indicators, but also constructs a closed-loop mechanism of "identification-judgment-response" for high-frequency anomalies through multiple control methods such as adaptive adjustment of spectral boundaries, nonlinear spectral compression, and adaptive window filtering. This significantly improves the modeling accuracy of the voice cloning model for intrinsic speech features, reduces the risk of harsh whistling and spectral distortion in synthesized speech, and thus enhances the naturalness, consistency, and user experience of translation headsets in multilingual speech output.

[0114] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0115] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

[0116] It should be noted that, in this document, the use of relational terms such as "first" and "second" is merely for distinguishing one entity or operation from another, and does not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0117] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0118] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0119] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0120] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0121] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0122] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0123] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A method for sound cloning in translation headphones, characterized in that, Includes the following steps: During the initialization phase before the translation headset is activated, multiple segments of voice data of the wearer in a natural context are collected through a high-fidelity microphone to comprehensively extract their personalized voiceprint features. The collected speech data is preprocessed, and the core indicators that characterize the presence of hidden high-frequency pollution in the speech data are extracted from the preprocessed data through feature engineering methods. The extracted core indicators are comprehensively analyzed to quantify the degree of pollution in the current speech data. The core indicators after comprehensive analysis are constructed into feature vectors and input into a pre-trained deep learning model. The model then performs intelligent real-time evaluation of the current data to determine whether there is any hidden high-frequency pollution in the speech samples. Once implicit high-frequency noise pollution is detected in the collected speech data, a high-frequency dynamic suppression mechanism is activated to adaptively adjust the spectral boundaries of the high-frequency part of the speech signal to limit the frequency range participating in subsequent feature modeling. Subsequently, a nonlinear spectral compression method is applied to suppress the amplitude of the identified abnormal frequency components, and an adaptive window filter with corresponding parameters is constructed based on the dynamic positioning results of the polluted frequency band to perform targeted filtering of the identified high-frequency pollution region and suppress the influence of abnormal frequency components. Feature engineering methods are used to extract core indicators characterizing the presence of latent high-frequency pollution in speech data from preprocessed data. The extracted indicators include the degree of deviation of the high-frequency spectral envelope shape from the normal template and the synchronicity between the speech intonation trend and high-frequency energy changes. The degree of deviation of the high-frequency spectral envelope shape from the normal template and the synchronicity between the speech intonation trend and high-frequency energy changes are comprehensively analyzed under the detection window to generate high-frequency distortion index and intonation-high-frequency mismatch index respectively. The degree of pollution of the current speech data is quantified by the high-frequency distortion index and intonation-high-frequency mismatch index. The specific steps for generating a high-frequency distortion index by comprehensively analyzing the deviation of the high-frequency spectrum envelope shape from the normal template within the detection window are as follows: After the speech signal undergoes short-time Fourier transform processing, the high-frequency spectrum corresponding to the detection window is extracted from the spectrogram. The spectral envelope curve of each frame within the frequency band is sampled and normalized to a unit interval [0, 1]. Simultaneously, the built-in standard speech high-frequency template envelope is invoked, and the normalized structure deviation is calculated for each frequency point. The calculation expression is as follows: In the formula, It is the actual speech sample at the frequency point Spectral amplitude at that location It is the maximum spectral amplitude of the actual speech sample across all frequency points in the high-frequency band within the current detection window. It is a standard speech template at the frequency point The spectral amplitude value at that location, It is the maximum amplitude value of the reference template in the high-frequency band. It is the actual speech sample at the frequency point The normalized structure of the spectrum deviates from that of the reference template. After obtaining the envelope deviation at each frequency point, a nonlinear enhancement mechanism is introduced to enhance the response sensitivity to severe high-frequency anomalies. The differences of all frequency points are superimposed through a nonlinear mapping function to generate a high-frequency distortion index, the calculation expression of which is as follows: In the formula, It is the total number of sampling frequency points included in the high-frequency band. It is the natural base. It is the deviation enhancement coefficient. It is a high-frequency distortion index; The specific steps for generating a pitch-high frequency mismatch index by comprehensively analyzing the synchronicity between speech intonation trends and high-frequency energy changes within a detection window are as follows: Within the detection window, the fundamental frequency sequence of the speech signal is first extracted. We construct a normalized dynamic curve of intonation variation using weighted change rate to highlight subtle trends in intonation; simultaneously, we extract the total energy of the target high-frequency band in each frame from the spectrogram. A normalized high-frequency energy variation curve is constructed, and amplitude rounding is performed to align the scale, as shown in the following formula: In the formula, Is the speech signal in the 1st The rate of change of fundamental frequency between frames is calculated using the following formula: ,in, Is the speech signal at the current... The fundamental frequency value in the frame, Is the speech signal in the 1st The fundamental frequency value in the frame, that is, the fundamental frequency of the previous frame. It is the intonation change response coefficient. It is a dynamic curve of intonation changes. Is the speech signal in the 1st In the frame, the total energy value of the high-frequency band, It is the high-frequency energy response coefficient. It is a logarithmic compression function. It is a high-frequency energy change curve, representing the energy change at the 1st... The high-frequency energy normalization result of the frame; Based on the acquired dynamic curve of intonation change and high-frequency energy change curve The intonation-high frequency mismatch index is calculated by non-linearly accumulating the point-to-point differences between the two within the detection window. The calculation expression is as follows: In the formula, It is an index of tone-high frequency mismatch. It is the total number of frames in the detection window. It is the asynchronous mutation amplification factor. It is the deviation amplitude enhancement index. It is the natural base.

2. The sound cloning method for translation headphones according to claim 1, characterized in that, The specific steps for personalized voiceprint acquisition during the initialization phase before activating the translation headset are as follows: Wearers complete multiple rounds of voice-guided recording in natural contexts, including daily greetings, emotional phrases, and commonly used conversational phrases, ensuring the diversity of language data and the coverage of language contexts; Each segment of speech is captured in real time using a high-fidelity microphone, fully recording the time and frequency domain signal characteristics of the speech. Sentence-level segmentation and metadata tagging are performed on the collected speech samples; All tagged speech samples are aggregated and stored in a local cache or cloud database.

3. The sound cloning method for translation headphones according to claim 1, characterized in that, The high-frequency distortion index and the tone-high frequency mismatch index, after comprehensive analysis, are constructed into feature vectors and input into a pre-trained deep learning model. The model generates a latent high-frequency pollution risk coefficient, and the current data is intelligently evaluated in real time based on the latent high-frequency pollution risk coefficient to determine whether there is latent high-frequency pollution in the speech sample.

4. The sound cloning method for translation headphones according to claim 3, characterized in that, The implicit high-frequency contamination risk coefficient generated by the pre-trained deep learning model during the intelligent real-time evaluation of the current data is compared and analyzed with a pre-set reference threshold for the implicit high-frequency contamination risk coefficient to determine whether implicit high-frequency contamination exists in the speech sample. The judgment logic is as follows: If the risk coefficient of latent high-frequency contamination is greater than the preset reference threshold for latent high-frequency contamination risk coefficient, then latent high-frequency contamination is determined to exist in the speech sample; if the risk coefficient of latent high-frequency contamination is less than or equal to the preset reference threshold for latent high-frequency contamination risk coefficient, then latent high-frequency contamination is determined not to exist in the speech sample.

5. The sound cloning method for translation headphones according to claim 4, characterized in that, Once implicit high-frequency noise pollution is identified in the collected speech data, a high-frequency dynamic suppression mechanism is activated to adaptively adjust the spectral boundaries of the high-frequency portion of the speech signal. Subsequently, a nonlinear spectral compression method is applied to suppress the amplitude of the identified abnormal frequency components. Based on the dynamic localization results of the polluted frequency band, an adaptive window filter with corresponding parameters is constructed. The specific steps for targeted filtering of the identified high-frequency pollution regions are as follows: When implicit high-frequency noise pollution is detected in the collected speech data, a dynamic high-frequency suppression mechanism is activated. This mechanism adaptively adjusts the spectral boundaries of the high-frequency portion of the speech signal, dynamically determining the highest frequency limit for the current speech sample to be used in subsequent modeling. The calculation formula is as follows: In the formula, It is the upper limit of modeling frequency. It is the default highest frequency. It is the frequency boundary contraction coefficient. It is the risk coefficient of hidden high-frequency pollution. It is a reference threshold for the risk coefficient of hidden high-frequency pollution; Determine the upper limit of modeling frequency Then, regarding the frequency range All spectral components within the region are weighted using a risk-driven nonlinear compression mechanism. The compressed spectral amplitude is calculated using the following formula: In the formula, It is the original speech in frequency The range at that point, It is the amplitude compression strength coefficient. It is a power exponent that controls the compression of nonlinearity. It is the compressed spectral amplitude; After amplitude compression, an adaptive band-stop filter with adjustable parameters is constructed based on the spectral location and range of the contaminated region to perform targeted filtering of the high-frequency contaminated region. The transfer ratio of the filter at each frequency point is calculated using the following formula: In the formula, It is the filter suppression strength coefficient. It is a normalized window function. At frequency The proportion of the spectrum signal transmitted by the filter; The final output spectrum calculation formula is as follows: In the formula, It is the final output spectral amplitude, which is the frequency response of the speech signal after filtering. The amplitude value at that point.

Images