Earphone communication connection method and device
By introducing similarity coefficients into the headphone communication connection and adjusting the time-frequency masking, the problem of excessive noise removal in the prior art has been solved, and a higher quality voice enhancement effect is achieved.
Patent Information
- Application Number
- CN202411751097.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-02
- Publication Date
- 2025-03-14
AI Technical Summary
In the process of reconstructing speech, the prior art ignores the similarity of the amplitude spectrum between the target speech, noisy speech and noise interference, resulting in excessive noise removal and speech distortion when the similarity is high.
By receiving noisy voice data, time-frequency decomposition and speech feature extraction are performed, noise features are identified using deep neural network models, similarity coefficients are calculated, time-frequency masking is adjusted, amplitude spectrum estimates of the target speech are calculated, and inverse short-time Fourier transform is performed to obtain enhanced speech data.
Effectively remove interfering noise while avoiding excessive noise removal leading to speech distortion, retaining more target speech components, and improving the quality of speech enhancement.
Smart Images
Figure CN119626240A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of earphone technology, and in particular to an earphone communication connection method and device. Background Art
[0002] The principle of speech enhancement based on the masking effect of the human ear is to utilize frequency domain characteristic analysis to mask the low-energy frequency components with high-energy frequencies through the masking effect of the human ear, thereby achieving the purpose of reducing noise. It is best to perform an inverse transform to obtain the enhanced speech signal.
[0003] In the prior art, this method ignores the impact of the similarity of the amplitude spectra between the target speech, noisy speech and noise interference on the enhancement performance during speech reconstruction, resulting in the problem of excessive noise removal leading to speech distortion when the similarity is high. Summary of the invention
[0004] In view of the deficiencies in the prior art, the purpose of the present invention is to provide a headset communication connection method and device, aiming to solve the technical problem in the prior art that excessive noise removal easily leads to voice distortion.
[0005] In order to achieve the above object, the present invention is implemented by the following technical solution: a headset communication connection method, comprising the following steps:
[0006] Receiving noisy speech data, wherein the noisy speech data includes desired speech data and noise data;
[0007] Performing time-frequency decomposition and speech feature extraction on the speech data to obtain an amplitude spectrum of the noisy speech data, and identifying the noise features of the amplitude spectrum based on a deep neural network model to obtain a time-frequency mask corresponding to the noise data;
[0008] Based on the proportion of expected speech data and noise data in the noisy speech data, obtaining a similarity coefficient corresponding to the noisy speech data;
[0009] Adjusting the time-frequency mask based on the similarity coefficient to obtain a target mask;
[0010] An amplitude spectrum estimation value of the expected speech data is obtained based on the target masking calculation, and an inverse short-time Fourier transform is performed based on the amplitude spectrum estimation value and phase information of the noisy speech data to obtain enhanced speech data.
[0011] According to one aspect of the above technical solution, the calculation expression of the target mask is:
[0012]
[0013] Where M(t, k) is the target mask, t is the number of frames, k is the frequency, S(t, k) is the amplitude spectrum vector obtained by short-time Fourier transform of the expected speech data, V(t, k) is the amplitude spectrum vector obtained by short-time Fourier transform of the noise data, ρ s (t, k) is the first similarity coefficient between the noisy speech data and the expected speech data, ρ v (t, k) is the second similarity coefficient between the noisy speech data and the noise data.
[0014] According to one aspect of the above technical solution, the calculation expression of short-time Fourier transform is:
[0015]
[0016] Where n is a discrete time variable, L is the frame length of speech, ω(n) is the window function, N is the window length of the window function, x(n) is the time domain conversion function, φ(n) is the comprehensive window function, j is an imaginary unit, and e is a constant.
[0017] According to one aspect of the above technical solution, the method further includes:
[0018] Based on different types of noise data and standard speech data, synthesis is performed to obtain training noisy speech data;
[0019] The amplitude spectrum of the noisy speech data is extracted as a feature and target for deep neural network training, thereby obtaining a deep neural network model with network weight parameters whose errors meet the target range.
[0020] According to one aspect of the above technical solution, the calculation expression of the first similarity coefficient is:
[0021]
[0022] Wherein, Y(t, k) is the amplitude spectrum vector obtained by short-time Fourier transform of the noisy speech data, and T is the transposed symbol;
[0023] The calculation expression of the second similarity coefficient is:
[0024]
[0025] According to one aspect of the above technical solution, after the step of obtaining enhanced voice data, the method further includes:
[0026] Calculating the phase difference between the noisy speech data acquired by the left earphone and the right earphone, and adjusting the gain coefficient of the frequency domain noise reduction filter according to the phase difference;
[0027] Based on the frequency domain noise reduction filter and the gain coefficient, output voice data is obtained.
[0028] According to one aspect of the above technical solution, the calculation expression of the gain coefficient is:
[0029]
[0030] Where G(t, k) is the gain coefficient, φ is the phase difference, t is the number of frames, k is the frequency, ρ is the original gain coefficient, and X 0 (t, k) and X 1 (t, k) are the noisy speech data acquired by the left earphone and the right earphone respectively, ∠ is the phase symbol, and K is the value obtained by fast Fourier transform of the noisy speech data.
[0031] On the other hand, the present application also provides a headset communication connection device, comprising:
[0032] A receiving module, configured to receive noisy speech data, wherein the noisy speech data includes expected speech data and noise data;
[0033] A recognition module is used to perform time-frequency decomposition and speech feature extraction on the speech data to obtain an amplitude spectrum of the noisy speech data, and to recognize the noise features of the amplitude spectrum based on a deep neural network model to obtain a time-frequency mask corresponding to the noise data;
[0034] A similarity module, for obtaining a similarity coefficient corresponding to the noisy speech data based on the proportion of the expected speech data and the noise data in the noisy speech data;
[0035] An adjustment module, configured to adjust the time-frequency mask based on the similarity coefficient to obtain a target mask;
[0036] The enhancement processing module is used to obtain the amplitude spectrum estimation value of the expected speech data based on the target masking calculation, and to perform inverse short-time Fourier transform based on the amplitude spectrum estimation value and the phase information of the noisy speech data to obtain enhanced speech data.
[0037] According to one aspect of the above technical solution, the device further includes:
[0038] A training module, used for synthesizing based on different types of noise data and standard speech data to obtain training noisy speech data;
[0039] The amplitude spectrum of the noisy speech data is extracted as a feature and target for deep neural network training, thereby obtaining a deep neural network model with network weight parameters whose errors meet the target range.
[0040] According to one aspect of the above technical solution, the device further includes:
[0041] An output module, used to calculate the phase difference between the noisy speech data acquired by the left earphone and the right earphone, and adjust the gain coefficient of the frequency domain noise reduction filter according to the phase difference;
[0042] Based on the frequency domain noise reduction filter and the gain coefficient, output voice data is obtained.
[0043] Compared with the prior art, the beneficial effects of the present invention are as follows: by introducing a similarity coefficient, the energy proportions of noisy speech and noise as well as noisy speech and target speech are adjusted from the perspective of human ear perception, and the masking value of each time-frequency unit is adjusted, so that the trained target masking is closer to the masking value of pure speech under low signal-to-noise ratio, and while removing interference noise, the problem of speech distortion caused by excessive noise removal by ideal ratio masking is avoided, and more target speech components are effectively retained. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 This is a flow chart of the headset communication connection method in the first embodiment of the present invention;
[0045] Figure 2 It is a structural block diagram of the headset communication connection device in the second embodiment of the present invention;
[0046] The following specific implementation manner will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION
[0047] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Several embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.
[0048] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.
[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.
[0050] See also Figure 1 , which shows a headset communication connection method in a first embodiment of the present invention. As shown in the figure, the method includes the following steps:
[0051] Step S100, receiving noisy speech data, wherein the noisy speech data includes expected speech data and noise data. The expected speech data is the speech information that the caller expects to send based on the headset microphone.
[0052] Step S200, performing time-frequency decomposition and speech feature extraction on the speech data to obtain an amplitude spectrum of the noisy speech data, and identifying the noise features of the amplitude spectrum based on a deep neural network model to obtain a time-frequency mask corresponding to the noise data.
[0053] For ease of understanding, the above-mentioned time-frequency masking is a conventional ideal binary masking, which marks the time-frequency unit according to the signal-to-noise ratio (SNR) and the set threshold, and represents the time-frequency unit as 1 and 0. If the proportion of speech in a time-frequency unit is large, this time-frequency unit is marked as the target speech unit, that is, the target speech dominates in this time-frequency unit; if the proportion of noise in a time-frequency unit is large, this time-frequency unit is marked as the noise unit, that is, the noise dominates in this time-frequency unit. Its expression is:
[0054]
[0055] Where P s (t, k) and P v (t, k) represents the energy of pure speech and noise respectively, LC is the threshold, when the SNR value in the time-frequency unit exceeds the threshold, IBM takes the value of 1, when the SNR value in the time-frequency unit does not exceed the threshold, IBM takes the value of 0. The purpose of this step is to set the time-frequency unit with low signal-to-noise ratio to 0 to achieve the purpose of noise elimination.
[0056] Furthermore, the above method also includes:
[0057] Based on different types of noise data and standard speech data, synthesis is performed to obtain training noisy speech data;
[0058] The amplitude spectrum of the noisy speech data is extracted as a feature and target for deep neural network training, thereby obtaining a deep neural network model with network weight parameters whose errors meet the target range.
[0059] Step S300: obtaining a similarity coefficient corresponding to the noisy speech data based on the proportion of the expected speech data and the noise data in the noisy speech data.
[0060] Step S400: adjusting the time-frequency mask based on the similarity coefficient to obtain a target mask.
[0061] Specifically, in this embodiment, the calculation expression of the above target masking is:
[0062]
[0063] Where M(t, k) is the target mask, t is the number of frames, k is the frequency, S(t, k) is the amplitude spectrum vector obtained by short-time Fourier transform of the expected speech data, V(t, k) is the amplitude spectrum vector obtained by short-time Fourier transform of the noise data, ρ s (t, k) is the first similarity coefficient between the noisy speech data and the expected speech data, ρ v (t, k) is the second similarity coefficient between the noisy speech data and the noise data.
[0064] Specifically, the calculation expression of the first similarity coefficient is:
[0065]
[0066] Wherein, Y(t, k) is the amplitude spectrum vector obtained by short-time Fourier transform of the noisy speech data, and T is the transposed symbol;
[0067] The calculation expression of the second similarity coefficient is:
[0068]
[0069] Specifically, when the expected speech data component in the noisy speech signal is higher, ρ s The closer (t, k) is to 1, the greater the similarity between the expected speech data and the noisy speech signal, and the amplitude spectrum of the expected speech data in the noisy speech is dominant. When the interference noise signal component in the noisy speech signal is higher, ρ v The closer (t, k) is to 1, the greater the similarity between the interference noise signal and the noisy speech signal, and the more components of the amplitude spectrum of the expected speech data in the noisy speech. By introducing the first similarity coefficient and the second similarity coefficient into the time-frequency masking, the ratio of the expected speech data to the noise data is expressed, so that the trained target masking is closer to the masking value of the pure speech under low signal-to-noise ratio, while removing the interference noise, avoiding the problem of speech distortion caused by excessive noise removal by the ideal ratio masking, and effectively retaining more target speech components.
[0070] Among them, the calculation expression of short-time Fourier transform is:
[0071]
[0072] Where n is a discrete time variable, L is the frame length of speech, ω(n) is the window function, N is the window length of the window function, x(n) is the time domain conversion function, φ(n) is the comprehensive window function, j is an imaginary unit, and e is a constant.
[0073] Step S500, obtaining an amplitude spectrum estimation value of the desired speech data based on the target masking calculation, and performing an inverse short-time Fourier transform based on the amplitude spectrum estimation value and the phase information of the noisy speech data to obtain enhanced speech data.
[0074] Preferably, in this embodiment, after the step of obtaining enhanced voice data, the method further comprises:
[0075] Calculating the phase difference between the noisy speech data acquired by the left earphone and the right earphone, and adjusting the gain coefficient of the frequency domain noise reduction filter according to the phase difference;
[0076] Based on the frequency domain noise reduction filter and the gain coefficient, output voice data is obtained.
[0077] Specifically, the calculation expression of the above gain coefficient is:
[0078]
[0079] Where G(t, k) is the gain coefficient, φ is the phase difference, t is the number of frames, k is the frequency, ρ is the original gain coefficient, and X 0 (t, k) and X 1 (t, k) are the noisy speech data acquired by the left earphone and the right earphone respectively, ∠ is the phase symbol, and K is the value obtained by fast Fourier transform of the noisy speech data.
[0080] It can be understood that the phase difference between the two microphones is related to the signal-to-noise ratio in each time-frequency component. The estimated phase difference information is used to suppress the time-frequency units with large phase differences. The above gain coefficient is the filter Wiener gain. The value of the above original gain coefficient affects the overall noise reduction performance. If the value of ρ is too large, the noise reduction performance and strength are high, but it will cause speech distortion; if the value of ρ is too small, the noise reduction performance is insufficient. When the value of ρ is 1, it is a standard ordinary gain coefficient.
[0081] In summary, the headphone communication connection method in the above embodiment of the present invention, by introducing the similarity coefficient, adjusts the energy ratio of noisy speech and noise as well as noisy speech and target speech from the perspective of human ear perception, and adjusts the masking value of each time-frequency unit, so that the trained target masking is closer to the masking value of pure speech under low signal-to-noise ratio, removes interference noise while avoiding the problem of speech distortion caused by excessive noise removal by ideal ratio masking, and effectively retains more target speech components.
[0082] Please refer to Figure 2 , a second embodiment of the present invention provides a headset communication connection device, such as Figure 2 As shown, the device includes a receiving module, a recognition module, a similarity module, an adjustment module and an enhanced processing module.
[0083] The receiving module is used to receive noisy speech data, wherein the noisy speech data includes expected speech data and noise data;
[0084] The recognition module is used to perform time-frequency decomposition and speech feature extraction on the speech data to obtain an amplitude spectrum of the noisy speech data, and to identify the noise features of the amplitude spectrum based on a deep neural network model to obtain a time-frequency mask corresponding to the noise data;
[0085] The similarity module is used to obtain a similarity coefficient corresponding to the noisy speech data based on the proportion of the expected speech data and the noise data in the noisy speech data;
[0086] The adjustment module is used to adjust the time-frequency mask based on the similarity coefficient to obtain the target mask;
[0087] The enhancement processing module is used to obtain the amplitude spectrum estimation value of the desired speech data based on the target masking calculation, and to perform inverse short-time Fourier transform based on the amplitude spectrum estimation value and the phase information of the noisy speech data to obtain enhanced speech data.
[0088] Preferably, in this embodiment, the device further includes:
[0089] A training module, used for synthesizing based on different types of noise data and standard speech data to obtain training noisy speech data;
[0090] The amplitude spectrum of the noisy speech data is extracted as a feature and target for deep neural network training, thereby obtaining a deep neural network model with network weight parameters whose errors meet the target range.
[0091] Preferably, in this embodiment, the device further includes:
[0092] An output module, used to calculate the phase difference between the noisy speech data acquired by the left earphone and the right earphone, and adjust the gain coefficient of the frequency domain noise reduction filter according to the phase difference;
[0093] Based on the frequency domain noise reduction filter and the gain coefficient, output voice data is obtained.
[0094] In summary, the earphone communication connection device in the above embodiment of the present invention, by introducing a similarity coefficient, adjusts the energy ratio of noisy speech and noise as well as noisy speech and target speech from the perspective of human ear perception, and adjusts the masking value of each time-frequency unit, so that the trained target masking is closer to the masking value of pure speech under low signal-to-noise ratio, removes interference noise while avoiding the problem of speech distortion caused by excessive noise removal by ideal ratio masking, and effectively retains more target speech components.
[0095] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.
[0096] The above-described embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the attached claims.
Claims
1. A headset communication connection method, characterized in that: The following steps are involved: Receiving noisy speech data, wherein the noisy speech data includes desired speech data and noise data; Performing time-frequency decomposition and speech feature extraction on the speech data to obtain an amplitude spectrum of the noisy speech data, and identifying the noise features of the amplitude spectrum based on a deep neural network model to obtain a time-frequency mask corresponding to the noise data; Based on the proportion of expected speech data and noise data in the noisy speech data, obtaining a similarity coefficient corresponding to the noisy speech data; Adjusting the time-frequency mask based on the similarity coefficient to obtain a target mask; An amplitude spectrum estimation value of the expected speech data is obtained based on the target masking calculation, and an inverse short-time Fourier transform is performed based on the amplitude spectrum estimation value and phase information of the noisy speech data to obtain enhanced speech data.
2. The headset communication connection method according to claim 1, characterized in that: The calculation expression of the target mask is: Where M(t, k) is the target mask, t is the number of frames, k is the frequency, S(t, k) is the amplitude spectrum vector obtained by short-time Fourier transform of the expected speech data, V(t, k) is the amplitude spectrum vector obtained by short-time Fourier transform of the noise data, ρ s (t, k) is the first similarity coefficient between the noisy speech data and the expected speech data, ρ v (t, k) is the second similarity coefficient between the noisy speech data and the noise data.
3. The headset communication connection method according to claim 2, characterized in that: The calculation expression of short-time Fourier transform is: Where n is a discrete time variable, L is the frame length of speech, ω(n) is the window function, N is the window length of the window function, x(n) is the time domain conversion function, φ(n) is the comprehensive window function, j is an imaginary unit, and e is a constant.
4. The headset communication connection method according to claim 1, characterized in that: The method further comprises: Based on different types of noise data and standard speech data, synthesis is performed to obtain training noisy speech data; The amplitude spectrum of the noisy speech data is extracted as a feature and target for deep neural network training, thereby obtaining a deep neural network model with network weight parameters whose errors meet the target range.
5. The headset communication connection method according to claim 2, characterized in that: The calculation expression of the first similarity coefficient is: Wherein, Y(t, k) is the amplitude spectrum vector obtained by short-time Fourier transform of the noisy speech data, and T is the transposed symbol; The calculation expression of the second similarity coefficient is:
6. The headset communication connection method according to claim 1, characterized in that: After the step of obtaining enhanced voice data, the method further comprises: Calculating the phase difference between the noisy speech data acquired by the left earphone and the right earphone, and adjusting the gain coefficient of the frequency domain noise reduction filter according to the phase difference; Based on the frequency domain noise reduction filter and the gain coefficient, output voice data is obtained.
7. The headset communication connection method according to claim 6, characterized in that: The calculation expression of the gain coefficient is: Wherein, G(t, k) is the gain coefficient, φ is the phase difference, t is the frame number, k is the frequency, ρ is the original gain coefficient, X0(t, k) and X1(t, k) are the noisy speech data obtained by the left earphone and the right earphone respectively, ∠ is the phase sign, and K is the value obtained by fast Fourier transform of the noisy speech data.
8. A headset communication connection device, characterized in that: include: A receiving module, configured to receive noisy speech data, wherein the noisy speech data includes expected speech data and noise data; A recognition module is used to perform time-frequency decomposition and speech feature extraction on the speech data to obtain an amplitude spectrum of the noisy speech data, and to recognize the noise features of the amplitude spectrum based on a deep neural network model to obtain a time-frequency mask corresponding to the noise data; A similarity module, for obtaining a similarity coefficient corresponding to the noisy speech data based on the proportion of the expected speech data and the noise data in the noisy speech data; An adjustment module, configured to adjust the time-frequency mask based on the similarity coefficient to obtain a target mask; The enhancement processing module is used to obtain the amplitude spectrum estimation value of the expected speech data based on the target masking calculation, and to perform inverse short-time Fourier transform based on the amplitude spectrum estimation value and the phase information of the noisy speech data to obtain enhanced speech data.
9. The headset communication connection device according to claim 8, characterized in that: The device also includes: A training module, used for synthesizing based on different types of noise data and standard speech data to obtain training noisy speech data; The amplitude spectrum of the noisy speech data is extracted as a feature and target for deep neural network training, thereby obtaining a deep neural network model with network weight parameters whose errors meet the target range.
10. The headset communication connection device according to claim 8, characterized in that: The device also includes: An output module, used to calculate the phase difference between the noisy speech data acquired by the left earphone and the right earphone, and adjust the gain coefficient of the frequency domain noise reduction filter according to the phase difference; Based on the frequency domain noise reduction filter and the gain coefficient, output voice data is obtained.