Captured Speech Frequency Compensation for Face Coverings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Face coverings, such as masks, cause non-linear attenuation of audio frequencies, making speech difficult to understand, as simply increasing volume does not account for the varying effects on different frequencies.
Innovation Solution
A method and apparatus that adjust the amplitudes of frequencies in captured audio based on the type and presence of face coverings, using reference and training audio to determine attenuation patterns and apply compensatory amplification, and optionally replacing the face covering in video with a synthesized mouth for improved comprehension.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If the volume of speech is increased by normal means (talking louder, turning up volume, moving closer), then the overall speech level is improved, but the non-linear attenuation characteristics of the face covering cannot be recovered and speech remains difficult to understand
Solution Approach 1:
The speech signal is divided into multiple frequency components using Fourier transform. Each frequency component is processed separately to apply targeted amplification. This segmentation allows the system to address the non-linear attenuation at different frequencies individually rather than applying a uniform volume increase, thereby recovering speech intelligibility that simple volume adjustment cannot achieve.
Solution Approach 2:
Different amplification factors are applied to different frequency components based on their specific attenuation characteristics. The system identifies which frequencies are most affected by the face covering and applies greater amplification to those specific frequency ranges while leaving other frequencies less amplified. This local quality approach ensures that speech intelligibility is improved where it is most needed without unnecessarily amplifying all frequencies equally.
2Loss of information
If all frequencies are amplified uniformly to compensate for attenuation, then the overall speech level is improved, but unaffected frequencies are also amplified causing unnatural sound
Solution Approach 1:
The system applies different amplification factors to different frequency components based on measured attenuation characteristics. Frequencies that are heavily attenuated by the face covering receive greater amplification, while frequencies that are less affected receive minimal or no amplification. This selective local quality approach maintains natural sound quality by avoiding uniform amplification of all frequencies, thereby preventing the unnatural sounding effect while still improving speech intelligibility in the affected frequency ranges.
Solution Approach 2:
The system uses reference audio recorded without a face covering to establish baseline attenuation characteristics. By comparing the reference audio with audio recorded while wearing the face covering, the system generates feedback information about which frequencies are attenuated and by how much. This feedback is used to dynamically adjust the amplification factors for each frequency component, ensuring that amplification is applied only where needed to maintain natural sound quality while improving intelligibility.
3Loss of information
If reference audio and training audio processing is implemented to determine attenuation patterns, then speech comprehension is enhanced, but the device complexity increases
Solution Approach 1:
The system performs preliminary actions by collecting reference audio recordings without face coverings and training audio recordings with face coverings before actual speech enhancement is needed. During these preliminary stages, the system analyzes and stores the attenuation characteristics of different face coverings. When speech enhancement is subsequently required, the pre-analyzed attenuation patterns are directly applied, avoiding the need for real-time complex analysis and reducing the computational burden during actual use.
Solution Approach 2:
The system creates simplified models or profiles of face covering attenuation characteristics based on analysis of reference and training audio. These copied attenuation patterns are stored and can be quickly applied to enhance speech without requiring full re-analysis each time. The copying approach reduces device complexity by replacing complex real-time analysis with pre-computed models that can be efficiently applied during speech enhancement operations.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The technology disclosed herein enables compensation for attenuation caused by face coverings in captured audio. In a particular embodiment, a method includes determining that a face covering is positioned to cover the mouth of a user of a user system. The method further includes receiving audio that includes speech from the user and adjusting amplitudes of frequencies in the audio to compensate for the face covering.