Device and method for general speech information restoration based on built-in sensor undersampling data

Abstract

The invention discloses a device and method for recovering general speech information based on built-in sensor undersampling data. The device includes a signal preprocessing module, a fundamental The frequency estimation module, spectrum reconstruction module and spectrum speech conversion module are connected in sequence, which not only can restore the speech information in the sensor data with extremely narrow bandwidth and severe aliasing, but also solves the problem of poor transferability of learning-based models. The data collected by the built-in sensor of the mobile phone has different characteristics in different scenarios. The present invention starts from the intrinsic characteristics of the sensor data and the characteristics of the voice signal, and directly constructs a voice information recovery system without using the data set for model training, and It can adapt to changes in users, environments and devices, and effectively recover hidden voice signals from built-in sensors in mobile phones.

Description

technical field [0001] The invention discloses a device and method for recovering general voice information based on built-in sensor under-sampling data. Background technique [0002] With the development of smart phone assistants, voice has become more and more popular in human-computer interaction, and has even gradually become the first choice for special groups such as the blind, the elderly, and children. As a result, more and more IoT devices and mobile devices are deploying voice assistants. For example, on mobile devices, there’s Apple’s Siri, Google’s Google Assistant, and Samsung’s Bixby; on smart speakers, there’s Amazon’s Alexa and Google’s Google Home; and on traditional PC devices, there’s Apple’s Siri and Microsoft’s Microsoft Cortana. Studies have shown that by 2023, the global market value of voice assistants will reach about 7.8 billion US dollars. However, due to the personalized service nature of voice interaction, some sensitive information is usually...

AI Technical Summary

Problems solved by technology

However, the bandwidth of the sensor data is extremely narrow and there is serious aliasing, and the voice super-resolution technology cannot effectively restore the voice information in it.

The second is to use learning-based (machine learning, deep learning) speech restoration technology. However, this method has two major disadvantages. First, the collection and labeling of sensor data is inevitably introduced, and it takes a lot of time Perform model training

Second, learning-based speech restoration models from sensor data suffer significant performance degradation when transferred to different subjects, environments, and devices

Images