Universal voice information recovery device and method based on built-in sensor undersampling data

Abstract

The invention discloses a universal voice information recovery device and method based on built-in sensing undersampling data. The device comprises a signal preprocessing module, a fundamental frequency estimation module, a frequency spectrum reconstruction module and a frequency spectrum voice conversion module which are sequentially connected. According to the invention, not only can voice information in sensor data which is extremely narrow in bandwidth and seriously aliasing be recovered, but also the problem that a learning-based model is relatively poor in mobility is solved; and data collected by a built-in sensor of a mobile phone in different scenes have different characteristics, a voice information recovery system is directly constructed from the internal characteristics of sensor data and the characteristics of voice signals without using a data set for model training, and the voice information recovery system can adapt to the changes of users, environments and equipment, so that the hidden voice signal can be effectively recovered from the built-in sensor of the mobile phone.

Description

technical field [0001] The invention discloses a device and method for recovering general voice information based on built-in sensor under-sampling data. Background technique [0002] With the development of smart phone assistants, voice has become more and more popular in human-computer interaction, and has even gradually become the first choice for special groups such as the blind, the elderly, and children. As a result, more and more IoT devices and mobile devices are deploying voice assistants. For example, on mobile devices, there’s Apple’s Siri, Google’s Google Assistant, and Samsung’s Bixby; on smart speakers, there’s Amazon’s Alexa and Google’s Google Home; and on traditional PC devices, there’s Apple’s Siri and Microsoft’s Microsoft Cortana. Studies have shown that by 2023, the global market value of voice assistants will reach about 7.8 billion US dollars. However, due to the personalized service nature of voice interaction, some sensitive information is usually...

AI Technical Summary

Problems solved by technology

However, the bandwidth of the sensor data is extremely narrow and there is serious aliasing, and the voice super-resolution technology cannot effectively restore the voice information in it.

The second is to use learning-based (machine learning, deep learning) speech restoration technology. However, this method has two major disadvantages. First, the collection and labeling of sensor data is inevitably introduced, and it takes a lot of time Perform model training

Second, learning-based speech restoration models from sensor data suffer significant performance degradation when transferred to different subjects, environments, and devices

Images