Intelligent hearing aid system and method

Abstract

The invention relates to the technical field of hearing aid equipment, in particular to an intelligent hearing aid system and method, and the system comprises (1) a voice cloning and voiceprint recognition module which is used for collecting a user voice sample, generating a personalized voice model, and distinguishing different users through voiceprint recognition; (2) an STT module which is integrated in a hearing aid application program of the intelligent device and is used for converting environment voice into characters in real time; (3) the TTS module is used for synthesizing a voice signal based on the text content generated by the STT module, and supporting on-line generation of personalized voice by combining voice cloning and off-line use of a local engine; (4) a user interaction module which provides an intelligent device application interface and is used for volume adjustment, voice mode selection and multi-user management; the intelligent hearing aid system and the intelligent hearing aid method have the functions of multi-user adaptation capability, personalized voice output, high-precision voice recognition and seamless offline.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of hearing aid devices, and particularly relates to an intelligent hearing aid system and method. BACKGROUND

[0002] With the development of intelligent devices and Bluetooth technology, intelligent devices have powerful voice processing capabilities. In addition, Bluetooth earphones are widely used in daily life due to their portability and wireless transmission advantages. Although traditional hearing aids can effectively help hearing-impaired people improve their hearing, they are expensive, complex to use, noisy, and lack flexible personalized functions. Most existing hearing aid systems rely on special hardware and complex signal processing algorithms, making the devices large in size and limited in battery life. At the same time, the current hearing aid system has deficiencies in multi-user adaptation and personalized voice output, and it is difficult to meet the needs of different users and use scenarios. Therefore, in view of the above status, it is urgent to develop an intelligent hearing aid system and method to overcome the deficiencies in current practical applications. SUMMARY

[0003] The present application aims to provide an intelligent hearing aid system and method to solve the problems raised in the background.

[0004] To achieve the above-mentioned purpose, the present application provides the following technical solutions:

[0005] An intelligent hearing aid system, characterized in that it comprises:

[0006] (1) a voice cloning and voiceprint recognition module: used to collect user voice samples and generate personalized voice models, and distinguish different users through voiceprint recognition;

[0007] (2) an STT module: integrated into the hearing aid application program of the intelligent device, used to convert environmental voice into text in real time, supporting online calling of cloud API and offline switching of local engine in dual-mode processing;

[0008] (3) a TTS module: based on the text content generated by the STT module, synthesizes voice signals, supports online generation of personalized voice combined with voice cloning and offline use of local engine;

[0009] (4) a user interaction module: provides an application interface for the intelligent device, used for volume adjustment, voice mode selection and multi-user management;

[0010] (5) a Bluetooth earphone: used to receive voice signals processed by the intelligent device and output audio.

[0011] As a further solution of the present application: in the STT module, online mode calls cloud API to realize high-precision voice-to-text, and offline mode uses the local speech recognition engine built-in the intelligent device.

[0012] As a further scheme of the present application: in the TTS module, the cloud API is called in online mode to generate natural speech and fuse the speech cloning model to output personalized speech, and the local TTS engine is used for speech synthesis in offline mode.

[0013] As a further scheme of the present application: the user interaction module supports multi-user management, automatically switches the speech cloning model of different users through voiceprint recognition, and provides a voice mode selection function for quiet environment, noisy environment and face-to-face communication.

[0014] As a further scheme of the present application: the speech cloning and voiceprint recognition module comprises:

[0015] A speech data collection interface is used to collect speech samples of users and close friends;

[0016] A deep learning algorithm unit is used to train a voiceprint recognition model and generate a personalized speech cloning model.

[0017] As a further scheme of the present application: the voiceprint recognition module uses a deep learning algorithm based on a convolutional neural network to extract user voiceprint features.

[0018] As a further scheme of the present application: the user interaction module includes a user feedback interface for receiving auditory experience feedback to optimize system parameters.

[0019] As a further scheme of the present application: the Bluetooth headset uses a low-delay transmission protocol to receive and play the speech signal processed by the smart device in real time.

[0020] An intelligent hearing aid method based on the above-mentioned intelligent hearing aid system, comprising the following steps:

[0021] (1) Collecting environmental sound through a smart device, and confirming the identity of the current speaker through voiceprint recognition;

[0022] (2) If the speaker is a trained user, load the speech cloning model thereof; otherwise, use a standard speech mode;

[0023] (3) Convert the speech into text through the STT module, and then synthesize personalized speech through the TTS module and output the speech to the Bluetooth headset.

[0024] As a further scheme of the present application: when connected to the network, the cloud API is called to perform the STT and TTS functions, and when there is no network, the local engine is automatically switched to process.

[0025] Compared with the prior art, the present application has the following advantages:

[0026] 1. Multi-user adaptation capability: Through voiceprint recognition technology, the system can distinguish the voiceprint characteristics of different users, support personalized voice output for multiple users, and improve user experience;

[0027] 2. Personalized voice output: Voice cloning technology generates personalized voice output based on user's voice characteristics, improving the naturalness of voice interaction and user satisfaction;

[0028] 3. High-precision speech recognition: Through cloud API calling, the system can achieve higher accuracy of speech recognition;

[0029] 4. Seamless offline function: When there is no network connection, the system can automatically switch to the built-in speech recognition and speech synthesis engine of the smart device, ensuring the continuity and reliability of the hearing aid system;

[0030] 5. Economical and efficient: Using smart devices and Bluetooth earphones, there is no need to purchase expensive dedicated hearing aid devices, significantly reducing user usage costs;

[0031] 6. Convenient and easy to use: Through the smart device application, simple settings can be achieved, and users do not need complex installation and debugging, improving the operation experience;

[0032] 7. Personalized adjustment: Support for multiple voice modes and personalized settings to meet the hearing needs of different users and environmental adaptability. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 The figure is a schematic diagram of the intelligent hearing aid system architecture in the embodiment of the present application.

[0034] Figure 2 The figure is a functional module diagram of the hearing aid application in the embodiment of the present application.

[0035] Figure 3 The figure is a workflow diagram of the hearing aid system in the embodiment of the present application.

[0036] Figure 4 The figure is a schematic diagram of the hearing aid application interface of the smart device in the embodiment of the present application. DETAILED DESCRIPTION

[0037] The technical solutions in the embodiments of the present application will be described in detail below with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the present application.

[0038] The specific implementation of the present application will be described in detail below in conjunction with specific embodiments.

[0039] Please refer toFigures 1-4 The intelligent hearing aid system provided by the embodiment of the application comprises:

[0040] (1) A voice cloning and voiceprint recognition module: used for uniquely identifying and distinguishing the voices of different users through voiceprint recognition technology to support personalized voice output for multiple users; and used for generating personalized voice output highly similar to the voices of different users through voice cloning technology to improve the naturalness and personalization experience of voice interaction.

[0041] (2) A speech recognition (STT) module: integrating a speech recognition function in a hearing aid application program running on the intelligent device, which can capture human voices in the environment in real time and convert them into text.

[0042] (3) A speech synthesis (TTS) module: based on the text content generated by the speech recognition module, converting the text into clear voice signals through speech synthesis technology to provide personalized voice output.

[0043] (4) A user interaction module: providing an application interface of the intelligent device to allow the user to adjust the volume, select the voice mode and input the feedback.

[0044] The hearing aid application program further comprises:

[0045] (1) A voice mode selection function: used for switching between a quiet environment, a noisy environment and a face-to-face communication mode to adapt to different hearing needs and environmental changes;

[0046] (2) A voiceprint training function: allowing the user to add and manage multiple voiceprint models to realize personalized voice output for different speakers.

[0047] The STT module adopts an advanced speech recognition algorithm to effectively convert human voice frequency into accurate text, and the STT module performs high-precision speech-to-text processing through a cloud API in an online mode and uses a local speech recognition engine in an offline mode to ensure the continuity and reliability of the system.

[0048] The TTS module generates different voices or provides selectable standard voices based on the personalized needs of the user to meet a wide range of application scenarios and enhance the satisfaction of the user and the practicability of the system.

[0049] The user interaction module includes a user feedback interface to allow the user to provide feedback on the hearing experience through the application program to assist the system in optimizing parameters.

[0050] It also includes a wireless transmission module that adopts a Bluetooth low-latency transmission protocol to ensure the real-time and synchronization of voice signals.

[0051] Among them, the voice cloning and voiceprint recognition module includes an interface for collecting different user voice samples, and a deep learning algorithm module for training and identifying different user voiceprints;

[0052] The voice cloning module generates personalized voice output for each user based on the collected different user voice samples using real-time voice cloning technology;

[0053] The voiceprint recognition module is integrated with the voice cloning module to realize seamless collaboration of multi-user recognition and personalized voice output.

[0054] Therefore, the intelligent hearing aid system of the present application uses a smart phone or a wearable device (hereinafter referred to as a smart device) and a Bluetooth headset to realize personalized voice output by integrating voice cloning and voiceprint recognition, STT and TTS functions. When connected to the network, the system preferentially calls the cloud API to process the voice to improve the recognition accuracy and the naturalness of the voice; when there is no network, it automatically switches to the local engine to ensure the continuity of the system. Through voiceprint recognition, multiple users are supported, and voice cloning generates personalized voice, providing an economical, efficient, convenient and easy-to-use hearing aid solution to meet the hearing assistance needs of different users.

[0055] The intelligent hearing aid system of the present application is composed of the following components:

[0056] Smart device: responsible for capturing, preliminary processing and communication with the cloud server of environmental sound;

[0057] Bluetooth headset: with low-latency audio transmission capability, receiving the voice signal processed by the smart device, and presenting the voice output to the user;

[0058] Cloud server: provides API for advanced speech recognition, speech synthesis, voice cloning and voiceprint recognition functions, and supports efficient processing requirements of the system in a networked state.

[0059] The intelligent hearing aid system includes the following modules, and the functions of each module are as follows:

[0060] 1. Voice cloning and voiceprint recognition module;

[0061] (1) Voice data collection: when the user first uses the system, the speaker's voice samples can be collected and uploaded to the cloud for training of the voice cloning model and the voiceprint recognition model;

[0062] (2) Voiceprint algorithm: using a deep learning algorithm (such as Deep Speaker based on convolutional neural network) to extract the voiceprint features of the user, to ensure that the system can accurately distinguish different users;

[0063] (3) Voice cloning model: Utilizing advanced real-time voice cloning technology, it accurately captures and simulates the user's voice characteristics, converts the user's real-time voice input into natural and realistic synthesized voice, and achieves a highly personalized voice output effect.

[0064] 2. STT module;

[0065] (1) Online mode: Call cloud API (such as SenseVoice) to convert the captured environmental voice into text in real time, and use cloud computing resources and algorithms to improve recognition accuracy and processing speed;

[0066] (2) Offline mode: Automatically switches to the speech recognition engine built into the smart device (such as the end-to-end STT model), uses optimized local algorithms to realize basic speech-to-text functions, and ensures the continuity and reliability of the system in the absence of network environment.

[0067] 3. TTS module;

[0068] (1) Online mode: By calling cloud APIs (such as CosyVoice), the text content generated by the system is converted into a natural and fluent voice signal. Combined with voice cloning technology, personalized voice output is generated and played through Bluetooth headphones to improve the user experience.

[0069] (2) Offline mode: Use the TTS engine built into the smart device to convert the text content into speech and play it through Bluetooth headphones to ensure that the basic speech synthesis function is available in the absence of network environment.

[0070] 4. User interaction module;

[0071] (1) User Interface: Provides an intuitive and easy-to-use application interface on smart devices. Users can adjust the volume, select voice modes (such as quiet environment, noisy environment, face-to-face communication mode, etc.) and make personalized settings through the settings menu. Supports multilingual and multi-dialect speech recognition and synthesis functions to meet the needs of different users;

[0072] (2) Personalized settings: Users can automatically switch between different user voice clone models or standard voice modes through the voiceprint recognition module to adapt to the personalized needs of family members or multi-user environments. Users can select familiar relatives and friends for voice clone training to generate exclusive clone voices and distinguish different speakers;

[0073] (3) User feedback: Supports users to provide auditory experience feedback through the application, which helps the system to optimize parameters and make personalized adjustments.

[0074] The workflow of the intelligent hearing aid system is as follows:

[0075] 1. First-time use:

[0076] (1) User connects the smart device through the Bluetooth headset and starts the hearing aid application;

[0077] (2) The system prompts the user to collect voice samples, uploads the user's own voice data to the cloud for voice cloning and voiceprint recognition model training through voice input;

[0078] (3) Voice samples of relatives and friends can be collected for voice cloning and voiceprint recognition model training.

[0079] 2. Daily use:

[0080] (1) When the user is in different hearing environments, select the corresponding voice mode, the system automatically adjusts the voice processing parameters, and optimizes the voice output effect;

[0081] (2) The system automatically identifies the current speaker through voiceprint recognition. If it is a relative or friend who has been trained, the corresponding voice cloning model can be loaded to realize personalized voice output. Otherwise, select the standard voice output mode.

[0082] 3. Network state change:

[0083] (1) When there is a network connection, the system preferentially calls the cloud API to process voice recognition and voice synthesis functions to improve recognition accuracy and voice naturalness;

[0084] (2) When there is no network connection, the system automatically switches to the local voice recognition and voice synthesis engine to ensure the continuous availability of the hearing aid function.

[0085] In summary, the present application uses existing smart devices and Bluetooth headset technology, integrates voiceprint recognition and cloning, STT and TTS functions, and provides an economical, efficient, convenient and easy-to-use intelligent hearing aid system. The system captures environmental sound and voice recognition through the smart device, converts voice to text, and then realizes personalized clear voice playback through voice synthesis technology. In addition, the voice cloning and voiceprint recognition module can realize voiceprint recognition and personalized voice output for multiple users, further improving the applicability and user experience of the system. Optimized real-time processing algorithms and low-latency wireless transmission technology make the present application have significant innovation and market competitiveness in the field of hearing aid technology.

[0086] In addition, the present application is not only suitable for people with hearing impairment, but also widely used in cultural tourism and education fields, providing clear tour experience for tourists, and can integrate automatic translation engine to help students better understand different language content in the classroom, meeting the hearing aid needs of different user groups.

[0087] It should be noted that, in this invention, although the specification describes the embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. An intelligent hearing aid system, characterized in that, include: (1) Voice cloning and voiceprint recognition module: used to collect user voice samples and generate personalized voice models, and to distinguish different users through voiceprint recognition; (2) STT module: Integrated into the hearing aid application of smart devices, it is used to convert environmental speech into text in real time, and supports dual-mode processing of online calling cloud API and offline switching local engine; (3) TTS module: Based on the text content generated by the STT module, it synthesizes speech signals and supports online combination with speech cloning to generate personalized speech and offline use of the local engine; (4) User interaction module: Provides an application interface for smart devices, used for volume adjustment, voice mode selection and multi-user management; (5) Bluetooth headset: used to receive voice signals processed by smart devices and output audio.

2. The intelligent hearing aid system according to claim 1, characterized in that, In the STT module, the online mode calls the cloud API to achieve high-precision speech-to-text conversion, while the offline mode uses the local speech recognition engine built into the smart device.

3. The intelligent hearing aid system according to claim 1, characterized in that: In the TTS module, the online mode calls the cloud API to generate natural speech and integrates the speech clone model to output personalized speech, while the offline mode uses the local TTS engine for speech synthesis.

4. The intelligent hearing aid system according to claim 1, characterized in that, The user interaction module supports multi-user management, automatically switches between different users' voice clone models through voiceprint recognition, and provides voice mode selection functions for quiet environments, noisy environments, and face-to-face communication.

5. The intelligent hearing aid system according to claim 1, characterized in that, The voice cloning and voiceprint recognition module includes: Voice data collection interface, used to collect voice samples from users and their relatives and friends; The deep learning algorithm unit is used to train the voiceprint recognition model and generate personalized voice clone models.

6. The intelligent hearing aid system according to claim 5, characterized in that, The voiceprint recognition module uses a deep learning algorithm based on convolutional neural networks to extract user voiceprint features.

7. The intelligent hearing aid system according to claim 1, characterized in that, The user interaction module includes a user feedback interface for receiving auditory experience feedback to optimize system parameters.

8. The intelligent hearing aid system according to claim 1, characterized in that, The Bluetooth headset uses a low-latency transmission protocol to receive and play voice signals processed by smart devices in real time.

9. An intelligent hearing aid method based on the intelligent hearing aid system according to any one of claims 1-8, characterized in that, Includes the following steps: (1) Collect ambient sound through smart devices and confirm the identity of the current speaker through voiceprint recognition; (2) If the speaker is a trained user, load its speech clone model; otherwise, use the standard speech mode. (3) The speech is converted into text by the STT module and then synthesized into personalized speech by the TTS module and output to the Bluetooth headset.

10. The intelligent hearing aid method of the intelligent hearing aid system according to claim 9, characterized in that, When connected to the network, the cloud API is used to execute STT and TTS functions first; when there is no network, the local engine is automatically switched to process the data.

Images