A voice assistance mobile phone system for the deaf and dumb

Abstract

The present invention relates to a voice-assisted mobile phone system for deaf-mutes, comprising: a display module configured to display text information; a voice input module configured to input voice information of the user's daily communication language; an intelligent voice processing module configured to extract the voice information of the user's daily communication language and establish a database with a one-to-one correspondence with the text information; a voice recognition module configured to, through the input of the user's external voice information, recognize the content of the voice information, obtain the corresponding text information from the database, and output the text information to the display module; a voice playback module configured to be able to convert the text information displayed by the display module into voice information for playback. The present invention integrates a deaf-mute auxiliary communication system into a mobile phone. In daily communication, users can carry the mobile phone to assist deaf-mutes in completing daily communication, and this system helps deaf-mutes build confidence in communication.

Description

Technical Field

[0001] The present invention relates to the technical fields of speech recognition and mobile phones, and particularly relates to a speech-assisted mobile phone system for deaf-mutes. Background Art

[0002] At present, with the continuous development of speech recognition technology, more and more devices (such as household appliances like mobile phones, televisions, air conditioners, etc.) can perform corresponding functions through voice control. For example, when a controlled device detects a voice control instruction, it can perform corresponding operations according to the detected control instruction. Therefore, voice interaction brings a lot of convenience to users' daily lives.

[0003] However, for deaf-mutes, there are obstacles in communication in daily life and learning environments. Compared with healthy people, it poses new challenges to voice interaction. Summary of the Invention

[0004] The present invention provides a speech-assisted mobile phone system for deaf-mutes, aiming to solve the communication barriers of deaf-mutes in daily life and learning environments, making the communication between people smoother and enhancing the confidence of deaf-mutes in life.

[0005] Specifically, the present invention provides a speech-assisted mobile phone system for deaf-mutes, including:

[0006] A display module, configured to display text information;

[0007] A voice input module, configured to input the voice information of the communication language in the user's daily life;

[0008] An intelligent voice processing module, configured to extract the voice information of the user's daily communication language and establish a database with a one-to-one correspondence with the text information;

[0009] A speech recognition module, configured to recognize the content of the voice information through the input of the user's external voice information, obtain the corresponding text information from the database, and output the text information to the display module;

[0010] A voice playback module, configured to convert the text information displayed by the display module into voice information for playback.

[0011] As a preferred technical solution, the intelligent voice processing module is configured to regularly review the one-to-one correspondence of the text information in the database and correct the corresponding incorrect relationships in the database.

[0012] As a preferred technical solution, the database is uploaded to a cloud server for backup.

[0013] As a preferred technical solution, the intelligent voice processing module is set to regularly count the usage frequency of the correspondence between voice information and text information, and update the database according to the usage frequency.

[0014] As a preferred technical solution, the voice recognition module recognizes the voice characteristics according to the input of external voice information, and the voice characteristics include Mandarin, dialect or accent.

[0015] As a preferred technical solution, the recognition module determines a voice matching model corresponding to the voice characteristics according to the voice characteristics, and uses the voice matching model corresponding to the voice characteristics to recognize the input voice information.

[0016] As a preferred technical solution, the intelligent voice processing module extracts the voice information of the user's daily communication language, determines a voice matching model corresponding to the voice characteristics, and uses the voice matching model corresponding to the voice characteristics to recognize the input voice information, outputs the voice recognition result, and then establishes a database with a one-to-one correspondence with the text information.

[0017] As a preferred technical solution, the voice playback module can convert the text information displayed by the display module into voice information corresponding to the voice characteristics according to the voice characteristics for playback.

[0018] As a preferred technical solution, the voice playback module is provided with a reminder function and can send a text reminder message, and the reminder message is pop-up displayed beside the text information displayed by the display module; the reminder message can be to remind the user that the recognition result of this conversation is not very accurate and whether to re-recognize the voice.

[0019] As a preferred technical solution, the voice recognition module further includes a voice conversion and noise reduction module for removing noise and performing voice analysis.

[0020] The present invention has achieved the following technical effects compared with the prior art: The technical solution of the present invention integrates a deaf-mute auxiliary communication system in a mobile phone, including a voice recognition module, an intelligent voice processing module, a voice playback module, etc. In daily communication, the user can carry the mobile phone, record the voice of the conversation partner, recognize the conversation content through the voice recognition module, correspond to the answer information through the intelligent voice processing module, and then convert the answered information into voice through the voice playback module. In this way, a daily communication of a deaf-mute is completed. The system is compatible with the mobile phone system and is convenient for users to use daily. Moreover, the system makes the communication between people smoother and enhances the confidence of deaf-mutes in life.

[0021] Other features and advantages of the present application will be described in the following specification, and part of them will become obvious from the specification or be understood by implementing the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 Block diagram of a voice-assisted mobile phone system for deaf-mutes proposed in an embodiment of the present invention;

[0023] Figure 2 Block diagram of a voice-assisted mobile phone system for deaf-mutes proposed in an embodiment of the present invention.

[0024] Explanation of reference numerals in the drawings:

[0025] Display module 01; Voice input module 02; Intelligent voice processing module 03; Database 04; Voice recognition module 05; Voice playback module 06; Voice conversion and noise reduction module 07. Detailed implementation manners

[0026] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments of the present invention and the corresponding drawings. In the description of the present invention, it should be noted that the term "or" is generally used in the sense of including "and / or", unless otherwise clearly specified in the context.

[0027] It should be understood that the various steps recorded in the method embodiments of the present application can be executed in different orders and / or executed in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present application is not limited in this regard.

[0028] As used herein, the term "including" and its variants are open-ended, that is, "including but not limited to". The term "based on" is "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0029] It should be noted that the modifications of "one" and "multiple" mentioned in the present application are illustrative rather than restrictive. Those skilled in the art should understand that unless clearly specified otherwise in the context, it should be understood as "one or more". "Multiple" should be understood as two or more.

[0030] Embodiment

[0031] As Figure 1 shown, a voice-assisted mobile phone system for deaf-mutes proposed in this embodiment, the system includes:

[0032] Display module 01, configured to display text information;

[0033] Voice input module 02, configured to input voice information of the user's daily communication language;

[0034] The intelligent voice processing module 03 is set to extract the voice information of the user's daily life communication language and establish a database 04 with a one-to-one correspondence with the text information;

[0035] The voice recognition module 05 is set to input the external voice information of the user, recognize the content of the voice information, obtain the corresponding text information from the database 04, and output the text information to the display module 01;

[0036] The voice playback module 06 is set to be able to convert the text information displayed by the display module 01 into voice information for playback.

[0037] Regarding the database 04, in this mobile phone system, a healthy person can carry the mobile phone and record the voices (including questions and answers) that often appear in the daily life of the surrounding environment of the living area. After collecting the voices, the healthy person can then type, etc., to establish a one-to-one correspondence between the voices and the text, and this mobile phone system will record this correspondence. For example, in daily shopping, a deaf-mute user can hold the mobile phone and input the voice content of the conversation partner through the voice input module 02. Common questions are, "How much is this?", "Can it be cheaper?", etc. And the answers can often be numbers, or "OK", "I confirm to buy it", etc.

[0038] Preferably, the intelligent voice processing module 03 is set to regularly review the one-to-one correspondence established with the text information in the database 04 and correct the incorrect correspondence in the database 04. Since in daily communication, there will be voice information input such as accents and dialects, the voice recognition module 05 may have incorrect recognition, resulting in incorrect correspondence. Therefore, the intelligent voice processing module 03 needs to be set to regularly correct and review. For the correspondence that has always been incorrect, a certain correspondence between text and voice can also be forcibly established to complete the combination and re-establishment of the database 04. By reviewing and correcting the database 04, the correspondence between text and voice in the database 04 can be ensured, so as to more accurately help deaf-mute users recognize the conversation content and answer accurate answer information at the same time, making the communication smoother.

[0039] Preferably, the database 04 is uploaded to the cloud server for backup. Specifically, the established database 04 can be stored in the device and uploaded to the cloud server for backup. For example: the established database 04 can be stored on the mobile phone and uploaded to the cloud server through the mobile phone, which is convenient for calling the database 04 and can also avoid the loss of the database 04 after changing the device.

[0040] Preferably, the intelligent voice processing module 03 is set to regularly count the usage frequency of the correspondence between voice information and text information, and update the database 04 according to the usage frequency. By regularly counting the usage frequency of the correspondence to update the database 04, the priority is set to improve the response speed of the correspondence, output session information faster, and improve communication efficiency.

[0041] Preferably, the speech recognition module 05 inputs according to the external speech information and recognizes the speech characteristics. The speech characteristics include Mandarin, dialect or accent. Further, the recognition module determines a speech matching model corresponding to the speech characteristics according to the speech characteristics, and uses the speech matching model corresponding to the speech characteristics to recognize the input speech information. The corresponding speech matching model can be determined from multiple pre-established models, and then the corresponding speech matching model is used for speech recognition. For example, if the speech characteristic is Sichuan-Chongqing dialect, the speech matching model corresponding to the Sichuan-Chongqing dialect can be used to perform speech recognition on the input speech. The above describes determining the speech characteristics first and then determining the speech matching model. Optionally, the speech characteristics and the speech matching model can be determined synchronously.

[0042] Specifically, feature extraction can be performed on the voice information to obtain feature information. Those skilled in the art can understand that the feature extraction can be fundamental frequency feature extraction, spectral feature extraction, energy feature extraction, etc. According to the feature information and multiple pre-established speech matching models, speech recognition is performed to obtain the confidence value corresponding to each model. The multiple speech matching models can be all the pre-established models, or multiple models selected from all the pre-established models. Those skilled in the art should be able to understand that daily life communication is mostly in communities, and the language patterns are relatively fixed, making it easy to match the correct session information. However, with the convenience of life, the communication range of deaf-mute people is getting wider and wider, and the objects of daily life communication will come from different regions. Therefore, it is necessary to match multiple models to recognize the speech content. For example, the speech matching models can be the matching models corresponding to Northeast dialect, Northern Shaanxi dialect, Minnan dialect, and Sichuan-Chongqing dialect respectively. After the speech is input, each speech matching model performs speech recognition and generates a corresponding confidence value. According to the confidence value, the optimal speech matching model is obtained, and the speech characteristics and speech recognition results corresponding to the optimal speech matching model are obtained. During the establishment process of the database 04, through continuous iteration and domestication, the correspondence in the database 04 becomes more accurate.

[0043] In addition, it can be understood that if no voice feature and voice matching model consistent with the feature information can be found, the most similar voice matching model can be found according to the confidence value, and the most similar voice matching model can be used for speech recognition. After speech recognition, the corresponding response text or picture is obtained from database 04. When playing the voice, the corresponding text can be automatically played according to the voice feature obtained by the voice matching model.

[0044] Preferably, the intelligent voice processing module 03 extracts the voice information of the user's daily communication language, determines the voice matching model corresponding to the voice feature, and uses the voice matching model corresponding to the voice feature to recognize the input voice information, outputs the speech recognition result, and then establishes a one-to-one correspondence relationship with the text information in database 04.

[0045] As Figure 2 shown, in some preferred embodiments, the speech recognition module 05 further includes a voice conversion and noise reduction module 07 for removing noise and performing speech analysis. The input voice is preprocessed, and the preprocessing is noise reduction processing to facilitate more accurate recognition of the speech content.

[0046] Preferably, the voice playback module 06 can convert the text information displayed by the display module 01 into voice information corresponding to the voice feature for playback according to the voice feature.

[0047] More preferably, the voice playback module 06 is provided with a reminder function and can send out a text reminder message, and the reminder message is pop-up displayed beside the text information displayed by the display module 01; the reminder message can be to remind the user that the recognition result of this conversation is not very accurate and whether to re-recognize the voice. The speech recognition module 05 sets a confidence value according to the matching degree and matching time between the voice feature and the voice matching model during the speech recognition process. The higher the confidence value, the higher the matching degree, and the more correct the corresponding text response information will be. On the contrary, if the confidence value is very low, it means that the corresponding text response information does not match the input voice content very well. At this time, the speech recognition module 05 can send information to the voice playback module 06 and the display module 01, and display a reminder message beside the text before or synchronously during the voice playback to remind the user whether to re-enter the voice content. If the user wants to re-enter, the voice input module 02 can be reused to re-enter the voice content and start a new conversation.

[0048] Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Claims

1. A voice assistance mobile phone system for deaf-mutes, characterized in that, Including: A display module, configured to display text information; A voice input module, configured to input voice information of the user's daily communication language; An intelligent voice processing module, configured to extract the voice information of the user's daily communication language and establish a database with a one-to-one correspondence with the text information; the intelligent voice processing module extracts the voice information of the user's daily communication language, determines a voice matching model corresponding to the voice characteristics, and uses the voice matching model corresponding to the voice characteristics to identify the input voice information, outputs a voice recognition result, and then establishes a database with a one-to-one correspondence with the text information; The intelligent voice processing module is configured to periodically review the one-to-one correspondence of the text information in the database and correct the incorrect correspondence in the database; A voice recognition module, configured to identify the content of the voice information through the input of the user's external voice information, obtain the text information from the database correspondingly, and output the text information to the display module; the voice recognition module identifies the voice characteristics according to the input of the external voice information, and the voice characteristics include Mandarin, dialect or accent; the recognition module determines a voice matching model corresponding to the voice characteristics according to the voice characteristics, and uses the voice matching model corresponding to the voice characteristics to identify the input voice information; after the voice is input, each voice matching model performs voice recognition on the voice, generates a corresponding confidence value, obtains the optimal voice matching model according to the confidence value, and obtains the voice characteristics and voice recognition result corresponding to the optimal voice matching model; A voice playback module, configured to be able to convert the text information displayed by the display module into voice information for playback; the voice playback module can convert the text information displayed by the display module into voice information with corresponding voice characteristics for playback according to the voice characteristics.

2. The voice assistance mobile phone system for deaf-mutes according to claim 1, characterized in that, The database is uploaded to the cloud server for backup.

3. The voice assistance mobile phone system for deaf-mutes according to claim 1, characterized in that, The intelligent voice processing module is configured to periodically count the usage frequency of the correspondence between the voice information and the text information and update the database according to the usage frequency.

4. The voice assistance mobile phone system for deaf-mutes according to any one of claims 1-3, characterized in that, The voice playback module is provided with a reminder function and can send a text reminder message, and the reminder message is pop-up displayed beside the text information displayed by the display module; the reminder message is to remind the user that the recognition result of this conversation is not accurate enough and whether to re-recognize the voice.

5. The voice assistance mobile phone system for deaf-mutes according to any one of claims 1-3, characterized in that, The voice recognition module further includes a voice conversion and noise reduction module for removing noise and performing voice analysis.

Images