Intelligent assistant method and system for chatting of elderly at home

Abstract

The invention discloses an intelligent chatting assistant method and system for home elderly people. The system comprises a voice interaction device, an intelligent chatting terminal, a cloud application and a mobile phone application. When the old person is at home alone, the old person can directly speak when wanting to chat with family members or wanting to ask things like the family members, and a mobile phone does not need to be used. The chat reply of the family member received by the old person is voice, and the voice is highly similar to the voice of the family member. As the large language model is used for assisting in generating the chat text, when family members work, the family members occasionally chat sentences with the elderly, normal work of the family members is not affected, and meanwhile it can be guaranteed that the chat text generated by the model is suitable for the elderly.

Description

Technical Field

[0001] The present invention relates to the application of artificial intelligence technology, in particular to speech recognition to text, TTS speech synthesis and language large model application. Background Art

[0002] my country has entered an aging society. With the continuous increase in the elderly population, there are more and more elderly people living at home. These elderly people, whether they live with their families or live independently, have a problem: how to communicate with their families when their families are not around. Although mobile phones are already very popular, it is not realistic for family members to frequently call the elderly when they are at work. Text communication via WeChat or text messages has limited effect, because the elderly often just want to talk to their families, or ask a question when they think of something. On the other hand, there are still many elderly people who are not proficient in operating smartphones or typing text on smartphones. For these elderly people, communicating with their families through text is not a friendly way.

[0003] After the release of the language big model represented by ChatGPT, many virtual digital human products based on the big model will be developed, including digital humans for home-based elderly people. Such digital humans cannot bring a sense of companionship to the elderly at home. In addition, due to security and adaptability, not all topics are suitable for direct replies after the language big model is generated. In addition, if the digital humans are deployed in the form of mobile apps, it is not only unfriendly to the elderly who cannot operate smartphones proficiently, but also increases the risk of being defrauded when the elderly download mobile apps.

[0004] If the elderly can chat with their family members at home as usual, without using their mobile phones, and can directly talk to whoever they want to chat with, ask questions, and receive voice replies from their family members, this will effectively reduce the loneliness of the elderly at home and improve their quality of life and happiness. Of course, there must be a premise that this kind of chat cannot affect the daily work of the elderly's children and other family members, and the generation of a large language model can provide technical support for this key premise. Summary of the invention

[0005] The problem to be solved by the present invention is:

[0006] Design a chat intelligent assistant system. For elderly people at home, this system can make their family members chat and talk about daily life just like they are around when they are not around, without using mobile phones. For the elderly’s children and other family members, this system allows them to use the language big model service to quickly generate reply text messages for chatting with the elderly, and can synthesize the reply text messages into voices similar to their own voices. At the same time, the chat reply text messages automatically generated by the big model are confirmed or modified by the elderly’s family members, so that the elderly’s family members can ensure that the chat between the elderly and the language big model is safe and suitable for the elderly.

[0007] In order to solve the above problems, the solution adopted by the present invention is as follows:

[0008] According to the present invention, a method for intelligent chat assistant for elderly people at home involves a voice interaction device, an intelligent chat terminal, a cloud application and a mobile phone application. The voice interaction device is connected to the intelligent chat terminal, the intelligent chat terminal is connected to the cloud application, the mobile phone application is installed on the mobile phone of the family of the elderly at home and connected to the cloud application, the intelligent chat terminal is installed in the house where the elderly live, the voice interaction device is installed in the house where the elderly live, and at least one is installed in one room within the range of the elderly's activities. The method comprises the following steps:

[0009] When the elderly need to communicate with their family, they can speak directly to the voice interaction device, and the voice interaction device will transmit the collected voice information to the chat intelligent terminal. As an optimization solution, the elderly do not need to use a wake-up word to wake up the voice interaction device before speaking (Note: the wake-up word is the basic way to use smart speakers at present. For example, when using the Xiaodu speaker, first call "Xiaodu Xiaodu", and this "Xiaodu Xiaodu" is the wake-up word).

[0010] Further: After receiving the voice information, the chat intelligent terminal converts the voice information into text information and extracts chat keywords in the text information: the name of the chat object and the chat content classification words. If the elderly person currently has a chat object with which he is communicating, and there is no new chat object name in the extracted chat keywords, the current chat object is determined as the object of this voice chat. If there is no chat object at present, the chat keywords are used to select the object for the elderly person to communicate with from the pre-registered family information of the elderly person as an optimization solution. When the communication object for this communication cannot be selected based on the chat keywords, the voice sample library of the elderly person pre-entered in the chat intelligent terminal will be used, and a voice similarity algorithm will be used to search in the sample library to select the communication object. The voice sample library of the elderly person includes two categories: the voice of the elderly person when calling the names of family members and the voice of the elderly person when asking about family matters.

[0011] Furthermore, after selecting the communication partner of the elderly person's voice communication, the voice information and the converted text information are sent to the mobile phone application of the selected communication partner through the cloud application.

[0012] Further: After the selected communication object receives the voice message and text message, the reply message to the elderly is input through the mobile application. The reply message can be text or voice, and the reply message is forwarded to the chat intelligent terminal through the cloud application. As an optimization solution, when the communication object determines that the language large model can be used to generate a reply, the large model interface service provided by the cloud application can be called in the mobile application to automatically generate a reply text message, which is used as a text message to reply to the elderly after confirmation or modification by the communication object.

[0013] Further: after the chat intelligent terminal receives the reply message, if it is a text message, it is converted into a voice message, and the reply language message or the converted voice message is broadcast through the voice interaction device, so that the elderly and the selected communication object establish a conversation relationship. As an optimization solution, when the intelligent chat terminal converts the received reply text message of the selected communication object into voice, it will use the pre-recorded voice sample of the communication object himself, so that the converted voice sounds like the voice of the selected communication object himself.

[0014] Further: After the elderly establish a conversation relationship with the selected communication object, if the elderly do not specify a new communication object in the subsequent voice messages, the voice message and the converted text message will be sent to the mobile phone application of the selected communication object through the cloud application in the aforementioned manner within the validity period of the conversation. If there is no conversation within the set time, the current conversation relationship will be automatically terminated. If during the conversation, the elderly specifies a new communication object in the voice message, the conversation relationship with the current communication object will also be automatically terminated.

[0015] The technical effects of the present invention are as follows:

[0016] 1. When the elderly are alone at home, they can chat with their family members as if they were right beside them. If they want to chat with their family members, they can just talk directly. If they have something to ask, they can just ask. They don’t need to use their mobile phones. The chat replies received by the elderly are also voices, and the voices are very similar to those of their family members.

[0017] 2. Due to the use of a large language model, family members can occasionally chat with the elderly while they are at work, which is very convenient and quick and will not affect his (her) normal work.

[0018] 3. The conversation information generated by the language model can be confirmed or modified by the elderly’s family members. Compared with a simple digital person who is just chatting with them, it can ensure that the conversation content is safe and suitable for the elderly. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a schematic diagram of the overall structure of the system of the present invention. DETAILED DESCRIPTION

[0020] The following is combined with Figure 1 The present invention is described in further detail:

[0021] like Figure 1 As shown, according to a smart chat assistant system for elderly people at home, the system includes a voice interaction device 100, an intelligent chat terminal 200, a cloud application 300 and a mobile application 400. The voice interaction device 100 is connected to the chat smart terminal 200, the chat smart terminal 200 is connected to the cloud application 300, and the mobile application 400 is installed on the mobile phone of the family of the elderly at home and connected to the cloud application 300.

[0022] Only one chat intelligent terminal 200 is installed in the house where the elderly live, usually in the living room. The voice interaction device 100 is installed in the rooms within the elderly's activity range, one in each room, usually including: living room, elderly bedroom, kitchen. If the elderly need to take care of children, the room where the children sleep also needs to be installed.

[0023] Design and implementation of voice interaction device 100:

[0024] The main function of the voice interaction device 100 is to collect the voice of the elderly, play the voice replied by the elderly's chat partner or the text-converted voice of the exchange, and the connection method with the intelligent chat terminal 200 can be wireless or wired. As an optimization solution, the voice interaction device also needs to have the function of detecting human movement and the approximate distance from the voice interaction device, so as to facilitate the intelligent chat terminal 200 to be able to select the voice interaction device closest to the elderly to play the received reply voice.

[0025] In terms of hardware configuration, in the present invention, the hardware of the voice interaction device 100 uses an ordinary embedded Linux motherboard, is equipped with a voice chip, a speaker, a human motion sensor and a distance sensor, and is connected to the chat intelligent terminal 200 using Wifi.

[0026] In terms of software, all technologies are mature and it is more appropriate to use C++ for development. In terms of functions, in addition to the above functions, it is also necessary to implement matching and binding functions with the chat intelligent terminal 200. In addition, in order to ensure security, communication with the chat intelligent terminal 200 requires signature authentication.

[0027] Design and implementation of chat intelligent terminal 200:

[0028] The functions of the chat intelligent terminal are mainly divided into the following two categories.

[0029] a. Initial configuration functions, including:

[0030] a1. Matching and setting up the voice interaction device, establishing a secure communication relationship with the voice interaction device 100, and setting up the room where the voice interaction device is installed.

[0031] a2. Input the elderly’s information and voice samples. The voice samples mainly include two categories: elderly calling family members and elderly asking family members about things. Each voice sample needs to specify the family member to be contacted, which is used to determine the chat partner through the voice similarity algorithm.

[0032] a3. Enter the family information of the elderly. The main information includes family members' names, the names the elderly often use to address them, mobile phone numbers and chat content keywords. Family members' names, names and chat keywords are used to select chat partners. At the same time, family voice samples should also be entered. Voice samples include the sound of family members calling the elderly and the minimum samples required for synthesized speech.

[0033] b. Chat information processing functions, including:

[0034] b1. Convert the received voice messages from the elderly into text messages.

[0035] b2. Determine the communication object of the received voice message of the elderly: Extract chat keywords from the text converted from the elderly voice message, that is, the name of the chat object and the keywords for the classification of the chat content. The logic for confirming the chat object is: if the name of the chat object is extracted, it can be directly determined as the chat object, otherwise check whether the elderly is currently chatting with someone, if so, determine it as the chat object, if there is no object currently chatting, use the extracted chat content classification keywords to select the communication object from the pre-registered family information. If the chat object cannot be confirmed using the converted text information, use the voice similarity to find and confirm the communication object based on the voice sample library pre-entered by the elderly. If the chat object cannot be determined, it means that the elderly are not looking for family members to chat, and generally no response is required.

[0036] b3. Send the elderly’s voice message and the converted text message to the selected chat partner.

[0037] b4. Processing the reply information received from the chat object: If it is a text message, first use the pre-recorded voice sample of the chat object to convert the text into a voice similar to the voice of the chat object, and play the received voice or the converted voice through the voice interaction device 100 closest to the elderly.

[0038] b5. Maintaining the conversation relationship between the elderly and the chat object: When the elderly and the current chat communication object have no chat information exchange within a preset time, the current chat session is closed. If the elderly specifies a new chat session object in the voice, the current chat session is closed.

[0039] Hardware configuration of the chat intelligent terminal 100: It is recommended to use a high-performance Android motherboard or embedded Linux motherboard, and configure edge AI hardware, such as NPU, FPGA, etc., to support the deployment of AI models such as voice recognition, chat keyword extraction, and voice similarity comparison locally on the chat intelligent terminal. Engineers in this field understand that a voice interaction device 100 can be built into the chat intelligent terminal.

[0040] Chat smart terminal software design: mainly to realize the functions of the above chat smart terminal. It is recommended to use C++ language to develop non-UI programs. The communication between the chat smart terminal and the cloud application requires signature authentication. Engineers in this field understand that if the chat smart terminal is designed without a screen, the UI part of its software functions that interact with people, such as the UI for entering the information of the elderly and family members, can be implemented on the mobile phone application 400. The mobile phone application 400 can establish a communication connection with the smart chat terminal 200 through Wifi.

[0041] Design and implementation of cloud application 300:

[0042] The main functions of the cloud application 300 are mainly provided in the form of interface services, including: user registration, login, family information maintenance, smart chat terminal registration, connection, binding with the family, forwarding messages between the mobile application 400 and the chat smart terminal 200, and language big model service interface services. In the cloud application, except for the language big model service, all are mature technologies. The language big model needs to purchase services provided by language big model suppliers such as Baidu.

[0043] Design and implementation of mobile application 400:

[0044] The main functions of the mobile application include:

[0045] User management: registration, login, modification of personal information and passwords, etc.

[0046] Family information management, including family information creation, family member settings, family member invitation code generation, chat function terminal binding, etc.

[0047] Chat with the elderly: receive chat messages, call the big model service to generate replies, enter or modify reply messages, send reply messages, etc.

[0048] Main workflow:

[0049] The intelligent chat assistant system for elderly people at home of the present invention needs to be initialized before use, that is, the system initialization process includes:

[0050] c1. Enter the elderly person's information in the intelligent chat terminal 200 and collect the elderly person's voice sample; enter the elderly person's family information and collect family voice samples.

[0051] c2. The elderly family downloads and installs the mobile application 400, completes user registration, and one of the family members creates family information, sets family members, generates a family member invitation code (other family members use the invitation code to join the family), and binds the smart chat terminal 200 to the family.

[0052] Daily chat process:

[0053] P1. The elderly person is near the voice interaction device and speaks directly to the chat partner (if a wake-up word is used every time to wake up the voice interaction device, it will not feel like chatting).

[0054] P2. The voice interaction device collects the elderly person’s voice and sends it to the chat intelligent terminal.

[0055] P3. The chat intelligent terminal converts the received elderly voice into text messages.

[0056] P3. The intelligent chat terminal determines the object of this voice chat.

[0057] P4. Send the elderly person’s voice information and converted text information to the chat partner’s mobile phone application 400 through the cloud application 300.

[0058] P5. After receiving the chat message from the elderly, the chat partner can directly input the text message or voice message of the chat reply, or call the large model service provided by the cloud application to generate a chat reply text message, which can be used as a reply message after modification or confirmation.

[0059] P6. The mobile application sends the chat reply information to the chat intelligent terminal. After receiving it, if it is a text message, the chat intelligent terminal converts the text into a voice similar to that of the chat object.

[0060] P7. The chat intelligent terminal selects the voice interaction device that is currently closest to the elderly and plays the received chat partner's voice or the converted voice.

Claims

1. A smart chat assistant method for elderly people at home, It is characterized in that The method includes a voice interaction device, a chat intelligent terminal, a cloud application and a mobile phone application, wherein the voice interaction device is connected to the chat intelligent terminal, the chat intelligent terminal is connected to the cloud application, the mobile phone application is installed on the mobile phone of the family of the elderly at home and connected to the cloud application, the chat intelligent terminal is installed in the house where the elderly live, the voice interaction device is installed in the house where the elderly live, and at least one voice interaction device is installed in one room within the range of the elderly's activities. The method includes the following steps: S1: When the elderly need to communicate with their family, they speak directly to the voice interaction device, and the voice interaction device transmits the collected voice information to the chat intelligent terminal. As an optimization solution, the elderly do not need to use a wake-up word to wake up the voice interaction device before speaking; S2: After receiving the voice information, the chat intelligent terminal converts the voice information into text information, extracts chat keywords in the text information: the name of the chat object and the chat content classification words, if the elderly currently has a chat object, and there is no new chat object name in the extracted chat keywords, the current chat object is determined as the current voice chat object, if there is no chat object currently, use the chat keywords to select the elderly communication object from the pre-registered family information of the elderly as an optimization solution, when the communication object for this communication cannot be selected according to the chat keywords, the elderly's voice sample library pre-recorded in the chat intelligent terminal will be used, and the voice similarity algorithm will be used to search in the sample library to select the communication object, the elderly's voice sample library includes two categories of voice when the elderly calls the name of the family and the voice when asking about the family; S3: After selecting the communication partner of the elderly person's voice communication, the voice information and the converted text information are sent to the mobile phone application of the selected communication partner through the cloud application; S4: After the selected communication partner receives the voice message and text message, the reply message to the elderly is input through the mobile application. The reply message can be text or voice, and the reply message is forwarded to the chat intelligent terminal through the cloud application. As an optimization solution, when the communication partner determines that the language big model can be used to generate a reply, the big model interface service provided by the cloud application can be called in the mobile application to automatically generate a reply text message, which is used as a text message to reply the elderly after confirmation or modification by the communication partner; S5: After the chat intelligent terminal receives the reply message, if it is a text message, it converts it into a voice message, and broadcasts the reply language message or the converted voice message through the voice interaction device. At this point, the elderly and the selected communication object establish a conversation relationship. As an optimization solution, when the intelligent chat terminal converts the received reply text message from the selected communication object into voice, it will use the pre-recorded voice sample of the communication object himself, so that the converted voice sounds like the voice of the selected communication object himself; S6: After the elderly establish a conversation relationship with the selected communication object, if the elderly do not specify a new communication object in the subsequent voice messages, the voice message and the converted text message will be sent to the mobile phone application of the selected communication object through the cloud application in the aforementioned manner within the validity period of the conversation. If there is no conversation within the set time, the current conversation relationship will be automatically terminated. If during the conversation, the elderly specifies a new communication object in the voice message, the conversation relationship with the current communication object will also be automatically terminated.

2. A method for providing an intelligent assistant for elderly people to chat at home as claimed in claim 1, It is characterized in that When the elderly need to use the chat smart assistant, they do not need to use a specific wake-up word to wake up the voice interaction device.

3. A method for providing an intelligent assistant for elderly people to chat at home as claimed in claim 1, It is characterized in that When the chat intelligent terminal cannot select a communication partner through the text converted from the elderly person's voice, it will use the elderly person's voice sample library pre-recorded in the chat intelligent terminal, use a voice similarity algorithm to search in the sample library, and select a communication partner. The elderly person's voice sample library includes two categories: the elderly person's voice when calling family members' names and the elderly person's voice when asking family members about things.

4. A method for providing an intelligent assistant for elderly people to chat at home as claimed in claim 1, It is characterized in that When the chat terminal converts the text message received from the communication partner in reply to the elderly into voice, it uses the pre-recorded voice sample of the communication partner himself to make the converted voice sound like the voice of the communication partner himself.

5. A method for providing an intelligent assistant for elderly people to chat at home as claimed in claim 1, It is characterized in that When the elderly person's communication partner confirms that the language big model service can be used based on the received voice or text information, the big model interface service provided by the cloud application can be called in the mobile application to automatically generate a reply text, which is used as a text message to reply to the elderly person after manual confirmation or modification, to ensure that the text output by the language big model is safe and suitable for the elderly person to chat.

Images