System

Abstract

To provide systems.SOLUTION: A system includes the means for: providing an utterance system for interacting with users; adjusting responses of the utterance system on the basis of personal information on a family of the user, and a habit of a conversation; collecting conversation data between the user and the utterance system; analyzing the collected conversation data, and evaluating an emotional state of the user; and notifying a summary of the collected conversation data and an evaluation result of the emotional state of the user.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] In Japan, an ultra-aging society, the number of elderly people living alone is increasing year by year, which is becoming a major social problem that increases the risk of dementia and contributes to lonely deaths. However, it is difficult for families to constantly interact with the elderly, and there is a need for methods to reduce the sense of loneliness and the risk of dementia in the elderly. [Means for solving the problem]

[0005] This invention provides a speech ChatBot for communicating with elderly people who live alone. This ChatBot fine-tunes the family member's personal information and conversational habits, and generates and speaks conversations on their behalf. It also summarizes the conversation history and content with the elderly person and reports it to the family. Furthermore, it reproduces the elderly person's voice using voice reproduction AI. This makes it possible to reduce the elderly person's sense of loneliness and the risk of dementia. [Brief description of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a process flow of the data processing system in the first embodiment of the first form example. [Figure 12] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the application example 1 of the first embodiment. [Figure 13] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the second embodiment of the second embodiment. [Figure 14] FIG. 13 is a sequence diagram showing a process flow of the data processing system in application example 2 of embodiment 2. [Figure 15] FIG. 13 is a sequence diagram showing a process flow of a data processing system in a third embodiment of the present invention. [Figure 16] FIG. 13 is a sequence diagram showing a process flow of the data processing system in the third application example of the third embodiment. [Figure 17] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when combined with an emotion engine. [Figure 19] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system according to the embodiment 2 of the second embodiment in the case where an emotion engine is combined. [Figure 20] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when combined with an emotion engine. [Figure 21] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in the embodiment 3 of the form example 3 in the case where an emotion engine is combined. [Figure 22] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when combined with an emotion engine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0023] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] As an example embodiment of the present invention, a speech ChatBot for interacting with elderly people who live alone is provided. This ChatBot is fine-tuned using a machine learning algorithm for learning personal information and conversational habits of family members. Specifically, the family's conversation data is used as learning data, its features are extracted, and conversations are generated and spoken based on that. In addition, the conversation history and content with the elderly are summarized using natural language processing technology and reported to the family. Furthermore, a voice reproduction AI learns the elderly's voice data, extracts its features, and reproduces the elderly's voice. This allows the elderly to feel as if they are interacting with their family. "Example 2"

[0029] In another embodiment of the present invention, the speech ChatBot has a function to evaluate the dementia risk of elderly people. Specifically, the ChatBot detects signs of dementia from the content and frequency of conversations with elderly people, conversation patterns, etc. For example, if an elderly person shows symptoms such as repeating the same things, having difficulty in everyday conversations, or having difficulty finding words, the ChatBot evaluates this as a risk of dementia and reports it to the family.

[0030] "Example 3"

[0031] In another embodiment of the present invention, the speech ChatBot has a function to evaluate the elderly person's sense of loneliness. Specifically, the sense of loneliness is detected from the content and frequency of conversations with the elderly person, emotional expressions, etc. For example, if the elderly person frequently uses words expressing loneliness, does not express emotions much, or talks less frequently, the ChatBot evaluates these as indicators of loneliness and reports them to the family.

[0032] The process flow of each embodiment will be described below.

[0033] "Example 1"

[0034] Step 1: Speech ChatBot begins fine-tuning using machine learning algorithms to learn your family's personal information and conversational habits.

[0035] Step 2: Use family conversation data as learning data and extract its features.

[0036] Step 3: Generate and speak a conversation based on the extracted features.

[0037] Step 4: Summarize the conversation history and content with the elderly person using natural language processing technology.

[0038] Step 5: Report the summary to your family.

[0039] Step 6: The voice reproduction AI learns the voice data of the elderly and extracts its features.

[0040] Step 7: Reproduce the elderly person's voice using the extracted features.

[0041] "Example 2"

[0042] Step 1: The speech ChatBot will begin assessing the dementia risk of the elderly. Step 2: Detect signs of dementia from the content, frequency, and conversation patterns of conversations with the elderly.

[0043] Step 3: Report the dementia risk assessment results to family members.

[0044] "Example 3"

[0045] Step 1: The speech ChatBot starts the function of assessing the elderly person's loneliness.

[0046] Step 2: Detect loneliness based on the content and frequency of conversations with elderly people, as well as emotional expressions.

[0047] Step 3: Report the results of the assessment as an index of loneliness to family members.

[0048] Example 1

[0049] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0050] There is a need for a system that allows elderly people living alone to continue to interact with their families without feeling lonely. It is also important to evaluate the elderly's dementia risk and loneliness and provide appropriate information to their families. However, the current system is not able to fully resolve these issues.

[0051] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0052] In this invention, the server includes a means for collecting family conversation data, a means for pre-processing the collected data, a means for fine-tuning a machine learning model, a means for generating a conversation with the elderly person, a means for reproducing the voice of the elderly person, a means for summarizing a conversation history with the elderly person, and a means for reporting the summarized conversation history to the family member. This enables the elderly person living alone to have a natural conversation with the family member, reduce loneliness, and provide appropriate information to the family member through an evaluation of dementia risk and loneliness.

[0053] "Family conversation data" refers to text messages and voice data exchanged between family members and elderly people.

[0054] "Preprocessing" refers to the process of removing unnecessary information from collected data and converting it into a form suitable for machine learning models.

[0055] "Fine-tuning a machine learning model" refers to a technique for improving the performance of an existing machine learning model by performing additional training using a specific dataset.

[0056] "Conversation generation" refers to the process of automatically generating natural-sounding conversations using machine learning models.

[0057] "Voice reproduction" refers to the technology of reproducing a specific person's voice based on collected voice data.

[0058] "Conversation history summary" refers to the process of summarizing conversations with elderly people and extracting key points.

[0059] "Reporting to family" refers to the act of notifying family members of the summarized conversation history.

[0060] This invention relates to a speech ChatBot system for interacting with elderly people who live alone. This system collects family conversation data, generates natural conversations with elderly people using machine learning models, and reports the conversation history to the family.

[0061] Hardware and Software Used

[0062] The server uses the following hardware and software:

[0063] Hardware: A server machine equipped with a high-performance CPU and GPU

[0064] Software: Python, TENSORFLOW, PyTorch, NLTK, audio processing libraries

[0065] The terminal is a device used by the elderly and uses the following hardware and software:

[0066] Hardware: Microphone, speaker, tablet or smartphone

[0067] Software: Voice recognition software, voice synthesis software

[0068] Data processing and calculation

[0069] The server collects conversation data of family members. The collected data includes text messages and voice data. For example, messages sent by family members using smartphones or computers and recorded data of phone calls are stored in cloud storage.

[0070] Next, the server preprocesses the collected data. Specifically, it cleans the text data (removes unnecessary spaces and special characters) and removes noise from the audio data. It uses the Python library NLTK to tokenize the text data and a speech processing library to remove noise from the audio data.

[0071] The server then uses the preprocessed data to fine-tune the machine learning model. The models used are, for example, OpenAI's (registered trademark) GPT-3 (registered trademark) and Google's (registered trademark) BERT. Family conversation data is input into the model, and the model is trained to learn the characteristics of family conversations.

[0072] ChatBot uses the fine-tuned model to generate conversations with the elderly. For example, if an elderly person asks, "What shall we do today?", ChatBot responds, "The weather is nice today, so how about going for a walk?"

[0073] In addition, the server learns the voice data of the elderly and extracts its features, which allows the ChatBot to reproduce the voice of the elderly. Using voice reproduction AI, it learns the features of the elderly's voice and speaks the generated text in the elderly's voice.

[0074] The server summarizes the conversation history with the elderly person using natural language processing technology, extracts important points and topics of the conversation, and generates a concise summary. The server summarizes the conversation content using tools such as the BERT summarizer.

[0075] Finally, the server reports the summarized conversation history to the family. It notifies the family of the elderly person's situation and the contents of the conversation via email or a dedicated app. The summarized text is sent as the body of the email.

[0076] Examples

[0077] For example, when a ChatBot wants to have a conversation with an elderly person about "today's weather," it inputs the following prompt sentence into the generative AI model.

[0078] Example prompt:

[0079] "Good morning. How's the weather today?"

[0080] Based on this prompt, ChatBot will generate the following conversation:

[0081] An example of the generated conversation:

[0082] ChatBot: Good morning. It's a sunny day today. How about going for a walk?

[0083] Elderly person: Yes, I think I'll go outside for the first time in a while.

[0084] In this way, ChatBot can have natural conversations with the elderly and deepen their interactions. The conversation history is summarized by the server and reports to the family, "It's sunny today, so the elderly are going to go for a walk."

[0085] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0086] Step 1:

[0087] Data collection

[0088] The server collects family conversation data. As input, it receives text messages and recorded calls sent by family members using their smartphones and computers. These data are stored in cloud storage. As output, it obtains the collected text and audio data.

[0089] Step 2:

[0090] Data Preprocessing

[0091] The server preprocesses the collected data. As input, it receives the text data and audio data collected in step 1. Specifically, it cleans the text data (removes unnecessary spaces and special characters) and removes noise from the audio data. It tokenizes the text data using the Python library NLTK and removes noise from the audio data using an audio processing library. As output, it obtains the preprocessed text data and audio data.

[0092] Step 3:

[0093] Fine-tuning machine learning models

[0094] The server uses the preprocessed data to fine-tune the machine learning model. As input, it receives the text data and audio data preprocessed in step 2. Specifically, it uses models such as OpenAI's GPT-3 and Google's BERT, inputs family conversation data into the model, and trains it. As output, it obtains a fine-tuned machine learning model.

[0095] Step 4:

[0096] Conversation Generation

[0097] ChatBot uses the fine-tuned model to generate conversations with elderly people. As input, it receives questions and utterances from the user (elderly person). In concrete terms, it inputs prompt sentences into the generative AI model and generates natural conversations. For example, if an elderly person asks, "What shall we do today?", ChatBot responds with, "The weather is nice today, so how about going for a walk?" The generated conversation is obtained as output.

[0098] Step 5:

[0099] Audio Reproduction

[0100] The server learns the voice data of the elderly and extracts its features. As input, it receives the voice data preprocessed in step 2. Specifically, it uses a voice reproduction AI to learn the features of the elderly's voice and speaks the generated text in the elderly's voice. As output, a voice reproduced in the elderly's voice is obtained.

[0101] Step 6:

[0102] Conversation History Summary

[0103] The server summarizes the conversation history with the elderly person using natural language processing technology. As input, it receives the conversation history generated in step 4. Specifically, it summarizes the conversation content using tools such as the BERT summarizer and extracts important points and topics. As output, it obtains the summarized conversation history.

[0104] Step 7:

[0105] Reporting to family

[0106] The server reports the summarized conversation history to the family. As input, it receives the summarized conversation history from step 6. Specifically, it notifies the family of the elderly person's situation and the contents of the conversation via email or a dedicated app. As output, it obtains the report sent to the family.

[0107] (Application example 1)

[0108] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0109] In modern society, the increase in the number of elderly people living alone is becoming a serious problem. Elderly people living alone are particularly prone to feeling lonely and being at higher risk of dementia, and reduced interaction with family members can have a negative impact on their mental and physical health. In addition, if family members live far away, it is difficult to keep track of the elderly's condition. To solve these problems, it is necessary to provide an environment where the elderly can live with peace of mind, and a system that allows family members to properly keep track of the elderly's condition.

[0110] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0111] In this invention, the server includes a means for providing a speech ChatBot for communicating with elderly people living alone, a means for fine-tuning personal information and conversation habits of family members, a means for generating and speaking conversations on behalf of the elderly person, a means for summarizing conversation history and content with the elderly person and reporting it to the family, a means for reproducing the elderly person's voice using a voice reproduction AI, a means for voice recognition of the elderly person's speech, a means for converting the generated conversation into voice and playing it back, and a means for detecting abnormalities and notifying family members and emergency contacts. This allows the elderly person to feel as if they are interacting with their family without feeling lonely, and further allows the family member to properly grasp the elderly person's condition and quickly respond if an abnormality occurs.

[0112] "Single-person elderly" refers to elderly people who live alone.

[0113] A "Speech ChatBot" refers to artificial intelligence that can converse with humans through voice.

[0114] "Family personal information" refers to information about family members such as names, ages, and relationships.

[0115] "Conversational habits" refers to the way a particular person speaks or uses language.

[0116] "Fine-tuning" refers to the fine-tuning of an existing machine learning model to adapt it to a specific dataset.

[0117] "Means for generating and speaking a conversation" refers to technology that creates an appropriate response based on input information and outputs it as voice.

[0118] "Conversation history" refers to a record of past conversations.

[0119] A "summary" refers to summarizing a longer piece of text or the contents of a conversation.

[0120] "Voice reproduction AI" refers to artificial intelligence technology that generates a voice that imitates a specific person's voice.

[0121] "Speech recognition" refers to the technology that converts speech into text.

[0122] "Means for converting to audio and playing it back" refers to technology for converting text data into audio data and playing it back.

[0123] "Means of detecting abnormalities and notifying family members or emergency contacts" refers to technology that detects abnormalities in an elderly person's condition and notifies family members or designated contacts.

[0124] As an embodiment of the present invention, we provide a ChatBot system for communicating with elderly people who live alone. The system is configured using the following hardware and software.

[0125] Hardware

[0126] Smartphone: Equipped with a microphone and speaker for audio input and output.

[0127] Server: A powerful computer that processes and stores data.

[0128] software

[0129] Python: A programming language used to implement programs.

[0130] OpenAI API: An API for generating conversations using generative AI models.

[0131] Speech Recognition Library: A library for converting speech to text.

[0132] gTTS (Google Text-to-Speech): A library for converting text to speech.

[0133] Playback software: Software for playing audio files.

[0134] System Operation

[0135] 1. Voice recognition: The device (smartphone) uses a microphone to record the elderly person's speech and converts the speech into text using a voice recognition library.

[0136] 2. Conversation generation: The server uses the OpenAI API to generate appropriate responses using a generative AI model that has learned the family's personal information and conversational habits.

[0137] 3. Audio playback: The device uses gTTS to convert the generated text into audio and plays it over the speaker.

[0138] 4. Conversation summary: The server summarizes the conversation history and content with the elderly person and reports it to the family.

[0139] 5. Anomaly detection: If the server detects an abnormality, it will notify family members or emergency contacts.

[0140] Examples

[0141] For example, if an elderly person says, "The weather is nice today," the system operates as follows.

[0142] 1. The smartphone microphone records the elderly person's speech and uses a speech recognition library to convert it into text such as "The weather is nice today."

[0143] 2. The server uses the OpenAI API to input the prompt sentence, "Conversation with an elderly person: The weather is nice today. ChatBot's response, having learned the conversational habits of family members:" into the generative AI model and generates an appropriate response.

[0144] 3. The generated response (e.g., "Yes, today is a perfect day to go for a walk") is converted to speech using gTTS and played over the smartphone's speaker.

[0145] 4. The server summarizes the conversation and notifies the family that "The weather is nice today" was said.

[0146] 5. If an abnormality is detected, the server will notify family members or emergency contacts.

[0147] Examples of prompt statements

[0148] Conversation with an Elderly Person: The weather is nice today

[0149] The ChatBot's response, having learned the family's conversational habits:

[0150] In this way, the form for carrying out the invention can be concretely shown.

[0151] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0152] Step 1:

[0153] The device (smartphone) uses a microphone to record the elderly person's speech. The input is the elderly person's voice, and the output is the recorded voice data. This voice data is passed to a voice recognition library.

[0154] Step 2:

[0155] The terminal converts the recorded voice data into text using a voice recognition library. The input is voice data and the output is text data. This text data is sent to the server.

[0156] Step 3:

[0157] Based on the received text data, the server inputs a prompt sentence into the generative AI model using the OpenAI API. The input is text data, and the output is the generated response text. An example of a prompt sentence is "Conversation with an elderly person: The weather is nice today. ChatBot's response based on learning the conversational habits of family members:"

[0158] Step 4:

[0159] The server sends the generated response text to the terminal. The input is the generated response text and the output is the response text sent to the terminal.

[0160] Step 5:

[0161] The device converts the received response text into speech using gTTS. The input is the response text, and the output is audio data. This audio data is played on the smartphone speaker.

[0162] Step 6:

[0163] The server summarizes the conversation history and contents with the elderly person and reports them to the family. The input is the text data of the conversation, and the output is the summarized report data. This report data is notified to the family.

[0164] Step 7:

[0165] If the server detects an abnormality, it notifies the family and emergency contacts. The input is data for detecting the abnormality, and the output is notification data. This notification data is sent to the family and emergency contacts.

[0166] Example 2

[0167] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0168] There is a need to detect the risk of dementia and loneliness in the elderly at an early stage and take appropriate measures. However, conventional methods often lack communication with the elderly, and risk detection is often delayed. In addition, there are limited means for families to understand the condition of the elderly. This makes it difficult to prevent the deterioration of the elderly's health.

[0169] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0170] In this invention, the server includes a means for providing a speech bot for communicating with elderly people living alone, a means for adjusting personal information and conversation habits of family members, a means for generating and speaking a conversation on behalf of the elderly person, a means for summarizing the conversation history and content with the elderly person and reporting it to the family, a means for reproducing the voice of the elderly person using voice reproduction technology, a means for recording the conversation content in real time and transmitting it to the server, a means for storing the received conversation data in a database, a means for evaluating the conversation content using a generative AI model for analyzing the conversation data in the database, a means for calculating the dementia risk based on the evaluation result of the generative AI model, and a means for notifying the family member of the result if the dementia risk is determined to be high. This makes it possible to detect the dementia risk and loneliness of elderly people early and notify the family member promptly.

[0171] A "talkbot" is a conversational artificial intelligence system designed to interact with the elderly.

[0172] "Family personal information" refers to basic information about an elderly person's family members, including names, contact details, and relationships.

[0173] "Conversational habits" refers to the words, phrases and speaking patterns that older people use on a daily basis.

[0174] "Means of adjustment" refers to ways in which the system can optimize its settings and operation to suit the characteristics of the elderly person and their family.

[0175] "Means of generating and speaking conversation" refers to the system's ability to automatically create conversation content and output it as audio.

[0176] "Conversation history" refers to records of past conversations between the elderly person and the speech bot.

[0177] "Summarization techniques" refer to methods for summarizing long conversations and extracting important information.

[0178] "Voice reproduction technology" refers to technology that imitates and generates the voice of a specific person.

[0179] "Real-time recording means" refers to a method of recording the content of a conversation as it occurs.

[0180] "Server" refers to a computer system for storing, processing, and managing data.

[0181] "Database" refers to a system for efficiently storing, retrieving, and managing structured data.

[0182] A "generative AI model" refers to a model that uses artificial intelligence technology to analyze data and extract specific patterns and information.

[0183] "Dementia risk" refers to an indicator used to assess the likelihood that an elderly person will develop dementia.

[0184] "Means of notification" refers to the method by which specific information is communicated to families.

[0185] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0186] This invention is a system for assessing the risk of dementia and loneliness in elderly people and notifying their families. This system is realized using hardware and software such as a speech bot, a server, a generative AI model, and a database.

[0187] Hardware and Software Used

[0188] Hardware: Servers, terminals (smartphones, tablets, PCs, etc.)

[0189] Software: speech bots, generative AI models (e.g. GPT-4®), database management systems (e.g. MySQL®)

[0190] System Overview

[0191] The user uses a device to access the speech bot and begin a conversation. The speech bot records the content of the conversation in real time and sends it to the server. The server stores the received conversation data in a database and periodically analyzes the content of the conversation using a generative AI model. The generative AI model detects signs of dementia, and the server calculates the dementia risk based on the evaluation results. If the risk is determined to be high, the server notifies the family of the result.

[0192] Examples

[0193] For example, if a user had the following conversation with a speech bot:

[0194] User: "I can't remember what I had for dinner yesterday."

[0195] Speech Bot: "That's concerning. Have you been forgetful at any other times lately?"

[0196] User: "Yeah, I keep telling the same stories over and over."

[0197] This conversation data is sent to a server and analyzed by the generative AI model. The model detects signs of dementia from statements such as "I don't remember what I had for dinner yesterday" or "I keep repeating the same thing." The server uses this data to assess the risk of dementia and notify family members if necessary.

[0198] Examples of prompt statements

[0199] Here is an example of a prompt to be input to the generative AI model:

[0200] Analyze the user's conversation and evaluate whether they have signs of dementia. See the following conversation for reference:

[0201] User: "I can't remember what I had for dinner yesterday."

[0202] Speech Bot: "That's concerning. Have you been forgetful at any other times lately?"

[0203] User: "Yeah, I keep telling the same stories over and over."

[0204] By inputting this prompt into a generative AI model, the model analyzes the conversation content and assesses signs of dementia.

[0205] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0206] Step 1:

[0207] A user accesses the speech bot using a terminal.

[0208] Input: A user accesses the speechbot's application or website using a smartphone, tablet, or PC.

[0209] Output: The speechbot home screen is displayed.

[0210] Specific operation: A user opens the app and the login screen is displayed. The user enters their ID and password to log in.

[0211] Step 2:

[0212] A user starts a conversation with a speech bot.

[0213] Input: The user speaks to the speech bot.

[0214] Output: The speech bot recognizes what the user said and generates a response.

[0215] Specific behavior: After logging in, the user taps the "Start conversation" button, then taps the microphone icon to speak. For example, the user speaks to the speech bot, "Hello, how are you today?"

[0216] Step 3:

[0217] The speech bot records the conversation in real time and sends it to the server.

[0218] Input: What the user says.

[0219] Output: Sent to the server as text data.

[0220] Specific operation: The speech bot converts the user's speech into text using speech recognition technology and immediately sends it to the server. For example, the user's speech "Hello, how are you today?" is sent to the server as text data.

[0221] Step 4:

[0222] The server stores the received conversation data in a database.

[0223] Input: The text data sent to the server.

[0224] Output: Conversation data stored in a database.

[0225] Specific operation: The server saves the received text data with a timestamp in the "conversation history" table of the database. For example, the server saves the conversation data "Hello, how are you today?" in the database.

[0226] Step 5:

[0227] The server periodically analyzes the conversation data in the database and evaluates the conversation content using a generative AI model.

[0228] Input: Conversation data in the database.

[0229] Output: Evaluation results from the generative AI model.

[0230] Specific operation: The server executes a scheduled task at 2 a.m. every day and inputs the past 24 hours of conversation data into the generative AI model. For example, the server obtains the past 24 hours of conversation data and inputs it into the generative AI model as a prompt sentence.

[0231] Step 6:

[0232] Generative AI model detects signs of dementia.

[0233] Input: Conversation data entered as prompt sentences.

[0234] Output: Assessment results indicating signs of dementia.

[0235] Specific operation: The generative AI model analyzes the conversation data based on the prompt sentence and detects keywords and patterns that indicate signs of dementia. For example, the generative AI model detects signs of dementia from statements such as "I don't remember what I had for dinner yesterday" or "I keep repeating the same thing."

[0236] Step 7:

[0237] The server calculates the risk of dementia based on the evaluation results of the generated AI model.

[0238] Input: Evaluation results of a generative AI model.

[0239] Output: Dementia risk score.

[0240] How it works: The server analyzes the output of the generative AI model and runs an algorithm to calculate a risk score. For example, the server calculates a "dementia risk score" and determines whether the score exceeds a certain threshold.

[0241] Step 8:

[0242] If the server determines that there is a high risk of dementia, it notifies the family of the result.

[0243] Input: Dementia risk score.

[0244] Output: A notification message to family members.

[0245] Specific operation: If the risk score exceeds the threshold, the server generates a message to notify the family and sends it via the registered email address or app notification. For example, the server sends a message to the family saying, "Based on your father's recent conversations, we have determined that he is at high risk for dementia. We recommend a detailed evaluation."

[0246] (Application example 2)

[0247] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0248] There is a need to detect dementia risk in the elderly at an early stage and take appropriate measures. However, with the number of elderly people living alone increasing, it is difficult for family members and caregivers to keep track of the elderly's condition on a daily basis. In addition, reducing the elderly's sense of loneliness and maintaining their mental health are also important issues. To solve these issues, a system is needed that assesses the dementia risk through daily conversations with the elderly and notifies the family and caregivers as necessary.

[0249] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0250] In this invention, the server includes a means for providing a speech ChatBot for communicating with elderly people who live alone, a means for fine-tuning personal information and conversation habits of family members, a means for generating and speaking a conversation on behalf of the elderly person, a means for summarizing the conversation history and content with the elderly person and reporting it to the family, a means for reproducing the voice of the elderly person using voice reproduction artificial intelligence, a means for converting the conversation content from voice to text, a means for analyzing the conversation pattern and frequency, and a means for evaluating the dementia risk and notifying the family member or caregiver. This allows the dementia risk of the elderly person to be detected early, and the family member or caregiver to take appropriate measures. It also reduces the elderly person's sense of loneliness and maintains their mental health.

[0251] "Talk ChatBot" is a dialogue system that uses artificial intelligence to conduct voice conversations with elderly people.

[0252] "Fine-tuning" means that the system adjusts the optimal content and method of conversation based on a family member's personal information and conversational habits.

[0253] "Summarization" means concisely summarizing the conversation history and content with the elderly person.

[0254] "Voice reproduction artificial intelligence" is an artificial intelligence technology that imitates and reproduces the voice of a specific person.

[0255] "Convert to text" refers to the process of converting voice data into character data.

[0256] "Analysis" refers to analyzing conversation patterns and frequencies to detect certain trends or anomalies.

[0257] "Dementia risk" is an indicator that evaluates the degree of likelihood that an elderly person will develop dementia.

[0258] "Notification" refers to informing family members or caregivers of information detected by the system.

[0259] A system for implementing this invention has the following configuration: The server includes means for providing a speech ChatBot for communicating with elderly people living alone, means for fine-tuning personal information and conversational habits of family members, means for generating and speaking conversations on behalf of the elderly person, means for summarizing the conversation history and content with the elderly person and reporting it to the family, means for reproducing the elderly person's voice using voice reproduction AI, means for converting the content of the conversation from voice to text, means for analyzing the conversation pattern and frequency, and means for assessing the risk of dementia and notifying the family member or caregiver.

[0260] Hardware and Software Used

[0261] Hardware: Smartphone (iOS or ANDROID (registered trademark))

[0262] Software: Speech recognition API (Google Cloud Speech-to-Text), natural language processing library (spaCy), notification service (messaging service)

[0263] Data processing and calculation

[0264] 1. Audio recording and transcription:

[0265] Conversations with elderly people are recorded using the microphone on the device (smartphone).

[0266] The recorded audio data is converted to text using the Google Cloud Speech-to-Text API.

[0267] 2. Conversation Analysis:

[0268] The server analyzes the converted text data using a natural language processing library (spaCy).

[0269] Analyze conversation patterns, frequency and content to detect signs of dementia.

[0270] 3. Risk Assessment:

[0271] The server assesses dementia risk based on the symptoms detected, for example scoring how often a person repeats themselves or has difficulty finding words.

[0272] 4. Notification function:

[0273] The server uses a messaging service to send notifications to family members or caregivers if there is a high risk.

[0274] Examples

[0275] Scenario: An elderly person talks with their family members through an app every day. The device records the conversation, converts it into text, and sends it to a server. The server analyzes the content of the conversation and detects an increase in the frequency of repeating the same things. The server assesses that the risk of dementia is increasing and sends a notification to the family members to encourage early medical consultation.

[0276] Examples of prompt statements

[0277] Develop an application that records conversations with seniors, converts them into text, and analyzes them. Analyze conversation patterns and frequency to detect signs of dementia. Evaluate dementia risk based on detected signs, and send notifications to family members or caregivers if risk is high. Technologies used are Google Cloud Speech-to-Text API, spaCy, and messaging services.

[0278] In this way, we demonstrated a concrete example of implementing a talking ChatBot to assess the dementia risk of elderly people as a smartphone app.

[0279] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0280] Step 1:

[0281] The terminal records conversations with the user (elderly person). The input is the user's voice, and the output is the recorded voice data. Specifically, the terminal's microphone is used to capture the voice and save it as an audio file.

[0282] Step 2:

[0283] The device converts the recorded voice data into text using the Google Cloud Speech-to-Text API. The input is the recorded voice data, and the output is the converted text data. The specific operation is to send the audio file to the API and receive the returned text data.

[0284] Step 3:

[0285] The terminal transmits the converted text data to the server. The input is the text data, and the output is the text data transmitted to the server. In particular, the text data is uploaded to the server via a network.

[0286] Step 4:

[0287] The server analyzes the received text data using a natural language processing library (spaCy). The input is text data, and the output is the analysis results. Specifically, the text data is input into spaCy, and the conversation patterns and frequency are analyzed.

[0288] Step 5:

[0289] The server evaluates the dementia risk based on the analysis results. The input is the analysis results, and the output is a dementia risk assessment score. In specific operations, the analysis results are input to an assessment algorithm to calculate a risk score.

[0290] Step 6:

[0291] The server uses a messaging service to send a notification to family members or caregivers if the dementia risk assessment score is high. The input is the dementia risk assessment score and the output is the sent notification. Specific operations include evaluating the risk score and generating and sending a notification message.

[0292] Step 7:

[0293] The server summarizes the conversation history and content with the elderly person and reports it to the family. The input is text data and analysis results, and the output is a summarized report. Specifically, the server generates a summary based on the text data and analysis results and sends it to the family.

[0294] Example 3

[0295] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0296] There is a need to detect loneliness and dementia risk in elderly people at an early stage and take appropriate measures. However, with conventional systems, it was difficult to effectively analyze the content and frequency of conversations with elderly people and their emotional expressions to evaluate loneliness and dementia risk. There was also a lack of means to notify family members at an appropriate time. This led to a high possibility that loneliness and dementia risk in elderly people would be overlooked, and appropriate measures would be delayed, which was a problem.

[0297] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0298] In this invention, the server includes a means for recording a conversation with an elderly person and converting it into text data, a means for transmitting the converted text data to the server, a means for analyzing the content of the conversation using a generative AI model, a means for calculating a loneliness index based on the analysis result, and a means for notifying a family member when the loneliness index exceeds a certain threshold. This makes it possible to detect loneliness and dementia risk in an elderly person at an early stage and notify the family member at an appropriate time.

[0299] "Single-person elderly" refers to elderly people who live alone.

[0300] A "talkbot" is a system equipped with artificial intelligence that can converse with users via voice.

[0301] "Family personal information" refers to information such as family members' names, contact details, and relationships.

[0302] "Means of adjusting conversational habits" refers to technology that optimizes the voice bot's responses based on the user's conversational patterns and preferences.

[0303] "Means for generating and speaking a conversation" refers to technology that generates an appropriate response based on user input and speaks it aloud.

[0304] "Means of summarizing conversation history and content and reporting it to family members" refers to technology that summarizes the content of conversations with elderly people and notifies their family members.

[0305] "Voice reproduction technology" refers to technology that imitates and reproduces the voice of a specific person.

[0306] "Means for converting into text data" refers to technology for converting voice data into character data.

[0307] "Means for transmitting to a server" refers to a technique for transmitting data from a terminal to a server.

[0308] A "generative AI model" refers to a model that uses artificial intelligence to analyze data and generate a specific outcome.

[0309] "Means of analyzing the content of the conversation" refers to technology that uses a generative AI model to analyze the content of the conversation and extract specific information.

[0310] "Means for calculating an index of loneliness" refers to technology that quantifies the loneliness of elderly people based on the results of analyzing the content of their conversations.

[0311] "Means for notifying family members" refers to technology that sends alerts or notifications to family members when certain conditions are met.

[0312] This invention is a system that evaluates the loneliness and dementia risk of elderly people and notifies their families. The system is composed of a speech bot, a server, a terminal, and a generative AI model.

[0313] Hardware and Software Configuration

[0314] server

[0315] The server stores and processes data. Specifically, it uses a cloud service. For example, Amazon (registered trademark) Web Services (AWS (registered trademark)) can be used. The server has the following functions:

[0316] Saving text data

[0317] Data Analysis Using Generative AI Models

[0318] Calculating the loneliness index

[0319] Notification to family members

[0320] Terminal

[0321] The terminal is a device used by the elderly. Specifically, a smartphone or tablet is used. The terminal has the following functions:

[0322] Recording conversations with seniors

[0323] Convert audio recordings to text

[0324] Sending text data to the server

[0325] Generative AI Models

[0326] The generative AI model analyzes conversation content and evaluates loneliness. Specifically, it uses OpenAI's GPT-4. The generative AI model has the following functions:

[0327] Conversation analysis

[0328] Extracting keywords that indicate emotions and loneliness

[0329] Data processing and calculation

[0330] Collecting conversation data

[0331] The device records conversations with the elderly and converts them into text data in real time, using a voice recognition API (for example, Google Cloud Speech-to-Text API).

[0332] Sending conversation data

[0333] The terminal sends the converted text data to the server using the HTTPS protocol. The server stores the received text data in a database. Specifically, it uses Amazon RDS (Relational Database Service).

[0334] Conversation analysis

[0335] The server inputs the stored text data into the generative AI model. The prompt sentence is "Please analyze the elderly person's emotions from the content of this conversation." The generative AI model analyzes the content of the conversation and extracts keywords that indicate the elderly person's emotions and feelings of loneliness.

[0336] Loneliness assessment

[0337] The server calculates the loneliness index based on the analysis results obtained from the generated AI model. Specifically, it evaluates the frequency of keywords that indicate loneliness, the lack of emotional expression, and the frequency of conversations. If the loneliness index exceeds a certain threshold, an alert is generated.

[0338] Notification to family members

[0339] When the server generates an alert, it will send a notification to the family members via email or SMS. The family members will receive the notification and take action to check on the elderly person's condition.

[0340] Examples of concrete examples and prompts

[0341] Examples

[0342] Conversation content: Elderly people frequently use phrases such as "I haven't talked to anyone today" and "I feel lonely."

[0343] Emotional expression: Elderly people tend to express little emotion in their speech.

[0344] Conversation Frequency: Conversations recorded only once a week.

[0345] Examples of prompt statements

[0346] "Please analyze what emotions elderly people express in recent conversations."

[0347] "Please rate the sense of loneliness based on this conversation."

[0348] In this way, the system can evaluate the elderly person's sense of loneliness and notify the family if necessary. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0349] Step 1:

[0350] Collecting conversation data

[0351] Terminal: Elderly people talk to the speech bot using a smartphone or tablet. The conversation is conducted by voice and is recorded by the terminal.

[0352] Input: Elderly people's voice conversation

[0353] Output: Recorded audio data

[0354] Specific operation: The elderly person says, "I haven't talked to anyone today." The device records this statement.

[0355] Step 2:

[0356] Converting audio data to text

[0357] Terminal: Converts recorded voice data into text data in real time, using a voice recognition API.

[0358] Input: Recorded audio data

[0359] Output: The converted text data

[0360] Specific operation: The device converts the recorded voice data into text data such as "I haven't talked to anyone today."

[0361] Step 3:

[0362] Sending text data

[0363] Terminal: The converted text data is sent to the server using the HTTPS protocol.

[0364] Input: Converted text data

[0365] Output: Text data sent to the server

[0366] Specific operation: The device sends the text data "I haven't talked to anyone today" to the server.

[0367] Step 4:

[0368] Saving text data

[0369] Server: The received text data is stored in a database. Specifically, a database service is used.

[0370] Input: Text data sent to the server

[0371] Output: Text data stored in the database

[0372] Specific operation: The text data received by the server is saved in the "conversation log" table.

[0373] Step 5:

[0374] Conversation analysis

[0375] Server: Input the saved text data into the generative AI model. Use the prompt "Please analyze the elderly person's emotions from this conversation."

[0376] Input: Text data stored in a database

[0377] Output: Analysis results from generative AI models

[0378] Specific operation: The server inputs the text data "I haven't talked to anyone today" into the generative AI model, and obtains the analysis result "Keywords indicating loneliness: I haven't talked to anyone."

[0379] Step 6:

[0380] Loneliness assessment

[0381] Server: Calculates an index of loneliness based on the analysis results obtained from the generative AI model. Specifically, it evaluates the frequency of keywords that indicate loneliness, the lack of emotional expression, and the frequency of conversations.

[0382] Input: Analysis results by generative AI model

[0383] Output: Loneliness index

[0384] Specific operation: Based on the analysis results, the server evaluates the loneliness index as "high."

[0385] Step 7:

[0386] Notification to family members

[0387] Server: If the loneliness index exceeds a certain threshold, a notification is sent to the family via email or SMS.

[0388] Input: Loneliness index

[0389] Output:Notification to family members

[0390] Specific behavior: The server generates an alert and sends an email to the family saying, "An elderly person may be feeling lonely. Recent conversations: Not talking to anyone today."

[0391] (Application example 3)

[0392] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0393] There is a need to detect loneliness and dementia risk in the elderly at an early stage and take appropriate measures. However, current systems have difficulty accurately evaluating the emotions and loneliness of the elderly, and are unable to provide appropriate information to family members and care managers. In addition, there is a need to improve the operability and convenience of smartphones for the elderly.

[0394] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means. In this invention, the server includes a means for providing a speech ChatBot for interacting with elderly people living alone, a means for fine-tuning personal information and conversation habits of family members, a means for generating and speaking a conversation on behalf of the elderly person, a means for summarizing the conversation history and contents with the elderly person and reporting them to the family, a means for reproducing the voice of the elderly person using voice reproduction artificial intelligence, an emotion analysis means for evaluating the elderly person's sense of loneliness, a means for evaluating an index of loneliness and notifying the family member, and a means for operating as an application installed on a smartphone. This makes it possible to detect the elderly person's sense of loneliness and the risk of dementia at an early stage and provide appropriate information to the family member and the care manager.

[0395] "Speech ChatBot" is a dialogue system that uses artificial intelligence to conduct natural conversations with elderly people.

[0396] "Fine-tuning" is the process of optimizing the system's behavior and responses based on a family member's personal information and conversational habits.

[0397] "Conversation generation" is a function in which artificial intelligence automatically creates and speaks dialogue with elderly people.

[0398] "Summarization" is the process of concisely summarizing the conversation history and content with an elderly person.

[0399] "Voice reproduction artificial intelligence" is an artificial intelligence technology that imitates and reproduces the voice of a specific person.

[0400] "Sentiment analysis" is a technology that analyzes emotions from the content of a conversation and evaluates the type and intensity of those emotions.

[0401] The "loneliness index" is a numerical or evaluation standard that expresses the sense of loneliness of elderly people.

[0402] "Notification" is the process of informing family members and care managers of evaluation results and important information.

[0403] An "application installed on a smartphone" is a software program that runs on a smartphone.

[0404] The system for implementing this invention is configured as follows. First, the server provides a speech ChatBot for interacting with the elderly. The speech ChatBot is a dialogue system that uses artificial intelligence to have natural conversations with the elderly. The server also fine-tunes the system's operation and responses to optimize them based on the family's personal information and conversational habits.

[0405] Next, the server has a conversation generation function that automatically creates and speaks the contents of conversations with the elderly. In addition, it provides a summary function that briefly summarizes the conversation history and contents with the elderly and reports them to the family. It also includes a function to mimic and reproduce the voice of a specific person using voice reproduction artificial intelligence.

[0406] The server performs emotion analysis to evaluate the elderly person's loneliness. Emotion analysis is a technology that analyzes emotions from conversation content and evaluates the type and intensity of those emotions. It also includes a function to evaluate the loneliness index and notify the family of the result.

[0407] The system operates as an application installed on a smartphone, and is designed to be easily accessible to seniors through a software program that runs on the smartphone.

[0408] As a concrete example, consider the case where an elderly person talks to a ChatBot through a smartphone. If the elderly person types, "I haven't talked to anyone recently and I feel lonely," the server retrieves the content of the conversation and performs sentiment analysis using the TextBlob library. As a result of the sentiment analysis, negative emotions are detected, and the keyword "lonely" is also included, so the loneliness index becomes high. As a result, a notification is sent to the family saying, "An elderly person is feeling lonely. I will report this to the family."

[0409] An example of a prompt is as follows:

[0410] Enter the conversation you had with the senior:

[0411] Using this prompt, the user can input the content of the conversation, allowing the system to evaluate the sense of loneliness and notify family members if necessary.

[0412] The flow of the specific process in the application example 3 will be described with reference to FIG.

[0413] Step 1:

[0414] A user starts a conversation with the speech ChatBot through a smartphone. The user inputs the content of the conversation and sends it to the smartphone terminal. The input data is in text format, for example, "I haven't talked to anyone recently and I feel lonely."

[0415] Step 2:

[0416] The terminal sends the input conversation content to the server. The server first preprocesses the received text data in order to analyze it. Preprocessing includes normalizing the text (e.g., removing unnecessary spaces and special characters).

[0417] Step 3:

[0418] The server performs sentiment analysis on the preprocessed text data. It uses the TextBlob library to evaluate the sentiment polarity of the text. Based on the input data, it outputs a sentiment: positive, negative, or neutral.

[0419] Step 4:

[0420] The server evaluates the loneliness index based on the results of the emotion analysis. Specifically, if negative emotions are detected or if keywords such as "lonely" are included, the loneliness score is increased. A loneliness score is calculated based on the input data.

[0421] Step 5:

[0422] The server evaluates the loneliness score and sends a notification to the family if it exceeds a certain threshold. The notification includes the loneliness assessment result and a summary of the conversation. The output data is a notification message that is sent to the family.

[0423] Step 6:

[0424] Family members can receive notifications and take appropriate measures to address the elderly person's feelings of loneliness. The notification message includes specific conversation content and the results of sentiment analysis, allowing family members to accurately grasp the elderly person's situation.

[0425] Through the above processing steps, it is possible to evaluate the elderly person's sense of loneliness and provide appropriate information to the family.

[0426] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0427] "Example 1"

[0428] As one embodiment of the present invention, a system incorporating an emotion engine is provided. This system recognizes the emotional state of the elderly person through conversation with the elderly person and reports the result to the family. Specifically, the emotion engine estimates the emotional state of the elderly person from the elderly person's tone of voice, speaking style, choice of words, etc. For example, if the elderly person says "I feel lonely," the emotion engine captures the statement and determines that the elderly person is feeling lonely. This information is reported to the family, allowing the family to understand the elderly person's emotional state.

[0429] "Example 2"

[0430] The emotion engine also has the ability to adjust the content of speech depending on the emotional state of the elderly person. Specifically, if it is determined that the elderly person is feeling happy or excited, the Speech ChatBot will provide topics that will maintain that emotional state. For example, if there is information that the elderly person feels happy when talking about their grandchildren, the Speech ChatBot will provide topics about grandchildren. This enables more appropriate communication that takes into account the emotional state of the elderly person.

[0431] "Example 3"

[0432] Furthermore, the emotion engine has the function of evaluating the elderly person's emotional state and reporting the results to the family. Specifically, if it is determined that the elderly person is feeling sad or anxious, the information is reported to the family. For example, if the elderly person says "I'm worried," the emotion engine will capture the statement and determine that the elderly person is feeling anxious. This information is reported to the family, enabling the family to understand the elderly person's emotional state and take appropriate action.

[0433] The process flow of each embodiment will be described below.

[0434] "Example 1"

[0435] Step 1: The speech ChatBot starts a conversation with the elderly person.

[0436] Step 2: The emotion engine infers the emotional state of the elderly person from their tone of voice, speaking style, and word choice.

[0437] Step 3: The emotion engine evaluates the elderly person's emotional state and reports the results to the family.

[0438] "Example 2"

[0439] Step 1: The speech ChatBot starts a conversation with the elderly person.

[0440] Step 2: The emotion engine estimates the elderly person's emotional state.

[0441] Step 3: The emotion engine adjusts the speech content according to the elderly person's emotional state.

[0442] "Example 3"

[0443] Step 1: The speech ChatBot starts a conversation with the elderly person.

[0444] Step 2: The emotion engine estimates the elderly person's emotional state.

[0445] Step 3: The emotion engine evaluates the elderly person's emotional state and reports the results to the family.

[0446] Example 1

[0447] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0448] Elderly people who live alone often feel lonely and have a lack of interaction with their families. In addition, it is difficult for families to grasp the emotional state and dementia risk of the elderly, which can delay appropriate responses. This can lead to a decline in the quality of life of the elderly.

[0449] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0450] In this invention, the server includes a means for providing a speech system for communicating with elderly people living alone, a means for fine-tuning personal information and conversation habits of family members using a machine learning algorithm, a means for generating and speaking a conversation on behalf of the elderly person, a means for summarizing the conversation history and content with the elderly person using natural language processing technology and reporting it to the family, a means for reproducing the elderly person's voice using voice reproduction technology, and an emotion engine for recognizing the elderly person's emotional state and reporting the results to the family. This allows the elderly person to feel as if they are interacting with their family, and the family member can also understand the elderly person's emotional state and dementia risk.

[0451] "Single-person elderly" refers to elderly people who live alone.

[0452] A "speech system" refers to a system for communicating with people using voice.

[0453] A "machine learning algorithm" refers to an algorithm that trains a model based on data and makes predictions and classifications.

[0454] "Fine-tuning" refers to the process of fine-tuning an existing machine learning model to adapt it to a specific dataset.

[0455] "Natural language processing technology" refers to technology that enables computers to understand, interpret, and generate human language.

[0456] "Voice reproduction technology" refers to technology for imitating and reproducing the voice of a specific person.

[0457] An "emotion engine" refers to an engine for analyzing and inferring emotional states from voice and text data.

[0458] "Conversation history" refers to a record of past conversations.

[0459] A "summary" is a summary of a longer piece of text or a conversation.

[0460] The present invention provides a speech system for communicating with elderly people who live alone. A specific embodiment of this system will be described below.

[0461] The server first collects conversation data from family members. Family members input text and voice data through a dedicated app and send it to the server. The collected data is then stored in a database.

[0462] Next, the server uses Python and TensorFlow to build a machine learning model, using the collected family conversation data as training data to fine-tune the model, learning the habits and characteristics of family conversations.

[0463] When an elderly person speaks to the device, the device sends the input to the server. The server uses a machine learning model to generate an appropriate response and speaks it out loud through the device. At this time, voice reproduction technology is used to reproduce the elderly person's voice. Specifically, a Python voice processing library is used to analyze the voice data and model the characteristics of the elderly person's voice.

[0464] In addition, the device records conversation history with the elderly. The server analyzes the recorded conversation data and extracts important information using Python's NLTK library. The extracted information is reported to the family as a summary.

[0465] The emotion engine analyzes the elderly person's tone of voice and speaking style in real time. The server uses Python's OpenCV and Dlib libraries to estimate the emotional state. For example, if an elderly person says "I feel lonely," the emotion engine captures the statement and determines that the elderly person is feeling lonely. This information is reported to the family through the server.

[0466] As a concrete example, if an elderly person says, "I feel lonely today," the device will send this statement to the server. The server will analyze this statement using an emotion engine and determine that the elderly person is feeling lonely. This information will be reported to the family as a summary. The family will receive this information and can take action, such as calling the elderly person.

[0467] An example of a prompt sentence might be:

[0468] “If an elderly person feels lonely, explain how the emotion engine recognizes that emotion and reports it to the family.”

[0469] This system allows elderly people to feel as if they are interacting with their families, and enables families to understand the elderly person's emotional state and dementia risk.

[0470] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0471] Step 1:

[0472] The server collects family conversation data. The user inputs text and voice data through a dedicated app and sends it to the server. The server stores the received data in a database. The input is family conversation data, and the output is the data stored in the database.

[0473] Step 2:

[0474] The server uses Python and TensorFlow to build a machine learning model. The server retrieves family conversation data from the database and uses it as learning data. The server uses a machine learning algorithm to train the model and have it learn the habits and characteristics of family conversations. The input is the conversation data retrieved from the database, and the output is a fine-tuned machine learning model.

[0475] Step 3:

[0476] When an elderly person speaks to the device, the device sends the input to the server. The server uses a machine learning model to generate an appropriate response. The server sends the generated response to the device, which then speaks it out loud. The input is the elderly person's speech data, and the output is the audio data of the generated response.

[0477] Step 4:

[0478] The terminal records conversation history with the elderly. The server analyzes the recorded conversation data and extracts important information using Python's NLTK library. The server reports the extracted information to the family as a summary. The input is the recorded conversation data, and the output is the report data as a summary.

[0479] Step 5:

[0480] The server learns the voice data of the elderly and extracts its features. The server analyzes the voice data using a Python voice processing library and models the features of the elderly's voice. The device uses this model to reproduce the elderly's voice and simulate a conversation with a family member. The input is the elderly's voice data, and the output is the reproduced voice data.

[0481] Step 6:

[0482] The device analyzes the elderly person's tone of voice and speaking style in real time. The server estimates the emotional state using Python's OpenCV and Dlib libraries. The server reports the estimated emotional state to the family. The input is the elderly person's speech data, and the output is the report data of the estimated emotional state.

[0483] As a concrete example, if an elderly person says, "I feel lonely today," the device will send this statement to the server. The server will analyze this statement using an emotion engine and determine that the elderly person is feeling lonely. This information will be reported to the family as a summary. The family will receive this information and can take action, such as calling the elderly person.

[0484] (Application example 1)

[0485] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0486] There is a need for a system that ensures the safety and security of elderly people living alone and allows their families to understand their situation. In particular, there is a need for a means to quickly respond when elderly people feel lonely or face an emergency.

[0487] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for providing a speech ChatBot for communicating with elderly people living alone, a means for fine-tuning personal information and conversation habits of family members, a means for generating and speaking a conversation on behalf of the elderly person, a means for summarizing the conversation history and content with the elderly person and reporting it to the family, a means for reproducing the voice of the elderly person using a voice reproduction AI, a means for recognizing the emotional state of the elderly person and reporting the result to the family, and a means for detecting an abnormality and notifying an emergency contact. This makes it possible to quickly grasp the emotional state of the elderly person and emergency situations and provide appropriate information to the family and emergency contacts.

[0488] "Single-person elderly" refers to elderly people who live alone.

[0489] A "Speech ChatBot" is an artificial intelligence program that converses with people through voice.

[0490] "Family personal information" refers to information about individuals in family members, such as their names, ages, and relationships.

[0491] "Conversational habits" refers to the distinctive way a particular person speaks and chooses words.

[0492] "Fine-tuning" refers to the process of fine-tuning an existing machine learning model to fit specific data.

[0493] "Means of generating and speaking conversation" refers to the technology that creates natural conversation based on input information and outputs it as audio.

[0494] "Conversation history" refers to a record of past conversations.

[0495] A "summary" refers to a brief summary of a long piece of text or a conversation.

[0496] "Voice reproduction AI" refers to artificial intelligence technology that learns the characteristics of a specific person's voice and reproduces that voice.

[0497] "Emotional state" refers to a person's emotional state, such as happiness, sadness, anger, etc.

[0498] "Means of detecting anomalies" refers to technologies that identify unusual or emergency situations.

[0499] "Emergency Contact" refers to the person or organization to be contacted in the event of an emergency.

[0500] The system for implementing this invention provides a speech ChatBot for communicating with elderly people who live alone. The system includes means for fine-tuning personal information and conversational habits of family members, means for generating and speaking conversations on behalf of the elderly person, means for summarizing conversation history and content with the elderly person and reporting it to the family, means for reproducing the elderly person's voice using voice reproduction AI, means for recognizing the elderly person's emotional state and reporting the results to the family, and means for detecting abnormalities and notifying emergency contacts.

[0501] Hardware and Software Used

[0502] Hardware: Smartphone (microphone, internet connection)

[0503] Software: Python, speech recognition library, transformers library, smtplib library

[0504] Data processing and calculation

[0505] Speech Recognition

[0506] The server uses a voice recognition library to convert the voice input from the smartphone microphone into text. Through this process, the contents of the elderly person's speech are obtained as text data.

[0507] sentiment analysis

[0508] The server analyzes the emotional state of the acquired text data using the emotion analysis model of the transformers library, and classifies the elderly person's emotional state as "happiness," "sadness," "anger," etc.

[0509] Conversation generation and speech

[0510] The server uses a generative AI model to learn the family's personal information and conversational habits, and generates natural conversations. This generated conversation is then spoken in the elderly person's voice using a voice reproduction AI.

[0511] Conversation History Summary

[0512] The server uses natural language processing technology to summarize conversation histories with the elderly and reports them to the family, allowing the family to understand the elderly's situation.

[0513] Anomaly detection and notification

[0514] The server uses anomaly detection to send a notification to emergency contacts if the senior utters words like "help" or "emergency." The notification is sent via email using the smtplib library.

[0515] Examples

[0516] For example, if an elderly person says, "I feel a little lonely today," the sentiment analysis model will determine this as "negative," and notify the family member by email, "The elderly person feels lonely." Also, if the elderly person says, "Help me," an emergency notification will be sent to the emergency contacts, "The elderly person is requesting help."

[0517] Examples of prompt statements

[0518] An elderly person says, "I feel a little lonely today." The sentiment analysis model judges this to be "negative." An email is sent to the family saying, "An elderly person feels lonely."

[0519] In this way, a monitoring security app that provides safety and security to elderly people living alone has been realized.

[0520] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0521] Step 1:

[0522] The user (elderly person) speaks into the microphone of the smartphone. The input is the elderly person's voice, and the device collects this voice.

[0523] Step 2:

[0524] The terminal transmits the collected voice data to the server. The input is the voice data, and the output is the transfer of the voice data to the server.

[0525] Step 3:

[0526] The server converts the voice data into text data using a voice recognition library. The input is voice data and the output is text data. In concrete terms, the voice recognition algorithm analyzes the voice waveform and generates the corresponding text.

[0527] Step 4:

[0528] The server uses sentiment analysis models from the transformers library to analyze the emotional state of text data. The input is text data, and the output is an emotional state (e.g., happy, sad, angry). Specifically, the sentiment analysis algorithm analyzes the content of the text and assigns an emotional label.

[0529] Step 5:

[0530] The server uses a generative AI model to learn the family's personal information and conversational habits, and generates natural conversations. The input is the family's personal information, conversational habits, and emotional state, and the output is the generated conversation text. In concrete terms, the generative AI model generates new conversations based on past conversation data.

[0531] Step 6:

[0532] The server uses a voice reproduction AI to recite the generated conversation text in the elderly person's voice. The input is the generated conversation text, and the output is the reproduced voice. In concrete terms, the voice reproduction AI converts the text into voice and reproduces the characteristics of the elderly person's voice.

[0533] Step 7:

[0534] The server uses natural language processing technology to summarize the conversation history with the elderly person and reports it to the family. The input is the conversation history, and the output is the summarized conversation content. In concrete terms, the natural language processing algorithm analyzes the conversation history, extracts important information, and generates a summary.

[0535] Step 8:

[0536] The server uses anomaly detection to send a notification to emergency contacts if the elderly person utters words such as "help" or "emergency." The input is text data, and the output is an emergency notification. Specifically, the anomaly detection algorithm detects certain keywords and sends an email using the smtplib library.

[0537] Step 9:

[0538] The family receives the summarized conversation contents and emergency notifications sent from the server and understands the situation of the elderly person. The input is the notification from the server, and the output is the family's understanding. In concrete terms, the family checks emails and app notifications to understand the situation of the elderly person.

[0539] Example 2

[0540] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0541] There is a need to properly evaluate the dementia risk and emotional state of elderly people and report them to their families. It is also important to facilitate communication with elderly people and reduce their sense of loneliness. However, with conventional systems, it was difficult to comprehensively solve these issues.

[0542] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for collecting conversation data with the elderly person, a means for converting the collected voice data into text data, a means for analyzing the text data, a means for evaluating the dementia risk based on the analysis result, a means for evaluating the emotional state of the elderly person, and a means for adjusting the speech content according to the emotional state. This makes it possible to appropriately evaluate the dementia risk and emotional state of the elderly person and report them to the family. In addition, it is possible to facilitate communication with the elderly person and reduce the feeling of loneliness.

[0543] "Single-person elderly" refers to elderly people who live alone.

[0544] "Speech device" refers to a device for generating speech and interacting with a user.

[0545] "Personal information" refers to information that can identify a specific individual.

[0546] "Conversational habits" refers to the conversational patterns or characteristics that a particular individual possesses.

[0547] "Adjust" means to change or optimize something to suit specific conditions or purposes.

[0548] "Generate" means to create something new.

[0549] "Speaking" refers to uttering words through sound.

[0550] To summarise means to briefly summarise the main points.

[0551] "Voice reproduction technology" refers to technology that imitates and generates the voice of a specific person.

[0552] "Conversation data" refers to data that records the contents of a conversation.

[0553] "Audio data" refers to data that records audio in digital format.

[0554] "Text data" refers to data recorded as character information.

[0555] "Analyze" means to examine data in detail and extract meaning and patterns.

[0556] "Dementia risk" refers to the possibility of developing dementia.

[0557] "Emotional state" refers to an individual's emotional state at a particular point in time.

[0558] "Adjust" means to change or optimize something to suit specific conditions or purposes.

[0559] "Loneliness" refers to the feeling of lacking connection with others.

[0560] The present invention provides a speech device for communicating with elderly people who live alone, and is a system for evaluating the dementia risk and emotional state of the elderly and reporting the results to their families. A specific embodiment of the system will be described below.

[0561] System Configuration

[0562] The system consists of the following main components:

[0563] 1. Speech devices: Devices for communicating with the elderly, such as smart speakers and tablets.

[0564] 2. Server: A central processing unit that collects, analyzes, and evaluates data.

[0565] 3. Family devices: Devices used to report on the elderly person's condition, such as smartphones and computers.

[0566] Hardware and Software Used

[0567] Speech Recognition Software: Uses the Google Speech-to-Text API to convert voice data into text data.

[0568] Natural Language Processing (NLP) techniques: Use spaCy and NLTK to analyze text data.

[0569] Machine learning models: Algorithms for assessing dementia risk.

[0570] Emotion Engine: IBM Watson® Tone Analyzer is used to assess the emotional state of the elderly.

[0571] System Operation

[0572] The user (elderly person) starts the system by speaking into the speech device. For example, they might say, "Hello, how was your day?" The device records this conversation and saves it as voice data. The device then transmits the recorded voice data to the server.

[0573] The server converts the voice data into text data using the Google Speech-to-Text API. The converted text data is then analyzed using spaCy. The analysis results detect whether the elderly person is repeating themselves or is having difficulty with everyday conversation.

[0574] Next, the server uses a machine learning model to assess the risk of dementia based on the analysis results. If the risk is determined to be high, the server notifies the family member's device of that information. For example, it may notify the family member, "The elderly person is repeating the same story. There is a possibility that the risk of dementia is high."

[0575] In addition, the server uses IBM Watson Tone Analyzer to evaluate the emotional state of the elderly. If it is determined that the elderly would be happy to hear stories about their grandchildren, the server adjusts the speech device's speech content to "Tell me a story about your grandchild." The speech device speaks to the elderly, "Tell me a story about your grandchild."

[0576] Examples of concrete examples and prompts

[0577] Examples:

[0578] Elderly person A begins a conversation with the speech device.

[0579] The device records Person A's conversation and saves it as audio data.

[0580] The server converts the voice data into text data using the Google Speech-to-Text API.

[0581] The server analyzes the text data using spaCy and detects that Person A is repeating the same story.

[0582] The server uses a machine learning model to assess Mr. A's risk of dementia and determines that he is at high risk.

[0583] The server notifies the family, "An elderly person is repeating the same story. They may be at high risk of dementia."

[0584] The server evaluates Mr. A's emotional state using IBM Watson Tone Analyzer and determines that Mr. A would be happy if he talked about his grandchildren.

[0585] The speech device speaks to Mr. A, saying, "Tell me about your grandchild."

[0586] Example prompt:

[0587] Create a program that collects conversations with elderly people and assesses their dementia risk. Convert the conversations into text data using the Google Speech-to-Text API and analyze them using spaCy. Based on the analysis results, assess the dementia risk using a machine learning model and evaluate the emotional state using IBM Watson Tone Analyzer. Add a function to adjust the speech content according to the emotional state.

[0588] In this way, the system can properly assess the dementia risk and emotional state of the elderly and report it to their families, facilitating communication with the elderly and reducing their sense of loneliness.

[0589] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0590] Step 1:

[0591] The user (elderly person) speaks to the speech device. For example, he / she says, "Hello, how was your day?" The input is the elderly person's voice, and the output is the recorded voice data. The terminal records this conversation and saves it as voice data.

[0592] Step 2:

[0593] The terminal sends the recorded voice data to the server. The input is the recorded voice data, and the output is the voice data sent to the server. The server converts the voice data to text data using the Google Speech-to-Text API. The input is the voice data, and the output is the converted text data.

[0594] Step 3:

[0595] The server analyzes the converted text data using natural language processing (NLP) technology (e.g., spaCy). The input is the text data, and the output is the analysis results. The server analyzes the conversation patterns and frequency to detect whether the elderly person is repeating the same things or is having difficulty in everyday conversation.

[0596] Step 4:

[0597] The server uses a machine learning model based on the analysis results to assess the risk of dementia. The input is the analysis result, and the output is the dementia risk assessment result. If the server determines that the risk is high, it notifies the family member's device of that information. For example, it may notify, "An elderly person keeps repeating the same thing. There is a possibility that they are at high risk of dementia."

[0598] Step 5:

[0599] The server evaluates the emotional state of the elderly person using IBM Watson Tone Analyzer. The input is text data, and the output is the evaluation result of the emotional state. The server judges whether the elderly person is feeling joy or pleasure.

[0600] Step 6:

[0601] The server adjusts the speech content of the speech device based on the evaluation result of the emotion engine. The input is the evaluation result of the emotional state, and the output is the adjusted speech content. The speech device speaks to the elderly person, saying, "Tell me a story about your grandchild."

[0602] (Application example 2)

[0603] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0604] It is necessary to protect the safety and health of the elderly by properly assessing their dementia risk and emotional state and promptly notifying their families and caregivers. In addition, there is a need for communication methods that allow the elderly to live their daily lives with peace of mind without feeling lonely.

[0605] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0606] In this invention, the server includes a means for providing a speech ChatBot for communicating with elderly people living alone, a means for adjusting personal information and conversation habits of family members, a means for generating and speaking a conversation on behalf of the elderly person, a means for summarizing the conversation history and content with the elderly person and reporting it to the family, a means for reproducing the elderly person's voice using voice reproduction technology, a means for analyzing the elderly person's emotional state and providing appropriate topics, and a means for assessing dementia risk and notifying the family. This makes it possible to appropriately assess the elderly person's dementia risk and emotional state and quickly notify the family or caregiver.

[0607] "Speech ChatBot" is a conversational system using artificial intelligence designed to interact with the elderly.

[0608] "Personal information" refers to information about family members and seniors, such as name, age, address, and contact details.

[0609] "Conversational habits" refers to the words and speaking patterns that older people use on a daily basis.

[0610] "Tune" refers to changing settings or parameters to suit a particular purpose.

[0611] "Generating and speaking a conversation" refers to the artificial intelligence creating appropriate responses for the elderly and outputting them as voice.

[0612] "Conversation history" refers to records of past conversations between seniors and the speech ChatBot.

[0613] "Summarizing" refers to summarizing a long conversation in a concise manner.

[0614] "Voice reproduction technology" refers to technology that imitates and generates the voice of a specific person.

[0615] "Emotional state" refers to emotions such as happiness, sadness, and anger felt by older adults.

[0616] "Appropriate topics" refer to conversation topics chosen to suit the emotional state and interests of the elderly.

[0617] "Dementia risk" refers to an indicator that assesses the likelihood that an elderly person will develop dementia.

[0618] "Inform" refers to giving specific information to family members or caregivers.

[0619] The system for implementing this invention is configured as follows: The server includes a means for providing a speech ChatBot for communicating with elderly people living alone, a means for adjusting personal information and conversational habits of family members, a means for generating and speaking a conversation on behalf of the elderly person, a means for summarizing the conversation history and content with the elderly person and reporting it to the family, a means for reproducing the elderly person's voice using voice reproduction technology, a means for analyzing the emotional state of the elderly person and providing appropriate topics, and a means for assessing the risk of dementia and notifying the family member.

[0620] System program processing

[0621] The server first records the conversation between the elderly person and the speech ChatBot. The recorded conversation content is subjected to sentiment analysis using a natural language processing library (e.g., NLTK or spaCy). Based on the results of the sentiment analysis, the emotion engine determines the elderly person's emotional state and provides appropriate topics.

[0622] The DementiaRiskAssessment module then analyzes conversation patterns and frequency to assess dementia risk, and if the assessment result exceeds a certain risk, a notification service is used to notify family members.

[0623] In addition, conversation histories are periodically summarized and reported to family members, allowing them to keep track of the elderly person's condition.

[0624] Hardware and Software Used

[0625] Hardware: Smartphone

[0626] Software: Python, natural language processing libraries (NLTK, spaCy), notification services

[0627] Examples

[0628] For example, if an elderly person says, "My grandchildren came to visit me today," the server records this conversation and performs sentiment analysis. If the emotion of joy is detected, the emotion engine generates a response such as, "That's wonderful! Tell me more about your grandchildren."

[0629] You can also input prompts like the following into the generative AI model to generate appropriate responses:

[0630] Example prompt:

[0631] If an elderly person says, "My grandchild came to visit me today," perform sentiment analysis and, if the emotion of joy is detected, generate a response that continues the topic of the grandchild.

[0632] In this way, appropriate communication according to the emotional state of the elderly person becomes possible, and dementia risk can also be assessed and notified to family members.

[0633] The flow of the identification process in the application example 2 will be described with reference to FIG.

[0634] Step 1:

[0635] The server records the conversation between the elderly person and the Talking ChatBot. As input, the elderly person's speech is provided as voice data. The server converts this voice data into text data and adds it to the conversation history. As output, the conversation data in text format is obtained.

[0636] Step 2:

[0637] The server performs sentiment analysis on the recorded conversation content using natural language processing libraries (NLTK and spaCy). Text-formatted conversation data is provided as input. The server analyzes this data and determines the emotional state (happiness, sadness, anger, etc.). The output is a label for the emotional state.

[0638] Step 3:

[0639] The server uses an emotion engine to provide appropriate topics based on the elderly person's emotional state. As input, the label of the emotional state is provided. Based on this label, the server inputs a prompt sentence into the generative AI model to generate an appropriate response. As output, the generated response text is obtained.

[0640] Step 4:

[0641] The server converts the generated response text into voice data and speaks it to the elderly person through the speech ChatBot. The response text is provided as input. The server converts the text into voice using voice reproduction technology. Voice data is obtained as output.

[0642] Step 5:

[0643] The server analyzes the conversation patterns and frequency and assesses the dementia risk using the DementiaRiskAssessment module. As input, the conversation history is provided. The server analyzes this data and assesses the dementia risk. As output, the risk assessment result is obtained.

[0644] Step 6:

[0645] The server uses a notification service to send notifications to family members when the dementia risk assessment result exceeds a certain risk. As input, the risk assessment result is provided. The server generates notification content and sends the notification to the family contact. As output, it confirms that the notification has been sent.

[0646] Step 7:

[0647] The server periodically summarizes the conversation history and reports it to the family. As input, the conversation history is provided. The server uses a summarization algorithm to summarize the long conversation content concisely. As output, the summarized conversation content is obtained and reported to the family.

[0648] Example 3

[0649] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0650] There is a need to assess the loneliness and emotional state of elderly people in real time and provide appropriate information to their families. However, with conventional systems, it was difficult to accurately assess the emotions and loneliness of elderly people and quickly report them to their families. In addition, there was a lack of means to effectively collect and analyze conversation data with elderly people and encourage appropriate responses. This made it difficult to provide appropriate support to maintain the mental health of elderly people.

[0651] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0652] In this invention, the server includes a means for collecting conversation data with the elderly person, a means for transmitting the collected conversation data to the server, a means for analyzing the conversation data and evaluating the sense of loneliness, a means for evaluating the emotional state, a means for storing the evaluation result in a database, and a means for notifying the family of the evaluation result. This makes it possible to evaluate the sense of loneliness and the emotional state of the elderly person in real time and to report the results to the family quickly and accurately.

[0653] "Single-person elderly" refers to elderly people who live alone.

[0654] A "speech system" refers to a system that allows humans to communicate through voice or text.

[0655] "Family personal information" refers to information about family members, such as their names, addresses, and contact details.

[0656] "Conversational habits" refers to a particular person's way of speaking and using words.

[0657] "Means for adjusting" refers to the way in which the system changes and optimizes certain conditions or parameters.

[0658] "Means for generating and speaking conversation" refers to a method by which the system automatically creates conversation content and outputs it as voice or text.

[0659] "Means of summarizing conversation history and content" refers to a method of organizing past conversation data, extracting important points, and summarizing them concisely.

[0660] "Voice reproduction technology" refers to technology that imitates and generates the voice of a specific person.

[0661] "Means of collecting conversation data" refers to methods of recording conversations with seniors and storing that data.

[0662] "Means of transmitting to the server" refers to the method of transferring collected data to the server via a network.

[0663] "Means for analyzing conversation data" refers to methods for processing collected conversation data and understanding its meaning and sentiment.

[0664] "Measures for assessing loneliness" refers to methods for measuring and assessing the loneliness of older adults from conversation data.

[0665] "Means for assessing emotional state" refers to a method for analyzing the emotions of elderly people from conversation data and assessing their state.

[0666] "Means of storing in a database" refers to the method of recording and storing the analysis results and evaluation results in a database.

[0667] "Means of notifying family members" refers to the method by which the results of the evaluation will be communicated to family members.

[0668] This invention relates to a system for evaluating the loneliness and emotional state of elderly people and reporting it to their families. Specifically, this invention is implemented using hardware and software such as a speech system, an emotion engine, a database, and a notification system.

[0669] Hardware and Software Used

[0670] Hardware: Servers, terminals (devices used by the elderly)

[0671] Software: speech system, emotion engine, database, notification system

[0672] System Overview

[0673] This system assesses the elderly person's sense of loneliness and emotional state and reports back to their family members using the following steps:

[0674] 1. Collecting conversation data with elderly people

[0675] The terminal is a device used by elderly people and has a microphone and text input function.

[0676] The device collects conversations with elderly people in real time.

[0677] For example, if an elderly person says, "I haven't talked to anyone today," the device will collect this voice data.

[0678] 2. Sending conversation data to the server

[0679] The terminal transmits the collected conversation data to a server via the Internet.

[0680] Data is encrypted and transmitted using secure communication protocols (e.g. HTTPS).

[0681] 3. Analysis of conversation data

[0682] The server passes the received conversation data to the speech system.

[0683] The speech system uses natural language processing (NLP) technology to convert conversational content into text and analyze it.

[0684] For example, a statement such as "I haven't talked to anyone today" is analyzed as text data.

[0685] 4. Assessment of loneliness

[0686] The server evaluates the elderly person's sense of loneliness based on the results of the speech system's analysis.

[0687] The survey checks indicators such as whether elderly people frequently use words that express loneliness and whether the frequency of conversations is decreasing.

[0688] For example, the statement "I'm not talking to anyone" is evaluated as an indicator of loneliness.

[0689] 5. Evaluating emotional state

[0690] The server uses an emotion engine to assess the emotional state of the senior citizen.

[0691] The emotion engine extracts emotions from the elderly's speech and determines their emotional state, such as sadness or anxiety.

[0692] For example, the word "worry" is recognized by the emotion engine as an expression of anxiety.

[0693] 6. Saving the evaluation results to a database

[0694] The server stores the evaluation results of loneliness and emotional state in a database.

[0695] The data stored includes the date and time of the evaluation, the content of the evaluation, specific comments, etc.

[0696] For example, data such as "October 10, 2023, loneliness rating: high, anxiety rating: medium" would be saved.

[0697] 7. Notifying Family Members

[0698] The server passes the evaluation results to the notification system.

[0699] The notification system will report the evaluation results to family members via email or app notifications.

[0700] For example, the family may be notified with something like, "Your mother seems to be feeling lonely lately. Specific comment: 'I haven't talked to anyone today.'"

[0701] Examples

[0702] For example, if an elderly person says, "I haven't talked to anyone today," the speech system will interpret this as an indicator of loneliness. If the elderly person frequently uses words like "worry," the emotion engine will determine that the elderly person is feeling anxious. This information is then reported to the family, who can understand the elderly person's condition and take appropriate action.

[0703] Examples of prompt statements

[0704] "Assess what emotions older adults have expressed in recent conversations and determine whether they are feeling lonely or anxious."

[0705] This system makes it possible to evaluate the elderly person's sense of loneliness and emotional state in real time and report it to their family members, thereby encouraging them to take appropriate measures. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0706] Step 1:

[0707] The device collects conversation data with elderly people.

[0708] Specifically, the device uses a microphone and text input function to record what the elderly person says in real time.

[0709] Input: Senior's voice or text data

[0710] Output: Collected conversation data (audio file or text data)

[0711] Step 2:

[0712] The terminal transmits the collected conversation data to the server.

[0713] Specifically, the terminal encrypts data transmitted over the Internet and transmits it to the server using a secure communication protocol (e.g., HTTPS).

[0714] Input: Collected conversation data

[0715] Output: Conversation data sent to the server

[0716] Step 3:

[0717] The server passes the received conversation data to the speech system.

[0718] Specifically, the server inputs conversation data into the speech system, converts it into text using natural language processing (NLP) technology, and performs analysis.

[0719] Input: Conversation data sent to the server

[0720] Output: Conversation data converted to text

[0721] Step 4:

[0722] The server evaluates the elderly person's sense of loneliness based on the results of the speech system's analysis.

[0723] Specifically, the server checks the frequency of words and conversations that express loneliness in the text data and calculates an index of loneliness.

[0724] Input: Conversation data converted to text

[0725] Output: Loneliness evaluation result

[0726] Step 5:

[0727] The server uses an emotion engine to assess the emotional state of the senior citizen.

[0728] Specifically, the server extracts emotions from the text data and determines the emotional state, such as sadness or anxiety.

[0729] Input: Conversation data converted to text

[0730] Output: Emotional state evaluation result

[0731] Step 6:

[0732] The server stores the evaluation results of loneliness and emotional state in a database.

[0733] Specifically, the server records data such as the evaluation date and time, the evaluation content, and specific comments in a database.

[0734] Input: Loneliness assessment results, emotional state assessment results

[0735] Output: Evaluation results stored in a database

[0736] Step 7:

[0737] The server passes the evaluation results to the notification system.

[0738] Specifically, the server enters the evaluation results into a notification system and reports them to family members via email, app notifications, etc.

[0739] Input: Evaluation results stored in the database

[0740] Output: Evaluation results communicated to family members

[0741] (Application example 3)

[0742] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0743] The challenge is to ensure the safety and security of the elderly by properly assessing their loneliness and emotional state and quickly notifying their family and nursing care services. In particular, when an elderly person living alone feels lonely or anxious, there is a need for a means to quickly convey that information to their family. There is also a need for a system that can accurately analyze the content of the elderly's comments and their emotional state and encourage appropriate responses.

[0744] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means. In this invention, the server includes a means for providing a speech ChatBot for interacting with the elderly person living alone, a means for adjusting the personal information and conversation habits of the family, a means for generating and speaking a conversation on behalf of the elderly person, a means for summarizing the conversation history and contents with the elderly person and reporting them to the family, a means for reproducing the voice of the elderly person using voice reproduction technology, a means for evaluating the elderly person's sense of loneliness, a means for evaluating the elderly person's emotional state and notifying the family, a means including an application to be installed on a smartphone, a natural language processing means for analyzing the contents of the elderly person's remarks, a means for evaluating the elderly person's emotional state using an emotion analysis engine, and a means for notifying the family of the evaluation result using a notification service. This makes it possible to accurately evaluate the elderly person's sense of loneliness and emotional state and quickly notify the family member or care service.

[0745] "Speech ChatBot" is an artificial intelligence system that engages in voice conversations to interact with the elderly.

[0746] "Family personal information" is information about the elderly person's family members, including names, ages, relationships, etc.

[0747] "Methods for adjusting conversational habits" are techniques for imitating the speaking styles and language used by family members when conversing with elderly people.

[0748] "Means for generating and speaking a conversation" refers to technology for automatically generating a dialogue with an elderly person and speaking it aloud.

[0749] "Means of summarizing conversation history and content and reporting it to family members" refers to technology for recording the content of conversations with elderly people, summarizing it, and notifying family members.

[0750] "Voice reproduction technology" is a technology that imitates and reproduces the voice of a specific person.

[0751] The "means for assessing loneliness" is a technology for detecting and assessing loneliness from the content of elderly people's statements and frequency of conversations.

[0752] "Means for assessing emotional state and notifying family members" refers to technology for analyzing the emotions of elderly people and notifying family members of the results.

[0753] An "application installed on a smartphone" is a software program that runs on a smartphone.

[0754] "Natural language processing means" refers to technology for analyzing voice and text data and understanding their meaning.

[0755] A "sentiment analysis engine" is a technology for analyzing emotions from text and voice data.

[0756] "Notification service" refers to technology for notifying specific recipients of evaluation results or important information.

[0757] The system for implementing this invention is configured as follows. First, the server provides a speech ChatBot for interacting with elderly people who live alone. The speech ChatBot is used to evaluate the sense of loneliness and emotional state through conversation with elderly people. The server also has a means for adjusting personal information and conversation habits of family members, and has a function for generating and speaking conversations with elderly people.

[0758] Hardware and Software Used

[0759] Hardware: Smartphone, microphone, internet connection

[0760] Software: Natural language processing libraries (e.g. Google Cloud Natural Language API), sentiment analysis engines (e.g. IBM Watson Tone Analyzer), notification services (e.g. messaging services)

[0761] Data processing and calculation

[0762] 1. Getting voice input:

[0763] The user (elderly person) inputs voice using the microphone on the smartphone.

[0764] The server uses a speech recognition API (e.g. Google Cloud Speech-to-Text) to convert the voice data into text.

[0765] 2. Natural Language Processing:

[0766] The server analyzes the acquired text data using a natural language processing library.

[0767] Extract the content, frequency, and emotional expressions of elderly people's statements.

[0768] 3. Sentiment analysis:

[0769] The server uses a sentiment analysis engine to assess the emotional state of the senior citizen.

[0770] Determine whether the elderly person is expressing emotions such as "worried" or "sad."

[0771] 4. Loneliness assessment:

[0772] The server evaluates the elderly person's sense of loneliness based on the content and frequency of their comments and their emotional expressions.

[0773] An index of loneliness is calculated, and if it exceeds a certain threshold, it is determined that the sense of loneliness is high.

[0774] 5. Notice:

[0775] The server notifies the evaluation results of loneliness and emotional state to family members and care services.

[0776] Use the notification service to send push notifications to your smartphone.

[0777] Examples

[0778] If an elderly person says, "I haven't talked to anyone lately," the server will pick up on this statement and determine that the person is feeling very lonely.

[0779] If an elderly person says "I'm worried," the server will use a sentiment analysis engine to capture the statement and determine that the elderly person is feeling anxious.

[0780] Examples of prompt statements

[0781] Please enter what the senior said:

[0782] "I haven't talked to anyone lately."

[0783] "anxiety"

[0784] In this way, the server can accurately assess the elderly person's loneliness and emotional state and quickly notify family members and care services.

[0785] The flow of the specific process in the application example 3 will be described with reference to FIG.

[0786] Step 1:

[0787] The user (elderly person) inputs voice using the microphone of the smartphone. The input voice data is sent to the smartphone.

[0788] Step 2:

[0789] The terminal (smartphone) uses a speech recognition API (e.g. Google Cloud Speech-to-Text) to convert voice data into text. It sends the voice data to the API and obtains the text data. The input is voice data and the output is text data.

[0790] Step 3:

[0791] The server analyzes the acquired text data using a natural language processing library (e.g., Google Cloud Natural Language API). The text data is sent to the API, which extracts the content of the speech, frequency, and emotional expression. The input is the text data, and the output is the analysis results.

[0792] Step 4:

[0793] The server uses a sentiment analysis engine (e.g., IBM Watson Tone Analyzer) to evaluate the emotional state of the elderly person. The analysis result is sent to the sentiment analysis engine to evaluate the emotional state. The input is the analysis result, and the output is the emotion evaluation result.

[0794] Step 5:

[0795] The server evaluates the elderly person's sense of loneliness based on the content and frequency of their comments and their emotional expressions. It calculates an index of loneliness based on the emotion evaluation result, and judges the sense of loneliness to be high if it exceeds a certain threshold. The input is the emotion evaluation result, and the output is the loneliness evaluation result.

[0796] Step 6:

[0797] The server notifies the family or care service of the evaluation results of loneliness and emotional state. It uses a notification service (e.g., messaging service) to send a push notification to the smartphone. The input is the loneliness evaluation result and the emotion evaluation result, and the output is the notification message.

[0798] Step 7:

[0799] Family members or care services can receive notification messages via smartphones and understand the elderly person's feelings of loneliness and emotional state. They can take appropriate action based on the notification messages. The input is the notification message, and the output is the corresponding action.

[0800] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0801] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0802] Another example of generative AI is Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0803] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0804] [Second embodiment]

[0805] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0806] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0807] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0808] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0809] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0810] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0811] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0812] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0813] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0814] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0815] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0816] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0817] "Example 1"

[0818] As an example embodiment of the present invention, a speech ChatBot for interacting with elderly people who live alone is provided. This ChatBot is fine-tuned using a machine learning algorithm for learning personal information and conversational habits of family members. Specifically, the family's conversation data is used as learning data, its features are extracted, and conversations are generated and spoken based on that. In addition, the conversation history and content with the elderly are summarized using natural language processing technology and reported to the family. Furthermore, a voice reproduction AI learns the elderly's voice data, extracts its features, and reproduces the elderly's voice. This allows the elderly to feel as if they are interacting with their family. "Example 2"

[0819] In another embodiment of the present invention, the speech ChatBot has a function to evaluate the dementia risk of elderly people. Specifically, the ChatBot detects signs of dementia from the content and frequency of conversations with elderly people, conversation patterns, etc. For example, if an elderly person shows symptoms such as repeating the same things, having difficulty in everyday conversations, or having difficulty finding words, the ChatBot evaluates this as a risk of dementia and reports it to the family.

[0820] "Example 3"

[0821] In another embodiment of the present invention, the speech ChatBot has a function to evaluate the elderly person's sense of loneliness. Specifically, the sense of loneliness is detected from the content and frequency of conversations with the elderly person, emotional expressions, etc. For example, if the elderly person frequently uses words expressing loneliness, does not express emotions much, or talks less frequently, the ChatBot evaluates these as indicators of loneliness and reports them to the family.

[0822] The process flow of each embodiment will be described below.

[0823] "Example 1"

[0824] Step 1: Speech ChatBot begins fine-tuning using machine learning algorithms to learn your family's personal information and conversational habits.

[0825] Step 2: Use family conversation data as learning data and extract its features.

[0826] Step 3: Generate and speak a conversation based on the extracted features.

[0827] Step 4: Summarize the conversation history and content with the elderly person using natural language processing technology.

[0828] Step 5: Report the summary to your family.

[0829] Step 6: The voice reproduction AI learns the voice data of the elderly and extracts its features.

[0830] Step 7: Reproduce the elderly person's voice using the extracted features.

[0831] "Example 2"

[0832] Step 1: The speech ChatBot will begin assessing the dementia risk of the elderly. Step 2: Detect signs of dementia from the content, frequency, and conversation patterns of conversations with the elderly.

[0833] Step 3: Report the dementia risk assessment results to family members.

[0834] "Example 3"

[0835] Step 1: The speech ChatBot starts the function of assessing the elderly person's loneliness.

[0836] Step 2: Detect loneliness based on the content and frequency of conversations with elderly people, as well as emotional expressions.

[0837] Step 3: Report the results of the assessment as an index of loneliness to family members.

[0838] Example 1

[0839] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0840] There is a need for a system that allows elderly people living alone to continue to interact with their families without feeling lonely. It is also important to evaluate the elderly's dementia risk and loneliness and provide appropriate information to their families. However, the current system is not able to fully resolve these issues.

[0841] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0842] In this invention, the server includes a means for collecting family conversation data, a means for pre-processing the collected data, a means for fine-tuning a machine learning model, a means for generating a conversation with the elderly person, a means for reproducing the voice of the elderly person, a means for summarizing a conversation history with the elderly person, and a means for reporting the summarized conversation history to the family member. This enables the elderly person living alone to have a natural conversation with the family member, reduce loneliness, and provide appropriate information to the family member through an evaluation of dementia risk and loneliness.

[0843] "Family conversation data" refers to text messages and voice data exchanged between family members and elderly people.

[0844] "Preprocessing" refers to the process of removing unnecessary information from collected data and converting it into a form suitable for machine learning models.

[0845] "Fine-tuning a machine learning model" refers to a technique for improving the performance of an existing machine learning model by performing additional training using a specific dataset.

[0846] "Conversation generation" refers to the process of automatically generating natural-sounding conversations using machine learning models.

[0847] "Voice reproduction" refers to the technology of reproducing a specific person's voice based on collected voice data.

[0848] "Conversation history summary" refers to the process of summarizing conversations with elderly people and extracting key points.

[0849] "Reporting to family" refers to the act of notifying family members of the summarized conversation history.

[0850] This invention relates to a speech ChatBot system for interacting with elderly people who live alone. This system collects family conversation data, generates natural conversations with elderly people using machine learning models, and reports the conversation history to the family.

[0851] Hardware and Software Used

[0852] The server uses the following hardware and software:

[0853] Hardware: A server machine equipped with a high-performance CPU and GPU

[0854] Software: Python, TensorFlow, PyTorch, NLTK, audio processing libraries

[0855] The terminal is a device used by the elderly and uses the following hardware and software:

[0856] Hardware: Microphone, speaker, tablet or smartphone

[0857] Software: Voice recognition software, voice synthesis software

[0858] Data processing and calculation

[0859] The server collects conversation data of family members. The collected data includes text messages and voice data. For example, messages sent by family members using smartphones or computers and recorded data of phone calls are stored in cloud storage.

[0860] Next, the server preprocesses the collected data. Specifically, it cleans the text data (removes unnecessary spaces and special characters) and removes noise from the audio data. It uses the Python library NLTK to tokenize the text data and a speech processing library to remove noise from the audio data.

[0861] The server then uses the preprocessed data to fine-tune a machine learning model. The model used is, for example, OpenAI's GPT-3 or Google's BERT. Family conversation data is input into the model, and the model is trained to learn the characteristics of family conversations.

[0862] ChatBot uses the fine-tuned model to generate conversations with the elderly. For example, if an elderly person asks, "What shall we do today?", ChatBot responds, "The weather is nice today, so how about going for a walk?"

[0863] In addition, the server learns the voice data of the elderly and extracts its features, which allows the ChatBot to reproduce the voice of the elderly. Using voice reproduction AI, it learns the features of the elderly's voice and speaks the generated text in the elderly's voice.

[0864] The server summarizes the conversation history with the elderly person using natural language processing technology, extracts important points and topics of the conversation, and generates a concise summary. The server summarizes the conversation content using tools such as the BERT summarizer.

[0865] Finally, the server reports the summarized conversation history to the family. It notifies the family of the elderly person's situation and the contents of the conversation via email or a dedicated app. The summarized text is sent as the body of the email.

[0866] Examples

[0867] For example, when a ChatBot wants to have a conversation with an elderly person about "today's weather," it inputs the following prompt sentence into the generative AI model.

[0868] Example prompt:

[0869] "Good morning. How's the weather today?"

[0870] Based on this prompt, ChatBot will generate the following conversation:

[0871] An example of the generated conversation:

[0872] ChatBot: Good morning. It's a sunny day today. How about going for a walk?

[0873] Elderly person: Yes, I think I'll go outside for the first time in a while.

[0874] In this way, ChatBot can have natural conversations with the elderly and deepen their interactions. The conversation history is summarized by the server and reports to the family, "It's sunny today, so the elderly are going to go for a walk."

[0875] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0876] Step 1:

[0877] Data collection

[0878] The server collects family conversation data. As input, it receives text messages and recorded calls sent by family members using their smartphones and computers. These data are stored in cloud storage. As output, it obtains the collected text and audio data.

[0879] Step 2:

[0880] Data Preprocessing

[0881] The server preprocesses the collected data. As input, it receives the text data and audio data collected in step 1. Specifically, it cleans the text data (removes unnecessary spaces and special characters) and removes noise from the audio data. It tokenizes the text data using the Python library NLTK and removes noise from the audio data using an audio processing library. As output, it obtains the preprocessed text data and audio data.

[0882] Step 3:

[0883] Fine-tuning machine learning models

[0884] The server uses the preprocessed data to fine-tune the machine learning model. As input, it receives the text data and audio data preprocessed in step 2. Specifically, it uses models such as OpenAI's GPT-3 and Google's BERT, and inputs family conversation data into the model to train it. As output, it obtains a fine-tuned machine learning model.

[0885] Step 4:

[0886] Conversation Generation

[0887] ChatBot uses the fine-tuned model to generate conversations with elderly people. As input, it receives questions and utterances from the user (elderly person). In concrete terms, it inputs prompt sentences into the generative AI model and generates natural conversations. For example, if an elderly person asks, "What shall we do today?", ChatBot responds with, "The weather is nice today, so how about going for a walk?" The generated conversation is obtained as output.

[0888] Step 5:

[0889] Audio Reproduction

[0890] The server learns the voice data of the elderly and extracts its features. As input, it receives the voice data preprocessed in step 2. Specifically, it uses a voice reproduction AI to learn the features of the elderly's voice and speaks the generated text in the elderly's voice. As output, a voice reproduced in the elderly's voice is obtained.

[0891] Step 6:

[0892] Conversation History Summary

[0893] The server summarizes the conversation history with the elderly person using natural language processing technology. As input, it receives the conversation history generated in step 4. Specifically, it summarizes the conversation content using tools such as the BERT summarizer and extracts important points and topics. As output, it obtains the summarized conversation history.

[0894] Step 7:

[0895] Reporting to family

[0896] The server reports the summarized conversation history to the family. As input, it receives the summarized conversation history from step 6. Specifically, it notifies the family of the elderly person's situation and the contents of the conversation via email or a dedicated app. As output, it obtains the report sent to the family.

[0897] (Application example 1)

[0898] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0899] In modern society, the increase in the number of elderly people living alone is becoming a serious problem. Elderly people living alone are particularly prone to feeling lonely and being at higher risk of dementia, and reduced interaction with family members can have a negative impact on their mental and physical health. In addition, if family members live far away, it is difficult to keep track of the elderly's condition. To solve these problems, it is necessary to provide an environment where the elderly can live with peace of mind, and a system that allows family members to properly keep track of the elderly's condition.

[0900] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0901] In this invention, the server includes a means for providing a speech ChatBot for communicating with elderly people living alone, a means for fine-tuning personal information and conversation habits of family members, a means for generating and speaking conversations on behalf of the elderly person, a means for summarizing conversation history and content with the elderly person and reporting it to the family, a means for reproducing the elderly person's voice using a voice reproduction AI, a means for voice recognition of the elderly person's speech, a means for converting the generated conversation into voice and playing it back, and a means for detecting abnormalities and notifying family members and emergency contacts. This allows the elderly person to feel as if they are interacting with their family without feeling lonely, and further allows the family member to properly grasp the elderly person's condition and quickly respond if an abnormality occurs.

[0902] "Single-person elderly" refers to elderly people who live alone.

[0903] A "Speech ChatBot" refers to artificial intelligence that can converse with humans through voice.

[0904] "Family personal information" refers to information about family members such as names, ages, and relationships.

[0905] "Conversational habits" refers to the way a particular person speaks or uses language.

[0906] "Fine-tuning" refers to the fine-tuning of an existing machine learning model to adapt it to a specific dataset.

[0907] "Means for generating and speaking a conversation" refers to technology that creates an appropriate response based on input information and outputs it as voice.

[0908] "Conversation history" refers to a record of past conversations.

[0909] A "summary" refers to summarizing a longer piece of text or the contents of a conversation.

[0910] "Voice reproduction AI" refers to artificial intelligence technology that generates a voice that imitates a specific person's voice.

[0911] "Speech recognition" refers to the technology that converts speech into text.

[0912] "Means for converting to audio and playing it back" refers to technology for converting text data into audio data and playing it back.

[0913] "Means of detecting abnormalities and notifying family members or emergency contacts" refers to technology that detects abnormalities in an elderly person's condition and notifies family members or designated contacts.

[0914] As an embodiment of the present invention, we provide a ChatBot system for communicating with elderly people who live alone. The system is configured using the following hardware and software.

[0915] Hardware

[0916] Smartphone: Equipped with a microphone and speaker for audio input and output.

[0917] Server: A powerful computer that processes and stores data.

[0918] software

[0919] Python: A programming language used to implement programs.

[0920] OpenAI API: An API for generating conversations using generative AI models.

[0921] Speech Recognition Library: A library for converting speech to text.

[0922] gTTS (Google Text-to-Speech): A library for converting text to speech.

[0923] Playback software: Software for playing audio files.

[0924] System Operation

[0925] 1. Voice recognition: The device (smartphone) uses a microphone to record the elderly person's speech and converts the speech into text using a voice recognition library.

[0926] 2. Conversation generation: The server uses the OpenAI API to generate appropriate responses using a generative AI model that has learned the family's personal information and conversational habits.

[0927] 3. Audio playback: The device uses gTTS to convert the generated text into audio and plays it over the speaker.

[0928] 4. Conversation summary: The server summarizes the conversation history and content with the elderly person and reports it to the family.

[0929] 5. Anomaly detection: If the server detects an abnormality, it will notify family members or emergency contacts.

[0930] Examples

[0931] For example, if an elderly person says, "The weather is nice today," the system operates as follows.

[0932] 1. The smartphone microphone records the elderly person's speech and uses a speech recognition library to convert it into text such as "The weather is nice today."

[0933] 2. The server uses the OpenAI API to input the prompt sentence, "Conversation with an elderly person: The weather is nice today. ChatBot's response, having learned the conversational habits of family members:" into the generative AI model and generates an appropriate response.

[0934] 3. The generated response (e.g., "Yes, today is a perfect day to go for a walk") is converted to speech using gTTS and played over the smartphone's speaker.

[0935] 4. The server summarizes the conversation and notifies the family that "The weather is nice today" was said.

[0936] 5. If an abnormality is detected, the server will notify family members or emergency contacts.

[0937] Examples of prompt statements

[0938] Conversation with an Elderly Person: The weather is nice today

[0939] The ChatBot's response, having learned the family's conversational habits:

[0940] In this way, the form for carrying out the invention can be concretely shown.

[0941] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0942] Step 1:

[0943] The device (smartphone) uses a microphone to record the elderly person's speech. The input is the elderly person's voice, and the output is the recorded voice data. This voice data is passed to a voice recognition library.

[0944] Step 2:

[0945] The terminal converts the recorded voice data into text using a voice recognition library. The input is voice data and the output is text data. This text data is sent to the server.

[0946] Step 3:

[0947] Based on the received text data, the server inputs a prompt sentence into the generative AI model using the OpenAI API. The input is text data, and the output is the generated response text. An example of a prompt sentence is "Conversation with an elderly person: The weather is nice today. ChatBot's response based on learning the conversational habits of family members:"

[0948] Step 4:

[0949] The server sends the generated response text to the terminal. The input is the generated response text and the output is the response text sent to the terminal.

[0950] Step 5:

[0951] The device converts the received response text into speech using gTTS. The input is the response text, and the output is audio data. This audio data is played on the smartphone speaker.

[0952] Step 6:

[0953] The server summarizes the conversation history and contents with the elderly person and reports them to the family. The input is the text data of the conversation, and the output is the summarized report data. This report data is notified to the family.

[0954] Step 7:

[0955] If the server detects an abnormality, it notifies the family and emergency contacts. The input is data for detecting the abnormality, and the output is notification data. This notification data is sent to the family and emergency contacts.

[0956] Example 2

[0957] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0958] There is a need to detect the risk of dementia and loneliness in the elderly at an early stage and take appropriate measures. However, conventional methods often lack communication with the elderly, and risk detection is often delayed. In addition, there are limited means for families to understand the condition of the elderly. This makes it difficult to prevent the deterioration of the elderly's health.

[0959] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0960] In this invention, the server includes a means for providing a speech bot for communicating with elderly people living alone, a means for adjusting personal information and conversation habits of family members, a means for generating and speaking a conversation on behalf of the elderly person, a means for summarizing the conversation history and content with the elderly person and reporting it to the family, a means for reproducing the voice of the elderly person using voice reproduction technology, a means for recording the conversation content in real time and transmitting it to the server, a means for storing the received conversation data in a database, a means for evaluating the conversation content using a generative AI model for analyzing the conversation data in the database, a means for calculating the dementia risk based on the evaluation result of the generative AI model, and a means for notifying the family member of the result if the dementia risk is determined to be high. This makes it possible to detect the dementia risk and loneliness of elderly people early and notify the family member promptly.

[0961] A "talkbot" is a conversational artificial intelligence system designed to interact with the elderly.

[0962] "Family personal information" refers to basic information about an elderly person's family members, including names, contact details, and relationships.

[0963] "Conversational habits" refers to the words, phrases and speaking patterns that older people use on a daily basis.

[0964] "Means of adjustment" refers to ways in which the system can optimize its settings and operation to suit the characteristics of the elderly person and their family.

[0965] "Means of generating and speaking conversation" refers to the system's ability to automatically create conversation content and output it as audio.

[0966] "Conversation history" refers to records of past conversations between the elderly person and the speech bot.

[0967] "Summarization techniques" refer to methods for summarizing long conversations and extracting important information.

[0968] "Voice reproduction technology" refers to technology that imitates and generates the voice of a specific person.

[0969] "Real-time recording means" refers to a method of recording the content of a conversation as it occurs.

[0970] "Server" refers to a computer system for storing, processing, and managing data.

[0971] "Database" refers to a system for efficiently storing, retrieving, and managing structured data.

[0972] A "generative AI model" refers to a model that uses artificial intelligence technology to analyze data and extract specific patterns and information.

[0973] "Dementia risk" refers to an indicator used to assess the likelihood that an elderly person will develop dementia.

[0974] "Means of notification" refers to the method by which specific information is communicated to families.

[0975] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0976] This invention is a system for assessing the risk of dementia and loneliness in elderly people and notifying their families. This system is realized using hardware and software such as a speech bot, a server, a generative AI model, and a database.

[0977] Hardware and Software Used

[0978] Hardware: Servers, terminals (smartphones, tablets, PCs, etc.)

[0979] Software: Talkbots, generative AI models (e.g. GPT-4), database management systems (e.g. MySQL)

[0980] System Overview

[0981] The user uses a device to access the speech bot and begin a conversation. The speech bot records the content of the conversation in real time and sends it to the server. The server stores the received conversation data in a database and periodically analyzes the content of the conversation using a generative AI model. The generative AI model detects signs of dementia, and the server calculates the dementia risk based on the evaluation results. If the risk is determined to be high, the server notifies the family of the result.

[0982] Examples

[0983] For example, if a user had the following conversation with a speech bot:

[0984] User: "I can't remember what I had for dinner yesterday."

[0985] Speech Bot: "That's concerning. Have you been forgetful at any other times lately?"

[0986] User: "Yeah, I keep telling the same stories over and over."

[0987] This conversation data is sent to a server and analyzed by the generative AI model. The model detects signs of dementia from statements such as "I don't remember what I had for dinner yesterday" or "I keep repeating the same thing." The server uses this data to assess the risk of dementia and notify family members if necessary.

[0988] Examples of prompt statements

[0989] Here is an example of a prompt to be input to the generative AI model:

[0990] Analyze the user's conversation and evaluate whether they have signs of dementia. See the following conversation for reference:

[0991] User: "I can't remember what I had for dinner yesterday."

[0992] Speech Bot: "That's concerning. Have you been forgetful at any other times lately?"

[0993] User: "Yeah, I keep telling the same stories over and over."

[0994] By inputting this prompt into a generative AI model, the model analyzes the conversation content and assesses signs of dementia.

[0995] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0996] Step 1:

[0997] A user accesses the speech bot using a terminal.

[0998] Input: A user accesses the speechbot's application or website using a smartphone, tablet, or PC.

[0999] Output: The speechbot home screen is displayed.

[1000] Specific operation: A user opens the app and the login screen is displayed. The user enters their ID and password to log in.

[1001] Step 2:

[1002] A user starts a conversation with a speech bot.

[1003] Input: The user speaks to the speech bot.

[1004] Output: The speech bot recognizes what the user said and generates a response.

[1005] Specific behavior: After logging in, the user taps the "Start conversation" button, then taps the microphone icon to speak. For example, the user speaks to the speech bot, "Hello, how are you today?"

[1006] Step 3:

[1007] The speech bot records the conversation in real time and sends it to the server.

[1008] Input: What the user says.

[1009] Output: Sent to the server as text data.

[1010] Specific operation: The speech bot converts the user's speech into text using speech recognition technology and immediately sends it to the server. For example, the user's speech "Hello, how are you today?" is sent to the server as text data.

[1011] Step 4:

[1012] The server stores the received conversation data in a database.

[1013] Input: The text data sent to the server.

[1014] Output: Conversation data stored in a database.

[1015] Specific operation: The server saves the received text data with a timestamp in the "conversation history" table of the database. For example, the server saves the conversation data "Hello, how are you today?" in the database.

[1016] Step 5:

[1017] The server periodically analyzes the conversation data in the database and evaluates the conversation content using a generative AI model.

[1018] Input: Conversation data in the database.

[1019] Output: Evaluation results from the generative AI model.

[1020] Specific operation: The server executes a scheduled task at 2 a.m. every day and inputs the past 24 hours of conversation data into the generative AI model. For example, the server obtains the past 24 hours of conversation data and inputs it into the generative AI model as a prompt sentence.

[1021] Step 6:

[1022] Generative AI model detects signs of dementia.

[1023] Input: Conversation data entered as prompt sentences.

[1024] Output: Assessment results indicating signs of dementia.

[1025] Specific operation: The generative AI model analyzes the conversation data based on the prompt sentence and detects keywords and patterns that indicate signs of dementia. For example, the generative AI model detects signs of dementia from statements such as "I don't remember what I had for dinner yesterday" or "I keep repeating the same thing."

[1026] Step 7:

[1027] The server calculates the risk of dementia based on the evaluation results of the generated AI model.

[1028] Input: Evaluation results of a generative AI model.

[1029] Output: Dementia risk score.

[1030] How it works: The server analyzes the output of the generative AI model and runs an algorithm to calculate a risk score. For example, the server calculates a "dementia risk score" and determines whether the score exceeds a certain threshold.

[1031] Step 8:

[1032] If the server determines that there is a high risk of dementia, it notifies the family of the result.

[1033] Input: Dementia risk score.

[1034] Output: A notification message to family members.

[1035] Specific operation: If the risk score exceeds the threshold, the server generates a message to notify the family and sends it via the registered email address or app notification. For example, the server sends a message to the family saying, "Based on your father's recent conversations, we have determined that he is at high risk for dementia. We recommend a detailed evaluation."

[1036] (Application example 2)

[1037] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1038] There is a need to detect dementia risk in the elderly at an early stage and take appropriate measures. However, with the number of elderly people living alone increasing, it is difficult for family members and caregivers to keep track of the elderly's condition on a daily basis. In addition, reducing the elderly's sense of loneliness and maintaining their mental health are also important issues. To solve these issues, a system is needed that assesses the dementia risk through daily conversations with the elderly and notifies the family and caregivers as necessary.

[1039] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1040] In this invention, the server includes a means for providing a speech ChatBot for communicating with elderly people who live alone, a means for fine-tuning personal information and conversation habits of family members, a means for generating and speaking a conversation on behalf of the elderly person, a means for summarizing the conversation history and content with the elderly person and reporting it to the family, a means for reproducing the voice of the elderly person using voice reproduction artificial intelligence, a means for converting the conversation content from voice to text, a means for analyzing the conversation pattern and frequency, and a means for evaluating the dementia risk and notifying the family member or caregiver. This allows the dementia risk of the elderly person to be detected early, and the family member or caregiver to take appropriate measures. It also reduces the elderly person's sense of loneliness and maintains their mental health.

[1041] "Talk ChatBot" is a dialogue system that uses artificial intelligence to conduct voice conversations with elderly people.

[1042] "Fine-tuning" means that the system adjusts the optimal content and method of conversation based on a family member's personal information and conversational habits.

[1043] "Summarization" means concisely summarizing the conversation history and content with the elderly person.

[1044] "Voice reproduction artificial intelligence" is an artificial intelligence technology that imitates and reproduces the voice of a specific person.

[1045] "Convert to text" refers to the process of converting voice data into character data.

[1046] "Analysis" refers to analyzing conversation patterns and frequencies to detect certain trends or anomalies.

[1047] "Dementia risk" is an indicator that evaluates the degree of likelihood that an elderly person will develop dementia.

[1048] "Notification" refers to informing family members or caregivers of information detected by the system.

[1049] A system for implementing this invention has the following configuration: The server includes means for providing a speech ChatBot for communicating with elderly people living alone, means for fine-tuning personal information and conversational habits of family members, means for generating and speaking conversations on behalf of the elderly person, means for summarizing the conversation history and content with the elderly person and reporting it to the family, means for reproducing the elderly person's voice using voice reproduction AI, means for converting the content of the conversation from voice to text, means for analyzing the conversation pattern and frequency, and means for assessing the risk of dementia and notifying the family member or caregiver.

[1050] Hardware and Software Used

[1051] Hardware: Smartphone (iOS or Android)

[1052] Software: Speech recognition API (Google Cloud Speech-to-Text), natural language processing library (spaCy), notification service (messaging service)

[1053] Data processing and calculation

[1054] 1. Audio recording and transcription:

[1055] Conversations with elderly people are recorded using the microphone on the device (smartphone).

[1056] The recorded audio data is converted to text using the Google Cloud Speech-to-Text API.

[1057] 2. Conversation Analysis:

[1058] The server analyzes the converted text data using a natural language processing library (spaCy).

[1059] Analyze conversation patterns, frequency and content to detect signs of dementia.

[1060] 3. Risk Assessment:

[1061] The server assesses dementia risk based on the symptoms detected, for example scoring how often a person repeats themselves or has difficulty finding words.

[1062] 4. Notification function:

[1063] The server uses a messaging service to send notifications to family members or caregivers if there is a high risk.

[1064] Examples

[1065] Scenario: An elderly person talks with their family members through an app every day. The device records the conversation, converts it into text, and sends it to a server. The server analyzes the content of the conversation and detects an increase in the frequency of repeating the same things. The server assesses that the risk of dementia is increasing and sends a notification to the family members to encourage early medical consultation.

[1066] Examples of prompt statements

[1067] Develop an application that records conversations with seniors, converts them into text, and analyzes them. Analyze conversation patterns and frequency to detect signs of dementia. Evaluate dementia risk based on detected signs, and send notifications to family members or caregivers if risk is high. Technologies used are Google Cloud Speech-to-Text API, spaCy, and messaging services.

[1068] In this way, we demonstrated a concrete example of implementing a talking ChatBot to assess the dementia risk of elderly people as a smartphone app.

[1069] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1070] Step 1:

[1071] The terminal records conversations with the user (elderly person). The input is the user's voice, and the output is the recorded voice data. Specifically, the terminal's microphone is used to capture the voice and save it as an audio file.

[1072] Step 2:

[1073] The device converts the recorded voice data into text using the Google Cloud Speech-to-Text API. The input is the recorded voice data, and the output is the converted text data. The specific operation is to send the audio file to the API and receive the returned text data.

[1074] Step 3:

[1075] The terminal transmits the converted text data to the server. The input is the text data, and the output is the text data transmitted to the server. In concrete terms, the text data is uploaded to the server via a network.

[1076] Step 4:

[1077] The server analyzes the received text data using a natural language processing library (spaCy). The input is text data, and the output is the analysis results. Specifically, the text data is input into spaCy, and the conversation patterns and frequency are analyzed.

[1078] Step 5:

[1079] The server evaluates the dementia risk based on the analysis results. The input is the analysis results, and the output is a dementia risk assessment score. In specific operations, the analysis results are input to an assessment algorithm to calculate a risk score.

[1080] Step 6:

[1081] The server uses a messaging service to send a notification to family members or caregivers if the dementia risk assessment score is high. The input is the dementia risk assessment score and the output is the sent notification. Specific operations include evaluating the risk score and generating and sending a notification message.

[1082] Step 7:

[1083] The server summarizes the conversation history and content with the elderly person and reports it to the family. The input is text data and analysis results, and the output is a summarized report. Specifically, the server generates a summary based on the text data and analysis results and sends it to the family.

[1084] Example 3

[1085] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[1086] There is a need to detect loneliness and dementia risk in elderly people at an early stage and take appropriate measures. However, with conventional systems, it was difficult to effectively analyze the content and frequency of conversations with elderly people and their emotional expressions to evaluate loneliness and dementia risk. There was also a lack of means to notify family members at an appropriate time. This led to a high possibility that loneliness and dementia risk in elderly people would be overlooked, and appropriate measures would be delayed, which was a problem.

[1087] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1088] In this invention, the server includes a means for recording a conversation with an elderly person and converting it into text data, a means for transmitting the converted text data to the server, a means for analyzing the content of the conversation using a generative AI model, a means for calculating a loneliness index based on the analysis result, and a means for notifying a family member when the loneliness index exceeds a certain threshold. This makes it possible to detect loneliness and dementia risk in an elderly person at an early stage and notify the family member at an appropriate time.

[1089] "Single-person elderly" refers to elderly people who live alone.

[1090] A "talkbot" is a system equipped with artificial intelligence that can converse with users via voice.

[1091] "Family personal information" refers to information such as family members' names, contact details, and relationships.

[1092] "Means of adjusting conversational habits" refers to technology that optimizes the voice bot's responses based on the user's conversational patterns and preferences.

[1093] "Means for generating and speaking a conversation" refers to technology that generates an appropriate response based on user input and speaks it aloud.

[1094] "Means of summarizing conversation history and content and reporting it to family members" refers to technology that summarizes the content of conversations with elderly people and notifies their family members.

[1095] "Voice reproduction technology" refers to technology that imitates and reproduces the voice of a specific person.

[1096] "Means for converting into text data" refers to technology for converting voice data into character data.

[1097] "Means for transmitting to a server" refers to a technique for transmitting data from a terminal to a server.

[1098] A "generative AI model" refers to a model that uses artificial intelligence to analyze data and generate a specific outcome.

[1099] "Means of analyzing the content of the conversation" refers to technology that uses a generative AI model to analyze the content of the conversation and extract specific information.

[1100] "Means for calculating an index of loneliness" refers to technology that quantifies the loneliness of elderly people based on the results of analyzing the content of their conversations.

[1101] "Means for notifying family members" refers to technology that sends alerts or notifications to family members when certain conditions are met.

[1102] This invention is a system that evaluates the loneliness and dementia risk of elderly people and notifies their families. The system is composed of a speech bot, a server, a terminal, and a generative AI model.

[1103] Hardware and Software Configuration

[1104] server

[1105] The server stores and processes data. Specifically, it uses a cloud service. For example, Amazon Web Services (AWS) can be used. The server has the following functions:

[1106] Saving text data

[1107] Data Analysis Using Generative AI Models

[1108] Calculating the loneliness index

[1109] Notification to family members

[1110] Terminal

[1111] The terminal is a device used by the elderly. Specifically, a smartphone or tablet is used. The terminal has the following functions:

[1112] Recording conversations with seniors

[1113] Convert audio recordings to text

[1114] Sending text data to the server

[1115] Generative AI Models

[1116] The generative AI model analyzes conversation content and evaluates loneliness. Specifically, it uses OpenAI's GPT-4. The generative AI model has the following functions:

[1117] Conversation analysis

[1118] Extracting keywords that indicate emotions and loneliness

[1119] Data processing and calculation

[1120] Collecting conversation data

[1121] The device records conversations with the elderly and converts them into text data in real time, using a voice recognition API (for example, Google Cloud Speech-to-Text API).

[1122] Sending conversation data

[1123] The terminal sends the converted text data to the server using the HTTPS protocol. The server stores the received text data in a database. Specifically, it uses Amazon RDS (Relational Database Service).

[1124] Conversation analysis

[1125] The server inputs the stored text data into the generative AI model. The prompt sentence is "Please analyze the elderly person's emotions from the content of this conversation." The generative AI model analyzes the content of the conversation and extracts keywords that indicate the elderly person's emotions and feelings of loneliness.

[1126] Loneliness assessment

[1127] The server calculates the loneliness index based on the analysis results obtained from the generated AI model. Specifically, it evaluates the frequency of keywords that indicate loneliness, the lack of emotional expression, and the frequency of conversations. If the loneliness index exceeds a certain threshold, an alert is generated.

[1128] Notification to family members

[1129] When the server generates an alert, it will send a notification to the family members via email or SMS. The family members will receive the notification and take action to check on the elderly person's condition.

[1130] Examples of concrete examples and prompts

[1131] Examples

[1132] Conversation content: Elderly people frequently use phrases such as "I haven't talked to anyone today" and "I feel lonely."

[1133] Emotional expression: Elderly people tend to express little emotion in their speech.

[1134] Conversation Frequency: Conversations recorded only once a week.

[1135] Examples of prompt statements

[1136] "Please analyze what emotions elderly people express in recent conversations."

[1137] "Please rate the sense of loneliness based on this conversation."

[1138] In this way, the system can evaluate the elderly person's sense of loneliness and notify the family if necessary. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1139] Step 1:

[1140] Collecting conversation data

[1141] Terminal: Elderly people talk to the speech bot using a smartphone or tablet. The conversation is conducted by voice and is recorded by the terminal.

[1142] Input: Elderly people's voice conversation

[1143] Output: Recorded audio data

[1144] Specific operation: The elderly person says, "I haven't talked to anyone today." The device records this statement.

[1145] Step 2:

[1146] Converting audio data to text

[1147] Terminal: Converts recorded voice data into text data in real time, using a voice recognition API.

[1148] Input: Recorded audio data

[1149] Output: The converted text data

[1150] Specific operation: The device converts the recorded voice data into text data such as "I haven't talked to anyone today."

[1151] Step 3:

[1152] Sending text data

[1153] Terminal: The converted text data is sent to the server using the HTTPS protocol.

[1154] Input: Converted text data

[1155] Output: Text data sent to the server

[1156] Specific operation: The device sends the text data "I haven't talked to anyone today" to the server.

[1157] Step 4:

[1158] Saving text data

[1159] Server: The received text data is stored in a database. Specifically, a database service is used.

[1160] Input: Text data sent to the server

[1161] Output: Text data stored in the database

[1162] Specific operation: The text data received by the server is saved in the "conversation log" table.

[1163] Step 5:

[1164] Conversation analysis

[1165] Server: Input the saved text data into the generative AI model. Use the prompt "Please analyze the elderly person's emotions from this conversation."

[1166] Input: Text data stored in a database

[1167] Output: Analysis results from generative AI models

[1168] Specific operation: The server inputs the text data "I haven't talked to anyone today" into the generative AI model, and obtains the analysis result "Keywords indicating loneliness: I haven't talked to anyone."

[1169] Step 6:

[1170] Loneliness assessment

[1171] Server: Calculates an index of loneliness based on the analysis results obtained from the generative AI model. Specifically, it evaluates the frequency of keywords that indicate loneliness, the lack of emotional expression, and the frequency of conversations.

[1172] Input: Analysis results by generative AI model

[1173] Output: Loneliness index

[1174] Specific operation: Based on the analysis results, the server evaluates the loneliness index as "high."

[1175] Step 7:

[1176] Notification to family members

[1177] Server: If the loneliness index exceeds a certain threshold, a notification is sent to the family via email or SMS.

[1178] Input: Loneliness index

[1179] Output:Notification to family members

[1180] Specific behavior: The server generates an alert and sends an email to the family saying, "An elderly person may be feeling lonely. Recent conversations: Not talking to anyone today."

[1181] (Application example 3)

[1182] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1183] There is a need to detect loneliness and dementia risk in the elderly at an early stage and take appropriate measures. However, current systems have difficulty accurately evaluating the emotions and loneliness of the elderly, and are unable to provide appropriate information to family members and care managers. In addition, there is a need to improve the operability and convenience of smartphones for the elderly.

[1184] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means. In this invention, the server includes a means for providing a speech ChatBot for interacting with elderly people living alone, a means for fine-tuning personal information and conversation habits of family members, a means for generating and speaking a conversation on behalf of the elderly person, a means for summarizing the conversation history and contents with the elderly person and reporting them to the family, a means for reproducing the voice of the elderly person using voice reproduction artificial intelligence, an emotion analysis means for evaluating the elderly person's sense of loneliness, a means for evaluating an index of loneliness and notifying the family member, and a means for operating as an application installed on a smartphone. This makes it possible to detect the elderly person's sense of loneliness and the risk of dementia at an early stage and provide appropriate information to the family member and the care manager.

[1185] "Speech ChatBot" is a dialogue system that uses artificial intelligence to conduct natural conversations with elderly people.

[1186] "Fine-tuning" is the process of optimizing the system's behavior and responses based on a family member's personal information and conversational habits.

[1187] "Conversation generation" is a function in which artificial intelligence automatically creates and speaks dialogue with elderly people.

[1188] "Summarization" is the process of concisely summarizing the conversation history and content with an elderly person.

[1189] "Voice reproduction artificial intelligence" is an artificial intelligence technology that imitates and reproduces the voice of a specific person.

[1190] "Sentiment analysis" is a technology that analyzes emotions from the content of a conversation and evaluates the type and intensity of those emotions.

[1191] The "loneliness index" is a numerical or evaluation standard that expresses the sense of loneliness of elderly people.

[1192] "Notification" is the process of informing family members and care managers of evaluation results and important information.

[1193] An "application installed on a smartphone" is a software program that runs on a smartphone.

[1194] The system for implementing this invention is configured as follows. First, the server provides a speech ChatBot for interacting with the elderly. The speech ChatBot is a dialogue system that uses artificial intelligence to have natural conversations with the elderly. The server also fine-tunes the system's operation and responses to optimize them based on the family's personal information and conversational habits.

[1195] Next, the server has a conversation generation function that automatically creates and speaks the contents of conversations with the elderly. In addition, it provides a summary function that briefly summarizes the conversation history and contents with the elderly and reports them to the family. It also includes a function to mimic and reproduce the voice of a specific person using voice reproduction artificial intelligence.

[1196] The server performs emotion analysis to evaluate the elderly person's loneliness. Emotion analysis is a technology that analyzes emotions from conversation content and evaluates the type and intensity of those emotions. It also includes a function to evaluate the loneliness index and notify the family of the result.

[1197] The system operates as an application installed on a smartphone, and is designed to be easily accessible to seniors through a software program that runs on the smartphone.

[1198] As a concrete example, consider the case where an elderly person talks to a ChatBot through a smartphone. If the elderly person types, "I haven't talked to anyone recently and I feel lonely," the server retrieves the content of the conversation and performs sentiment analysis using the TextBlob library. As a result of the sentiment analysis, negative emotions are detected, and the keyword "lonely" is also included, so the loneliness index becomes high. As a result, a notification is sent to the family saying, "An elderly person is feeling lonely. I will report this to the family."

[1199] An example of a prompt is as follows:

[1200] Enter the conversation you had with the senior:

[1201] Using this prompt, the user can input the content of the conversation, allowing the system to evaluate the sense of loneliness and notify family members if necessary.

[1202] The flow of the specific process in the application example 3 will be described with reference to FIG.

[1203] Step 1:

[1204] A user starts a conversation with the speech ChatBot through a smartphone. The user inputs the content of the conversation and sends it to the smartphone terminal. The input data is in text format, for example, "I haven't talked to anyone recently and I feel lonely."

[1205] Step 2:

[1206] The terminal sends the input conversation content to the server. The server first preprocesses the received text data in order to analyze it. Preprocessing includes normalizing the text (e.g., removing unnecessary spaces and special characters).

[1207] Step 3:

[1208] The server performs sentiment analysis on the preprocessed text data. It uses the TextBlob library to evaluate the sentiment polarity of the text. Based on the input data, it outputs a sentiment: positive, negative, or neutral.

[1209] Step 4:

[1210] The server evaluates the loneliness index based on the results of the emotion analysis. Specifically, if negative emotions are detected or if keywords such as "lonely" are included, the loneliness score is increased. A loneliness score is calculated based on the input data.

[1211] Step 5:

[1212] The server evaluates the loneliness score and sends a notification to the family if it exceeds a certain threshold. The notification includes the loneliness assessment result and a summary of the conversation. The output data is a notification message that is sent to the family.

[1213] Step 6:

[1214] Family members can receive notifications and take appropriate measures to address the elderly person's feelings of loneliness. The notification message includes specific conversation content and the results of sentiment analysis, allowing family members to accurately grasp the elderly person's situation.

[1215] Through the above processing steps, it is possible to evaluate the elderly person's sense of loneliness and provide appropriate information to the family.

[1216] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1217] "Example 1"

[1218] As one embodiment of the present invention, a system incorporating an emotion engine is provided. This system recognizes the emotional state of the elderly person through conversation with the elderly person and reports the result to the family. Specifically, the emotion engine estimates the emotional state of the elderly person from the elderly person's tone of voice, speaking style, choice of words, etc. For example, if the elderly person says "I feel lonely," the emotion engine captures the statement and determines that the elderly person is feeling lonely. This information is reported to the family, allowing the family to understand the elderly person's emotional state.

[1219] "Example 2"

[1220] The emotion engine also has the ability to adjust the content of speech depending on the emotional state of the elderly person. Specifically, if it is determined that the elderly person is feeling happy or excited, the Speech ChatBot will provide topics that will maintain that emotional state. For example, if there is information that the elderly person feels happy when talking about their grandchildren, the Speech ChatBot will provide topics about grandchildren. This enables more appropriate communication that takes into account the emotional state of the elderly person.

[1221] "Example 3"

[1222] Furthermore, the emotion engine has the function of evaluating the elderly person's emotional state and reporting the results to the family. Specifically, if it is determined that the elderly person is feeling sad or anxious, the information is reported to the family. For example, if the elderly person says "I'm worried," the emotion engine will capture the statement and determine that the elderly person is feeling anxious. This information is reported to the family, enabling the family to understand the elderly person's emotional state and take appropriate action.

[1223] The process flow of each embodiment will be described below.

[1224] "Example 1"

[1225] Step 1: The speech ChatBot starts a conversation with the elderly person.

[1226] Step 2: The emotion engine infers the emotional state of the elderly person from their tone of voice, speaking style, and word choice.

[1227] Step 3: The emotion engine evaluates the elderly person's emotional state and reports the results to the family.

[1228] "Example 2"

[1229] Step 1: The speech ChatBot starts a conversation with the elderly person.

[1230] Step 2: The emotion engine estimates the elderly person's emotional state.

[1231] Step 3: The emotion engine adjusts the speech content according to the elderly person's emotional state.

[1232] "Example 3"

[1233] Step 1: The speech ChatBot starts a conversation with the elderly person.

[1234] Step 2: The emotion engine estimates the elderly person's emotional state.

[1235] Step 3: The emotion engine evaluates the elderly person's emotional state and reports the results to the family.

[1236] Example 1

[1237] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[1238] Elderly people who live alone often feel lonely and have a lack of interaction with their families. In addition, it is difficult for families to grasp the emotional state and dementia risk of the elderly, which can delay appropriate responses. This can lead to a decline in the quality of life of the elderly.

[1239] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1240] In this invention, the server includes a means for providing a speech system for communicating with elderly people living alone, a means for fine-tuning personal information and conversation habits of family members using a machine learning algorithm, a means for generating and speaking a conversation on behalf of the elderly person, a means for summarizing the conversation history and content with the elderly person using natural language processing technology and reporting it to the family, a means for reproducing the elderly person's voice using voice reproduction technology, and an emotion engine for recognizing the elderly person's emotional state and reporting the results to the family. This allows the elderly person to feel as if they are interacting with their family, and the family member can also understand the elderly person's emotional state and dementia risk.

[1241] "Single-person elderly" refers to elderly people who live alone.

[1242] A "speech system" refers to a system for communicating with people using voice.

[1243] A "machine learning algorithm" refers to an algorithm that trains a model based on data and makes predictions and classifications.

[1244] "Fine-tuning" refers to the process of fine-tuning an existing machine learning model to adapt it to a specific dataset.

[1245] "Natural language processing technology" refers to technology that enables computers to understand, interpret, and generate human language.

[1246] "Voice reproduction technology" refers to technology for imitating and reproducing the voice of a specific person.

[1247] An "emotion engine" refers to an engine for analyzing and inferring emotional states from voice and text data.

[1248] "Conversation history" refers to a record of past conversations.

[1249] A "summary" is a summary of a longer piece of text or a conversation.

[1250] The present invention provides a speech system for communicating with elderly people who live alone. A specific embodiment of this system will be described below.

[1251] The server first collects conversation data from family members. Family members input text and voice data through a dedicated app and send it to the server. The collected data is then stored in a database.

[1252] Next, the server uses Python and TensorFlow to build a machine learning model, using the collected family conversation data as training data to fine-tune the model, learning the habits and characteristics of family conversations.

[1253] When an elderly person speaks to the device, the device sends the input to the server. The server uses a machine learning model to generate an appropriate response and speaks it out loud through the device. At this time, voice reproduction technology is used to reproduce the elderly person's voice. Specifically, a Python voice processing library is used to analyze the voice data and model the characteristics of the elderly person's voice.

[1254] In addition, the device records conversation history with the elderly. The server analyzes the recorded conversation data and extracts important information using Python's NLTK library. The extracted information is reported to the family as a summary.

[1255] The emotion engine analyzes the elderly person's tone of voice and speaking style in real time. The server uses Python's OpenCV and Dlib libraries to estimate the emotional state. For example, if an elderly person says "I feel lonely," the emotion engine captures the statement and determines that the elderly person is feeling lonely. This information is reported to the family through the server.

[1256] As a concrete example, if an elderly person says, "I feel lonely today," the device will send this statement to the server. The server will analyze this statement using an emotion engine and determine that the elderly person is feeling lonely. This information will be reported to the family as a summary. The family will receive this information and can take action, such as calling the elderly person.

[1257] An example of a prompt sentence might be:

[1258] “If an elderly person feels lonely, explain how the emotion engine recognizes that emotion and reports it to the family.”

[1259] This system allows elderly people to feel as if they are interacting with their families, and enables families to understand the elderly person's emotional state and dementia risk.

[1260] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1261] Step 1:

[1262] The server collects family conversation data. The user inputs text and voice data through a dedicated app and sends it to the server. The server stores the received data in a database. The input is family conversation data, and the output is the data stored in the database.

[1263] Step 2:

[1264] The server uses Python and TensorFlow to build a machine learning model. The server retrieves family conversation data from the database and uses it as learning data. The server uses a machine learning algorithm to train the model and have it learn the habits and characteristics of family conversations. The input is the conversation data retrieved from the database, and the output is a fine-tuned machine learning model.

[1265] Step 3:

[1266] When an elderly person speaks to the device, the device sends the input to the server. The server uses a machine learning model to generate an appropriate response. The server sends the generated response to the device, which then speaks it out loud. The input is the elderly person's speech data, and the output is the audio data of the generated response.

[1267] Step 4:

[1268] The terminal records conversation history with the elderly. The server analyzes the recorded conversation data and extracts important information using Python's NLTK library. The server reports the extracted information to the family as a summary. The input is the recorded conversation data, and the output is the report data as a summary.

[1269] Step 5:

[1270] The server learns the voice data of the elderly and extracts its features. The server analyzes the voice data using a Python voice processing library and models the features of the elderly's voice. The device uses this model to reproduce the elderly's voice and simulate a conversation with a family member. The input is the elderly's voice data, and the output is the reproduced voice data.

[1271] Step 6:

[1272] The device analyzes the elderly person's tone of voice and speaking style in real time. The server estimates the emotional state using Python's OpenCV and Dlib libraries. The server reports the estimated emotional state to the family. The input is the elderly person's speech data, and the output is the report data of the estimated emotional state.

[1273] As a concrete example, if an elderly person says, "I feel lonely today," the device will send this statement to the server. The server will analyze this statement using an emotion engine and determine that the elderly person is feeling lonely. This information will be reported to the family as a summary. The family will receive this information and can take action, such as calling the elderly person.

[1274] (Application example 1)

[1275] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[1276] There is a need for a system that ensures the safety and security of elderly people living alone and allows their families to understand their situation. In particular, there is a need for a means to quickly respond when elderly people feel lonely or face an emergency.

[1277] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for providing a speech ChatBot for communicating with elderly people living alone, a means for fine-tuning personal information and conversation habits of family members, a means for generating and speaking a conversation on behalf of the elderly person, a means for summarizing the conversation history and content with the elderly person and reporting it to the family, a means for reproducing the voice of the elderly person using a voice reproduction AI, a means for recognizing the emotional state of the elderly person and reporting the result to the family, and a means for detecting an abnormality and notifying an emergency contact. This makes it possible to quickly grasp the emotional state of the elderly person and emergency situations and provide appropriate information to the family and emergency contacts.

[1278] "Single-person elderly" refers to elderly people who live alone.

[1279] A "Speech ChatBot" is an artificial intelligence program that converses with people through voice.

[1280] "Family personal information" refers to information about individuals in family members, such as their names, ages, and relationships.

[1281] "Conversational habits" refers to the distinctive way a particular person speaks and chooses words.

[1282] "Fine-tuning" refers to the process of fine-tuning an existing machine learning model to fit specific data.

[1283] "Means of generating and speaking conversation" refers to the technology that creates natural conversation based on input information and outputs it as audio.

[1284] "Conversation history" refers to a record of past conversations.

[1285] A "summary" refers to a brief summary of a long piece of text or a conversation.

[1286] "Voice reproduction AI" refers to artificial intelligence technology that learns the characteristics of a specific person's voice and reproduces that voice.

[1287] "Emotional state" refers to a person's emotional state, such as happiness, sadness, anger, etc.

[1288] "Means of detecting anomalies" refers to technologies that identify unusual or emergency situations.

[1289] "Emergency Contact" refers to the person or organization to be contacted in the event of an emergency.

[1290] The system for implementing this invention provides a speech ChatBot for communicating with elderly people who live alone. The system includes means for fine-tuning personal information and conversational habits of family members, means for generating and speaking conversations on behalf of the elderly person, means for summarizing conversation history and content with the elderly person and reporting it to the family, means for reproducing the elderly person's voice using voice reproduction AI, means for recognizing the elderly person's emotional state and reporting the results to the family, and means for detecting abnormalities and notifying emergency contacts.

[1291] Hardware and Software Used

[1292] Hardware: Smartphone (microphone, internet connection)

[1293] Software: Python, speech recognition library, transformers library, smtplib library

[1294] Data processing and calculation

[1295] Speech Recognition

[1296] The server uses a voice recognition library to convert the voice input from the smartphone microphone into text. Through this process, the contents of the elderly person's speech are obtained as text data.

[1297] sentiment analysis

[1298] The server analyzes the emotional state of the acquired text data using the emotion analysis model of the transformers library, and classifies the elderly person's emotional state as "happiness," "sadness," "anger," etc.

[1299] Conversation generation and speech

[1300] The server uses a generative AI model to learn the family's personal information and conversational habits, and generates natural conversations. This generated conversation is then spoken in the elderly person's voice using a voice reproduction AI.

[1301] Conversation History Summary

[1302] The server uses natural language processing technology to summarize conversation histories with the elderly and reports them to the family, allowing the family to understand the elderly's situation.

[1303] Anomaly detection and notification

[1304] The server uses anomaly detection to send a notification to emergency contacts if the senior utters words like "help" or "emergency." The notification is sent via email using the smtplib library.

[1305] Examples

[1306] For example, if an elderly person says, "I feel a little lonely today," the sentiment analysis model will determine this as "negative," and notify the family member by email, "The elderly person feels lonely." Also, if the elderly person says, "Help me," an emergency notification will be sent to the emergency contacts, "The elderly person is requesting help."

[1307] Examples of prompt statements

[1308] An elderly person says, "I feel a little lonely today." The sentiment analysis model judges this to be "negative." An email is sent to the family saying, "An elderly person feels lonely."

[1309] In this way, a monitoring security app that provides safety and security to elderly people living alone has been realized.

[1310] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1311] Step 1:

[1312] The user (elderly person) speaks into the microphone of the smartphone. The input is the elderly person's voice, and the device collects this voice.

[1313] Step 2:

[1314] The terminal transmits the collected voice data to the server. The input is the voice data, and the output is the transfer of the voice data to the server.

[1315] Step 3:

[1316] The server converts the voice data into text data using a voice recognition library. The input is voice data and the output is text data. In concrete terms, the voice recognition algorithm analyzes the voice waveform and generates the corresponding text.

[1317] Step 4:

[1318] The server uses sentiment analysis models from the transformers library to analyze the emotional state of text data. The input is text data, and the output is an emotional state (e.g., happy, sad, angry). Specifically, the sentiment analysis algorithm analyzes the content of the text and assigns an emotional label.

[1319] Step 5:

[1320] The server uses a generative AI model to learn the family's personal information and conversational habits, and generates natural conversations. The input is the family's personal information, conversational habits, and emotional state, and the output is the generated conversation text. In concrete terms, the generative AI model generates new conversations based on past conversation data.

[1321] Step 6:

[1322] The server uses a voice reproduction AI to recite the generated conversation text in the elderly person's voice. The input is the generated conversation text, and the output is the reproduced voice. In concrete terms, the voice reproduction AI converts the text into voice and reproduces the characteristics of the elderly person's voice.

[1323] Step 7:

[1324] The server uses natural language processing technology to summarize the conversation history with the elderly person and reports it to the family. The input is the conversation history, and the output is the summarized conversation content. In concrete terms, the natural language processing algorithm analyzes the conversation history, extracts important information, and generates a summary.

[1325] Step 8:

[1326] The server uses anomaly detection to send a notification to emergency contacts if the elderly person utters words such as "help" or "emergency." The input is text data, and the output is an emergency notification. Specifically, the anomaly detection algorithm detects certain keywords and sends an email using the smtplib library.

[1327] Step 9:

[1328] The family receives the summarized conversation contents and emergency notifications sent from the server and understands the situation of the elderly person. The input is the notification from the server, and the output is the family's understanding. In concrete terms, the family checks emails and app notifications to understand the situation of the elderly person.

[1329] Example 2

[1330] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[1331] There is a need to properly evaluate the dementia risk and emotional state of elderly people and report them to their families. It is also important to facilitate communication with elderly people and reduce their sense of loneliness. However, with conventional systems, it was difficult to comprehensively solve these issues.

[1332] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for collecting conversation data with the elderly person, a means for converting the collected voice data into text data, a means for analyzing the text data, a means for evaluating the dementia risk based on the analysis result, a means for evaluating the emotional state of the elderly person, and a means for adjusting the speech content according to the emotional state. This makes it possible to appropriately evaluate the dementia risk and emotional state of the elderly person and report them to the family. In addition, it is possible to facilitate communication with the elderly person and reduce the feeling of loneliness.

[1333] "Single-person elderly" refers to elderly people who live alone.

[1334] "Speech device" refers to a device for generating speech and interacting with a user.

[1335] "Personal information" refers to information that can identify a specific individual.

[1336] "Conversational habits" refers to the conversational patterns or characteristics that a particular individual possesses.

[1337] "Adjust" means to change or optimize something to suit specific conditions or purposes.

[1338] "Generate" means to create something new.

[1339] "Speaking" refers to uttering words through sound.

[1340] To summarise means to briefly summarise the main points.

[1341] "Voice reproduction technology" refers to technology that imitates and generates the voice of a specific person.

[1342] "Conversation data" refers to data that records the contents of a conversation.

[1343] "Audio data" refers to data that records audio in digital format.

[1344] "Text data" refers to data recorded as character information.

[1345] "Analyze" means to examine data in detail and extract meaning and patterns.

[1346] "Dementia risk" refers to the possibility of developing dementia.

[1347] "Emotional state" refers to an individual's emotional state at a particular point in time.

[1348] "Adjust" means to change or optimize something to suit specific conditions or purposes.

[1349] "Loneliness" refers to the feeling of lacking connection with others.

[1350] The present invention provides a speech device for communicating with elderly people who live alone, and is a system for evaluating the dementia risk and emotional state of the elderly and reporting the results to their families. A specific embodiment of the system will be described below.

[1351] System Configuration

[1352] The system consists of the following main components:

[1353] 1. Speech devices: Devices for communicating with the elderly, such as smart speakers and tablets.

[1354] 2. Server: A central processing unit that collects, analyzes, and evaluates data.

[1355] 3. Family devices: Devices used to report on the elderly person's condition, such as smartphones and computers.

[1356] Hardware and Software Used

[1357] Speech Recognition Software: Uses the Google Speech-to-Text API to convert voice data into text data.

[1358] Natural Language Processing (NLP) techniques: Use spaCy and NLTK to analyze text data.

[1359] Machine learning models: Algorithms for assessing dementia risk.

[1360] Emotion Engine: Uses IBM Watson Tone Analyzer to assess the emotional state of seniors.

[1361] System Operation

[1362] The user (elderly person) starts the system by speaking into the speech device. For example, they might say, "Hello, how was your day?" The device records this conversation and saves it as voice data. The device then transmits the recorded voice data to the server.

[1363] The server converts the voice data into text data using the Google Speech-to-Text API. The converted text data is then analyzed using spaCy. The analysis results detect whether the elderly person is repeating themselves or is having difficulty with everyday conversation.

[1364] Next, the server uses a machine learning model to assess the risk of dementia based on the analysis results. If the risk is determined to be high, the server notifies the family member's device of that information. For example, it may notify the family member, "The elderly person is repeating the same story. There is a possibility that the risk of dementia is high."

[1365] In addition, the server uses IBM Watson Tone Analyzer to evaluate the emotional state of the elderly. If it is determined that the elderly would be happy to hear stories about their grandchildren, the server adjusts the speech device's speech content to "Tell me a story about your grandchild." The speech device speaks to the elderly, "Tell me a story about your grandchild."

[1366] Examples of concrete examples and prompts

[1367] Examples:

[1368] Elderly person A begins a conversation with the speech device.

[1369] The device records Person A's conversation and saves it as audio data.

[1370] The server converts the voice data into text data using the Google Speech-to-Text API.

[1371] The server analyzes the text data using spaCy and detects that Person A is repeating the same story.

[1372] The server uses a machine learning model to assess Mr. A's risk of dementia and determines that he is at high risk.

[1373] The server notifies the family, "An elderly person is repeating the same story. They may be at high risk of dementia."

[1374] The server evaluates Mr. A's emotional state using IBM Watson Tone Analyzer and determines that Mr. A would be happy if he talked about his grandchildren.

[1375] The speech device speaks to Mr. A, saying, "Tell me about your grandchild."

[1376] Example prompt:

[1377] Create a program that collects conversations with elderly people and assesses their dementia risk. Convert the conversations into text data using the Google Speech-to-Text API and analyze them using spaCy. Based on the analysis results, assess the dementia risk using a machine learning model and evaluate the emotional state using IBM Watson Tone Analyzer. Add a function to adjust the speech content according to the emotional state.

[1378] In this way, the system can properly assess the dementia risk and emotional state of the elderly and report it to their families, facilitating communication with the elderly and reducing their sense of loneliness.

[1379] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1380] Step 1:

[1381] The user (elderly person) speaks to the speech device. For example, he / she says, "Hello, how was your day?" The input is the elderly person's voice, and the output is the recorded voice data. The terminal records this conversation and saves it as voice data.

[1382] Step 2:

[1383] The terminal sends the recorded voice data to the server. The input is the recorded voice data, and the output is the voice data sent to the server. The server converts the voice data to text data using the Google Speech-to-Text API. The input is the voice data, and the output is the converted text data.

[1384] Step 3:

[1385] The server analyzes the converted text data using natural language processing (NLP) technology (e.g., spaCy). The input is the text data, and the output is the analysis results. The server analyzes the conversation patterns and frequency to detect whether the elderly person is repeating the same things or is having difficulty in everyday conversation.

[1386] Step 4:

[1387] The server uses a machine learning model based on the analysis results to assess the risk of dementia. The input is the analysis result, and the output is the dementia risk assessment result. If the server determines that the risk is high, it notifies the family member's device of that information. For example, it may notify, "An elderly person keeps repeating the same thing. There is a possibility that they are at high risk of dementia."

[1388] Step 5:

[1389] The server evaluates the emotional state of the elderly person using IBM Watson Tone Analyzer. The input is text data, and the output is the evaluation result of the emotional state. The server judges whether the elderly person is feeling joy or pleasure.

[1390] Step 6:

[1391] The server adjusts the speech content of the speech device based on the evaluation result of the emotion engine. The input is the evaluation result of the emotional state, and the output is the adjusted speech content. The speech device speaks to the elderly person, saying, "Tell me a story about your grandchild."

[1392] (Application example 2)

[1393] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1394] It is necessary to protect the safety and health of the elderly by properly assessing their dementia risk and emotional state and promptly notifying their families and caregivers. In addition, there is a need for communication methods that allow the elderly to live their daily lives with peace of mind without feeling lonely.

[1395] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1396] In this invention, the server includes a means for providing a speech ChatBot for communicating with elderly people living alone, a means for adjusting personal information and conversation habits of family members, a means for generating and speaking a conversation on behalf of the elderly person, a means for summarizing the conversation history and content with the elderly person and reporting it to the family, a means for reproducing the elderly person's voice using voice reproduction technology, a means for analyzing the elderly person's emotional state and providing appropriate topics, and a means for assessing dementia risk and notifying the family. This makes it possible to appropriately assess the elderly person's dementia risk and emotional state and quickly notify the family or caregiver.

[1397] "Speech ChatBot" is a conversational system using artificial intelligence designed to interact with the elderly.

[1398] "Personal information" refers to information about family members and seniors, such as name, age, address, and contact details.

[1399] "Conversational habits" refers to the words and speaking patterns that older people use on a daily basis.

[1400] "Tune" refers to changing settings or parameters to suit a particular purpose.

[1401] "Generating and speaking a conversation" refers to the artificial intelligence creating appropriate responses for the elderly and outputting them as voice.

[1402] "Conversation history" refers to records of past conversations between seniors and the speech ChatBot.

[1403] "Summarizing" refers to summarizing a long conversation in a concise manner.

[1404] "Voice reproduction technology" refers to technology that imitates and generates the voice of a specific person.

[1405] "Emotional state" refers to emotions such as happiness, sadness, and anger felt by older adults.

[1406] "Appropriate topics" refer to conversation topics chosen to suit the emotional state and interests of the elderly.

[1407] "Dementia risk" refers to an indicator that assesses the likelihood that an elderly person will develop dementia.

[1408] "Inform" refers to giving specific information to family members or caregivers.

[1409] The system for implementing this invention is configured as follows: The server includes a means for providing a speech ChatBot for communicating with elderly people living alone, a means for adjusting personal information and conversational habits of family members, a means for generating and speaking a conversation on behalf of the elderly person, a means for summarizing the conversation history and content with the elderly person and reporting it to the family, a means for reproducing the elderly person's voice using voice reproduction technology, a means for analyzing the emotional state of the elderly person and providing appropriate topics, and a means for assessing the risk of dementia and notifying the family member.

[1410] System program processing

[1411] The server first records the conversation between the elderly person and the speech ChatBot. The recorded conversation content is subjected to sentiment analysis using a natural language processing library (e.g., NLTK or spaCy). Based on the results of the sentiment analysis, the emotion engine determines the elderly person's emotional state and provides appropriate topics.

[1412] The DementiaRiskAssessment module then analyzes conversation patterns and frequency to assess dementia risk, and if the assessment result exceeds a certain risk, a notification service is used to notify family members.

[1413] In addition, conversation histories are periodically summarized and reported to family members, allowing them to keep track of the elderly person's condition.

[1414] Hardware and Software Used

[1415] Hardware: Smartphone

[1416] Software: Python, natural language processing libraries (NLTK, spaCy), notification services

[1417] Examples

[1418] For example, if an elderly person says, "My grandchildren came to visit me today," the server records this conversation and performs sentiment analysis. If the emotion of joy is detected, the emotion engine generates a response such as, "That's wonderful! Tell me more about your grandchildren."

[1419] You can also input prompts like the following into the generative AI model to generate appropriate responses:

[1420] Example prompt:

[1421] If an elderly person says, "My grandchild came to visit me today," perform sentiment analysis and, if the emotion of joy is detected, generate a response that continues the topic of the grandchild.

[1422] In this way, appropriate communication according to the emotional state of the elderly person becomes possible, and dementia risk can also be assessed and notified to family members.

[1423] The flow of the identification process in the application example 2 will be described with reference to FIG.

[1424] Step 1:

[1425] The server records the conversation between the elderly person and the Talking ChatBot. As input, the elderly person's speech is provided as voice data. The server converts this voice data into text data and adds it to the conversation history. As output, the conversation data in text format is obtained.

[1426] Step 2:

[1427] The server performs sentiment analysis on the recorded conversation content using natural language processing libraries (NLTK and spaCy). Text-formatted conversation data is provided as input. The server analyzes this data and determines the emotional state (happiness, sadness, anger, etc.). The output is a label for the emotional state.

[1428] Step 3:

[1429] The server uses an emotion engine to provide appropriate topics based on the elderly person's emotional state. As input, the label of the emotional state is provided. Based on this label, the server inputs a prompt sentence into the generative AI model to generate an appropriate response. As output, the generated response text is obtained.

[1430] Step 4:

[1431] The server converts the generated response text into voice data and speaks it to the elderly person through the speech ChatBot. The response text is provided as input. The server converts the text into voice using voice reproduction technology. Voice data is obtained as output.

[1432] Step 5:

[1433] The server analyzes the conversation patterns and frequency and assesses the dementia risk using the DementiaRiskAssessment module. As input, the conversation history is provided. The server analyzes this data and assesses the dementia risk. As output, the risk assessment result is obtained.

[1434] Step 6:

[1435] The server uses a notification service to send notifications to family members when the dementia risk assessment result exceeds a certain risk. As input, the risk assessment result is provided. The server generates notification content and sends the notification to the family contact. As output, it confirms that the notification has been sent.

[1436] Step 7:

[1437] The server periodically summarizes the conversation history and reports it to the family. As input, the conversation history is provided. The server uses a summarization algorithm to summarize the long conversation content concisely. As output, the summarized conversation content is obtained and reported to the family.

[1438] Example 3

[1439] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[1440] There is a need to assess the loneliness and emotional state of elderly people in real time and provide appropriate information to their families. However, with conventional systems, it was difficult to accurately assess the emotions and loneliness of elderly people and quickly report them to their families. In addition, there was a lack of means to effectively collect and analyze conversation data with elderly people and encourage appropriate responses. This made it difficult to provide appropriate support to maintain the mental health of elderly people.

[1441] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1442] In this invention, the server includes a means for collecting conversation data with the elderly person, a means for transmitting the collected conversation data to the server, a means for analyzing the conversation data and evaluating the sense of loneliness, a means for evaluating the emotional state, a means for storing the evaluation result in a database, and a means for notifying the family of the evaluation result. This makes it possible to evaluate the sense of loneliness and the emotional state of the elderly person in real time and to report the results to the family quickly and accurately.

[1443] "Single-person elderly" refers to elderly people who live alone.

[1444] A "speech system" refers to a system that allows humans to communicate through voice or text.

[1445] "Family personal information" refers to information about family members, such as their names, addresses, and contact details.

[1446] "Conversational habits" refers to a particular person's way of speaking and using words.

[1447] "Means for adjusting" refers to the way in which the system changes and optimizes certain conditions or parameters.

[1448] "Means for generating and speaking conversation" refers to a method by which the system automatically creates conversation content and outputs it as voice or text.

[1449] "Means of summarizing conversation history and content" refers to a method of organizing past conversation data, extracting important points, and summarizing them concisely.

[1450] "Voice reproduction technology" refers to technology that imitates and generates the voice of a specific person.

[1451] "Means of collecting conversation data" refers to methods of recording conversations with seniors and storing that data.

[1452] "Means of transmitting to the server" refers to the method of transferring collected data to the server via a network.

[1453] "Means for analyzing conversation data" refers to methods for processing collected conversation data and understanding its meaning and sentiment.

[1454] "Measures for assessing loneliness" refers to methods for measuring and assessing the loneliness of older adults from conversation data.

[1455] "Means for assessing emotional state" refers to a method for analyzing the emotions of elderly people from conversation data and assessing their state.

[1456] "Means of storing in a database" refers to the method of recording and storing the analysis results and evaluation results in a database.

[1457] "Means of notifying family members" refers to the method by which the results of the evaluation will be communicated to family members.

[1458] This invention relates to a system for evaluating the loneliness and emotional state of elderly people and reporting it to their families. Specifically, this invention is implemented using hardware and software such as a speech system, an emotion engine, a database, and a notification system.

[1459] Hardware and Software Used

[1460] Hardware: Servers, terminals (devices used by the elderly)

[1461] Software: speech system, emotion engine, database, notification system

[1462] System Overview

[1463] This system assesses the elderly person's sense of loneliness and emotional state and reports back to their family members using the following steps:

[1464] 1. Collecting conversation data with elderly people

[1465] The terminal is a device used by elderly people and has a microphone and text input function.

[1466] The device collects conversations with elderly people in real time.

[1467] For example, if an elderly person says, "I haven't talked to anyone today," the device will collect this voice data.

[1468] 2. Sending conversation data to the server

[1469] The terminal transmits the collected conversation data to a server via the Internet.

[1470] Data is encrypted and transmitted using secure communication protocols (e.g. HTTPS).

[1471] 3. Analysis of conversation data

[1472] The server passes the received conversation data to the speech system.

[1473] The speech system uses natural language processing (NLP) technology to convert conversational content into text and analyze it.

[1474] For example, a statement such as "I haven't talked to anyone today" is analyzed as text data.

[1475] 4. Assessment of loneliness

[1476] The server evaluates the elderly person's sense of loneliness based on the results of the speech system's analysis.

[1477] The survey checks indicators such as whether elderly people frequently use words that express loneliness and whether the frequency of conversations is decreasing.

[1478] For example, the statement "I'm not talking to anyone" is evaluated as an indicator of loneliness.

[1479] 5. Evaluating emotional state

[1480] The server uses an emotion engine to assess the emotional state of the senior citizen.

[1481] The emotion engine extracts emotions from the elderly's speech and determines their emotional state, such as sadness or anxiety.

[1482] For example, the word "worry" is recognized by the emotion engine as an expression of anxiety.

[1483] 6. Saving the evaluation results to a database

[1484] The server stores the evaluation results of loneliness and emotional state in a database.

[1485] The data stored includes the date and time of the evaluation, the content of the evaluation, specific comments, etc.

[1486] For example, data such as "October 10, 2023, loneliness rating: high, anxiety rating: medium" would be saved.

[1487] 7. Notifying Family Members

[1488] The server passes the evaluation results to the notification system.

[1489] The notification system will report the evaluation results to family members via email or app notifications.

[1490] For example, the family may be notified with something like, "Your mother seems to be feeling lonely lately. Specific comment: 'I haven't talked to anyone today.'"

[1491] Examples

[1492] For example, if an elderly person says, "I haven't talked to anyone today," the speech system will interpret this as an indicator of loneliness. If the elderly person frequently uses words like "worry," the emotion engine will determine that the elderly person is feeling anxious. This information is then reported to the family, who can understand the elderly person's condition and take appropriate action.

[1493] Examples of prompt statements

[1494] "Assess what emotions older adults have expressed in recent conversations and determine whether they are feeling lonely or anxious."

[1495] This system makes it possible to evaluate the elderly person's sense of loneliness and emotional state in real time and report it to their family members, thereby encouraging them to take appropriate measures. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1496] Step 1:

[1497] The device collects conversation data with elderly people.

[1498] Specifically, the device uses a microphone and text input function to record what the elderly person says in real time.

[1499] Input: Senior's voice or text data

[1500] Output: Collected conversation data (audio file or text data)

[1501] Step 2:

[1502] The terminal transmits the collected conversation data to the server.

[1503] Specifically, the terminal encrypts data transmitted over the Internet and transmits it to the server using a secure communication protocol (e.g., HTTPS).

[1504] Input: Collected conversation data

[1505] Output: Conversation data sent to the server

[1506] Step 3:

[1507] The server passes the received conversation data to the speech system.

[1508] Specifically, the server inputs conversation data into the speech system, converts it into text using natural language processing (NLP) technology, and performs analysis.

[1509] Input: Conversation data sent to the server

[1510] Output: Conversation data converted to text

[1511] Step 4:

[1512] The server evaluates the elderly person's sense of loneliness based on the results of the speech system's analysis.

[1513] Specifically, the server checks the frequency of words and conversations that express loneliness in the text data and calculates an index of loneliness.

[1514] Input: Conversation data converted to text

[1515] Output: Loneliness evaluation result

[1516] Step 5:

[1517] The server uses an emotion engine to assess the emotional state of the senior citizen.

[1518] Specifically, the server extracts emotions from the text data and determines the emotional state, such as sadness or anxiety.

[1519] Input: Conversation data converted to text

[1520] Output: Emotional state evaluation result

[1521] Step 6:

[1522] The server stores the evaluation results of loneliness and emotional state in a database.

[1523] Specifically, the server records data such as the evaluation date and time, the evaluation content, and specific comments in a database.

[1524] Input: Loneliness assessment results, emotional state assessment results

[1525] Output: Evaluation results stored in a database

[1526] Step 7:

[1527] The server passes the evaluation results to the notification system.

[1528] Specifically, the server enters the evaluation results into a notification system and reports them to family members via email, app notifications, etc.

[1529] Input: Evaluation results stored in the database

[1530] Output: Evaluation results communicated to family members

[1531] (Application example 3)

[1532] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1533] The challenge is to ensure the safety and security of the elderly by properly assessing their loneliness and emotional state and quickly notifying their family and nursing care services. In particular, when an elderly person living alone feels lonely or anxious, there is a need for a means to quickly convey that information to their family. There is also a need for a system that can accurately analyze the content of the elderly's comments and their emotional state and encourage appropriate responses.

[1534] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means. In this invention, the server includes a means for providing a speech ChatBot for interacting with the elderly person living alone, a means for adjusting the personal information and conversation habits of the family, a means for generating and speaking a conversation on behalf of the elderly person, a means for summarizing the conversation history and contents with the elderly person and reporting them to the family, a means for reproducing the voice of the elderly person using voice reproduction technology, a means for evaluating the elderly person's sense of loneliness, a means for evaluating the elderly person's emotional state and notifying the family, a means including an application to be installed on a smartphone, a natural language processing means for analyzing the contents of the elderly person's remarks, a means for evaluating the elderly person's emotional state using an emotion analysis engine, and a means for notifying the family of the evaluation result using a notification service. This makes it possible to accurately evaluate the elderly person's sense of loneliness and emotional state and quickly notify the family member or care service.

[1535] "Speech ChatBot" is an artificial intelligence system that engages in voice conversations to interact with the elderly.

[1536] "Family personal information" is information about the elderly person's family members, including names, ages, relationships, etc.

[1537] "Methods for adjusting conversational habits" are techniques for imitating the speaking styles and language used by family members when conversing with elderly people.

[1538] "Means for generating and speaking a conversation" refers to technology for automatically generating a dialogue with an elderly person and speaking it aloud.

[1539] "Means of summarizing conversation history and content and reporting it to family members" refers to technology for recording the content of conversations with elderly people, summarizing it, and notifying family members.

[1540] "Voice reproduction technology" is a technology that imitates and reproduces the voice of a specific person.

[1541] The "means for assessing loneliness" is a technology for detecting and assessing loneliness from the content of elderly people's statements and frequency of conversations.

[1542] "Means for assessing emotional state and notifying family members" refers to technology for analyzing the emotions of elderly people and notifying family members of the results.

[1543] An "application installed on a smartphone" is a software program that runs on a smartphone.

[1544] "Natural language processing means" refers to technology for analyzing voice and text data and understanding their meaning.

[1545] A "sentiment analysis engine" is a technology for analyzing emotions from text and voice data.

[1546] "Notification service" refers to technology for notifying specific recipients of evaluation results or important information.

[1547] The system for implementing this invention is configured as follows. First, the server provides a speech ChatBot for interacting with elderly people who live alone. The speech ChatBot is used to evaluate the sense of loneliness and emotional state through conversation with elderly people. The server also has a means for adjusting the personal information and conversation habits of family members, and has a function for generating and speaking conversations with elderly people.

[1548] Hardware and Software Used

[1549] Hardware: Smartphone, microphone, internet connection

[1550] Software: Natural language processing libraries (e.g. Google Cloud Natural Language API), sentiment analysis engines (e.g. IBM Watson Tone Analyzer), notification services (e.g. messaging services)

[1551] Data processing and calculation

[1552] 1. Getting voice input:

[1553] The user (elderly person) inputs voice using the microphone on the smartphone.

[1554] The server uses a speech recognition API (e.g. Google Cloud Speech-to-Text) to convert the voice data into text.

[1555] 2. Natural Language Processing:

[1556] The server analyzes the acquired text data using a natural language processing library.

[1557] Extract the content, frequency, and emotional expressions of elderly people's statements.

[1558] 3. Sentiment analysis:

[1559] The server uses a sentiment analysis engine to assess the emotional state of the senior citizen.

[1560] Determine whether the elderly person is expressing emotions such as "worried" or "sad."

[1561] 4. Loneliness assessment:

[1562] The server evaluates the elderly person's sense of loneliness based on the content and frequency of their comments and their emotional expressions.

[1563] An index of loneliness is calculated, and if it exceeds a certain threshold, it is determined that the sense of loneliness is high.

[1564] 5. Notice:

[1565] The server notifies the evaluation results of loneliness and emotional state to family members and care services.

[1566] Use the notification service to send push notifications to your smartphone.

[1567] Examples

[1568] If an elderly person says, "I haven't talked to anyone lately," the server will pick up on this statement and determine that the person is feeling very lonely.

[1569] If an elderly person says "I'm worried," the server will use a sentiment analysis engine to capture the statement and determine that the elderly person is feeling anxious.

[1570] Examples of prompt statements

[1571] Please enter what the senior said:

[1572] "I haven't talked to anyone lately."

[1573] "anxiety"

[1574] In this way, the server can accurately assess the elderly person's loneliness and emotional state and quickly notify family members and care services.

[1575] The flow of the specific process in the application example 3 will be described with reference to FIG.

[1576] Step 1:

[1577] The user (elderly person) inputs voice using the microphone of the smartphone. The input voice data is sent to the smartphone.

[1578] Step 2:

[1579] The terminal (smartphone) uses a speech recognition API (e.g. Google Cloud Speech-to-Text) to convert voice data into text. It sends the voice data to the API and obtains the text data. The input is voice data and the output is text data.

[1580] Step 3:

[1581] The server analyzes the acquired text data using a natural language processing library (e.g., Google Cloud Natural Language API). The text data is sent to the API, which extracts the content of the speech, frequency, and emotional expression. The input is the text data, and the output is the analysis results.

[1582] Step 4:

[1583] The server uses a sentiment analysis engine (e.g., IBM Watson Tone Analyzer) to evaluate the emotional state of the elderly person. The analysis result is sent to the sentiment analysis engine to evaluate the emotional state. The input is the analysis result, and the output is the emotion evaluation result.

[1584] Step 5:

[1585] The server evaluates the elderly person's sense of loneliness based on the content and frequency of their comments and their emotional expressions. It calculates an index of loneliness based on the emotion evaluation result, and judges the sense of loneliness to be high if it exceeds a certain threshold. The input is the emotion evaluation result, and the output is the loneliness evaluation result.

[1586] Step 6:

[1587] The server notifies the family or care service of the evaluation results of loneliness and emotional state. It uses a notification service (e.g., messaging service) to send a push notification to the smartphone. The input is the loneliness evaluation result and the emotion evaluation result, and the output is the notification message.

[1588] Step 7:

[1589] Family members or care services can receive notification messages via smartphones and understand the elderly person's feelings of loneliness and emotional state. They can take appropriate action based on the notification messages. The input is the notification message, and the output is the corresponding action.

[1590] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1591] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1592] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1593] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1594] [Third embodiment]

[1595] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1596] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1597] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1598] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1599] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1600] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1601] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1602] Fig. 6 shows an example of main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1603] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1604] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1605] In the headset type terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1606] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1607] "Example 1"

[1608] As an example embodiment of the present invention, a speech ChatBot for interacting with elderly people who live alone is provided. This ChatBot is fine-tuned using a machine learning algorithm for learning personal information and conversational habits of family members. Specifically, the family's conversation data is used as learning data, its features are extracted, and conversations are generated and spoken based on that. In addition, the conversation history and content with the elderly are summarized using natural language processing technology and reported to the family. Furthermore, a voice reproduction AI learns the elderly's voice data, extracts its features, and reproduces the elderly's voice. This allows the elderly to feel as if they are interacting with their family. "Example 2"

[1609] In another embodiment of the present invention, the speech ChatBot has a function to evaluate the dementia risk of elderly people. Specifically, the ChatBot detects signs of dementia from the content and frequency of conversations with elderly people, conversation patterns, etc. For example, if an elderly person shows symptoms such as repeating the same things, having difficulty in everyday conversations, or having difficulty finding words, the ChatBot evaluates this as a risk of dementia and reports it to the family.

[1610] "Example 3"

[1611] In another embodiment of the present invention, the speech ChatBot has a function to evaluate the elderly person's sense of loneliness. Specifically, the sense of loneliness is detected from the content and frequency of conversations with the elderly person, emotional expressions, etc. For example, if the elderly person frequently uses words expressing loneliness, does not express emotions much, or talks less frequently, the ChatBot evaluates these as indicators of loneliness and reports them to the family.

[1612] The process flow of each embodiment will be described below.

[1613] "Example 1"

[1614] Step 1: The speech ChatBot uses machine learning to learn the personal information and conversational habits of family members.

[1615] Start fine-tuning using the learning algorithm.

[1616] Step 2: Use family conversation data as learning data and extract its features.

[1617] Step 3: Generate and speak a conversation based on the extracted features.

[1618] Step 4: Summarize the conversation history and content with the elderly person using natural language processing technology.

[1619] Step 5: Report the summary to your family.

[1620] Step 6: The voice reproduction AI learns the voice data of the elderly and extracts its features.

[1621] Step 7: Reproduce the elderly person's voice using the extracted features.

[1622] "Example 2"

[1623] Step 1: The speech ChatBot will begin assessing the dementia risk of the elderly. Step 2: Detect signs of dementia from the content, frequency, and conversation patterns of conversations with the elderly.

[1624] Step 3: Report the dementia risk assessment results to family members.

[1625] "Example 3"

[1626] Step 1: The speech ChatBot starts the function of assessing the elderly person's loneliness.

[1627] Step 2: Detect loneliness based on the content and frequency of conversations with elderly people, as well as emotional expressions.

[1628] Step 3: Report the results of the assessment as an index of loneliness to family members.

[1629] Example 1

[1630] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1631] There is a need for a system that allows elderly people living alone to continue to interact with their families without feeling lonely. It is also important to evaluate the elderly's dementia risk and loneliness and provide appropriate information to their families. However, the current system is not able to fully resolve these issues.

[1632] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1633] In this invention, the server includes a means for collecting family conversation data, a means for pre-processing the collected data, a means for fine-tuning a machine learning model, a means for generating a conversation with the elderly person, a means for reproducing the voice of the elderly person, a means for summarizing a conversation history with the elderly person, and a means for reporting the summarized conversation history to the family member. This enables the elderly person living alone to have a natural conversation with the family member, reduce loneliness, and provide appropriate information to the family member through an evaluation of dementia risk and loneliness.

[1634] "Family conversation data" refers to text messages and voice data exchanged between family members and elderly people.

[1635] "Preprocessing" refers to the process of removing unnecessary information from collected data and converting it into a form suitable for machine learning models.

[1636] "Fine-tuning a machine learning model" refers to a technique for improving the performance of an existing machine learning model by performing additional training using a specific dataset.

[1637] "Conversation generation" refers to the process of automatically generating natural-sounding conversations using machine learning models.

[1638] "Voice reproduction" refers to the technology of reproducing a specific person's voice based on collected voice data.

[1639] "Conversation history summary" refers to the process of summarizing conversations with elderly people and extracting key points.

[1640] "Reporting to family" refers to the act of notifying family members of the summarized conversation history.

[1641] This invention relates to a speech ChatBot system for interacting with elderly people who live alone. This system collects family conversation data, generates natural conversations with elderly people using machine learning models, and reports the conversation history to the family.

[1642] Hardware and Software Used

[1643] The server uses the following hardware and software:

[1644] Hardware: A server machine equipped with a high-performance CPU and GPU

[1645] Software: Python, TensorFlow, PyTorch, NLTK, audio processing libraries

[1646] The terminal is a device used by the elderly and uses the following hardware and software:

[1647] Hardware: Microphone, speaker, tablet or smartphone

[1648] Software: Voice recognition software, voice synthesis software

[1649] Data processing and calculation

[1650] The server collects conversation data of family members. The collected data includes text messages and voice data. For example, messages sent by family members using smartphones or computers and recorded data of phone calls are stored in cloud storage.

[1651] Next, the server preprocesses the collected data. Specifically, it cleans the text data (removes unnecessary spaces and special characters) and removes noise from the audio data. It uses the Python library NLTK to tokenize the text data and a speech processing library to remove noise from the audio data.

[1652] The server then uses the preprocessed data to fine-tune a machine learning model. The model used is, for example, OpenAI's GPT-3 or Google's BERT. Family conversation data is input into the model, and the model is trained to learn the characteristics of family conversations.

[1653] ChatBot uses the fine-tuned model to generate conversations with the elderly. For example, if an elderly person asks, "What shall we do today?", ChatBot responds, "The weather is nice today, so how about going for a walk?"

[1654] In addition, the server learns the voice data of the elderly and extracts its features, which allows the ChatBot to reproduce the voice of the elderly. Using voice reproduction AI, it learns the features of the elderly's voice and speaks the generated text in the elderly's voice.

[1655] The server summarizes the conversation history with the elderly person using natural language processing technology, extracts important points and topics of the conversation, and generates a concise summary. The server summarizes the conversation content using tools such as the BERT summarizer.

[1656] Finally, the server reports the summarized conversation history to the family. It notifies the family of the elderly person's situation and the contents of the conversation via email or a dedicated app. The summarized text is sent as the body of the email.

[1657] Examples

[1658] For example, when a ChatBot wants to have a conversation with an elderly person about "today's weather," it inputs the following prompt sentence into the generative AI model.

[1659] Example prompt:

[1660] "Good morning. How's the weather today?"

[1661] Based on this prompt, ChatBot will generate the following conversation:

[1662] An example of the generated conversation:

[1663] ChatBot: Good morning. It's a sunny day today. How about going for a walk?

[1664] Elderly person: Yes, I think I'll go outside for the first time in a while.

[1665] In this way, ChatBot can have natural conversations with the elderly and deepen their interactions. The conversation history is summarized by the server and reports to the family, "It's sunny today, so the elderly are going to go for a walk."

[1666] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1667] Step 1:

[1668] Data collection

[1669] The server collects family conversation data. As input, it receives text messages and recorded calls sent by family members using their smartphones and computers. These data are stored in cloud storage. As output, it obtains the collected text and audio data.

[1670] Step 2:

[1671] Data Preprocessing

[1672] The server preprocesses the collected data. As input, it receives the text data and audio data collected in step 1. Specifically, it cleans the text data (removes unnecessary spaces and special characters) and removes noise from the audio data. It tokenizes the text data using the Python library NLTK and removes noise from the audio data using an audio processing library. As output, it obtains the preprocessed text data and audio data.

[1673] Step 3:

[1674] Fine-tuning machine learning models

[1675] The server uses the preprocessed data to fine-tune the machine learning model. As input, it receives the text data and audio data preprocessed in step 2. Specifically, it uses models such as OpenAI's GPT-3 and Google's BERT, inputs family conversation data into the model, and trains it. As output, it obtains a fine-tuned machine learning model.

[1676] Step 4:

[1677] Conversation Generation

[1678] ChatBot uses the fine-tuned model to generate conversations with elderly people. As input, it receives questions and utterances from the user (elderly person). In concrete terms, it inputs prompt sentences into the generative AI model and generates natural conversations. For example, if an elderly person asks, "What shall we do today?", ChatBot responds with, "The weather is nice today, so how about going for a walk?" The generated conversation is obtained as output.

[1679] Step 5:

[1680] Audio Reproduction

[1681] The server learns the voice data of the elderly and extracts its features. As input, it receives the voice data preprocessed in step 2. Specifically, it uses a voice reproduction AI to learn the features of the elderly's voice and speaks the generated text in the elderly's voice. As output, a voice reproduced in the elderly's voice is obtained.

[1682] Step 6:

[1683] Conversation History Summary

[1684] The server summarizes the conversation history with the elderly person using natural language processing technology. As input, it receives the conversation history generated in step 4. Specifically, it summarizes the conversation content using tools such as the BERT summarizer and extracts important points and topics. As output, it obtains the summarized conversation history.

[1685] Step 7:

[1686] Reporting to family

[1687] The server reports the summarized conversation history to the family. As input, it receives the summarized conversation history from step 6. Specifically, it notifies the family of the elderly person's situation and the contents of the conversation via email or a dedicated app. As output, it obtains the report sent to the family.

[1688] (Application example 1)

[1689] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1690] In modern society, the increase in the number of elderly people living alone is becoming a serious problem. Elderly people living alone are particularly prone to feeling lonely and being at higher risk of dementia, and reduced interaction with family members can have a negative impact on their mental and physical health. In addition, if family members live far away, it is difficult to keep track of the elderly's condition. To solve these problems, it is necessary to provide an environment where the elderly can live with peace of mind, and a system that allows family members to properly keep track of the elderly's condition.

[1691] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1692] In this invention, the server includes a means for providing a speech ChatBot for communicating with elderly people living alone, a means for fine-tuning personal information and conversation habits of family members, a means for generating and speaking conversations on behalf of the elderly person, a means for summarizing conversation history and content with the elderly person and reporting it to the family, a means for reproducing the elderly person's voice using a voice reproduction AI, a means for voice recognition of the elderly person's speech, a means for converting the generated conversation into voice and playing it back, and a means for detecting abnormalities and notifying family members and emergency contacts. This allows the elderly person to feel as if they are interacting with their family without feeling lonely, and further allows the family member to properly grasp the elderly person's condition and quickly respond if an abnormality occurs.

[1693] "Single-person elderly" refers to elderly people who live alone.

[1694] A "Speech ChatBot" refers to artificial intelligence that can converse with humans through voice.

[1695] "Family personal information" refers to information about family members such as names, ages, and relationships.

[1696] "Conversational habits" refers to the way a particular person speaks or uses language.

[1697] "Fine-tuning" refers to the fine-tuning of an existing machine learning model to adapt it to a specific dataset.

[1698] "Means for generating and speaking a conversation" refers to technology that creates an appropriate response based on input information and outputs it as voice.

[1699] "Conversation history" refers to a record of past conversations.

[1700] A "summary" refers to summarizing a longer piece of text or the contents of a conversation.

[1701] "Voice reproduction AI" refers to artificial intelligence technology that generates a voice that imitates a specific person's voice.

[1702] "Speech recognition" refers to the technology that converts speech into text.

[1703] "Means for converting to audio and playing it back" refers to technology for converting text data into audio data and playing it back.

[1704] "Means of detecting abnormalities and notifying family members or emergency contacts" refers to technology that detects abnormalities in an elderly person's condition and notifies family members or designated contacts.

[1705] As an embodiment of the present invention, we provide a ChatBot system for communicating with elderly people who live alone. The system is configured using the following hardware and software.

[1706] Hardware

[1707] Smartphone: Equipped with a microphone and speaker for audio input and output.

[1708] Server: A powerful computer that processes and stores data.

[1709] software

[1710] Python: A programming language used to implement programs.

[1711] OpenAI API: An API for generating conversations using generative AI models.

[1712] Speech Recognition Library: A library for converting speech to text.

[1713] gTTS (Google Text-to-Speech): A library for converting text to speech.

[1714] Playback software: Software for playing audio files.

[1715] System Operation

[1716] 1. Voice recognition: The device (smartphone) uses a microphone to record the elderly person's speech and converts the speech into text using a voice recognition library.

[1717] 2. Conversation generation: The server uses the OpenAI API to generate appropriate responses using a generative AI model that has learned the family's personal information and conversational habits.

[1718] 3. Audio playback: The device uses gTTS to convert the generated text into audio and plays it over the speaker.

[1719] 4. Conversation summary: The server summarizes the conversation history and content with the elderly person and reports it to the family.

[1720] 5. Anomaly detection: If the server detects an abnormality, it will notify family members or emergency contacts.

[1721] Examples

[1722] For example, if an elderly person says, "The weather is nice today," the system operates as follows.

[1723] 1. The smartphone microphone records the elderly person's speech and uses a speech recognition library to convert it into text such as "The weather is nice today."

[1724] 2. The server uses the OpenAI API to input the prompt sentence, "Conversation with an elderly person: The weather is nice today. ChatBot's response, having learned the conversational habits of family members:" into the generative AI model and generates an appropriate response.

[1725] 3. The generated response (e.g., "Yes, today is a perfect day to go for a walk") is converted to speech using gTTS and played over the smartphone's speaker.

[1726] 4. The server summarizes the conversation and notifies the family that "The weather is nice today" was said.

[1727] 5. If an abnormality is detected, the server will notify family members or emergency contacts.

[1728] Examples of prompt statements

[1729] Conversation with an Elderly Person: The weather is nice today

[1730] The ChatBot's response, having learned the family's conversational habits:

[1731] In this way, the form for carrying out the invention can be concretely shown.

[1732] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1733] Step 1:

[1734] The device (smartphone) uses a microphone to record the elderly person's speech. The input is the elderly person's voice, and the output is the recorded voice data. This voice data is passed to a voice recognition library.

[1735] Step 2:

[1736] The terminal converts the recorded voice data into text using a voice recognition library. The input is voice data and the output is text data. This text data is sent to the server.

[1737] Step 3:

[1738] Based on the received text data, the server inputs a prompt sentence into the generative AI model using the OpenAI API. The input is text data, and the output is the generated response text. An example of a prompt sentence is "Conversation with an elderly person: The weather is nice today. ChatBot's response based on learning the conversational habits of family members:"

[1739] Step 4:

[1740] The server sends the generated response text to the terminal. The input is the generated response text and the output is the response text sent to the terminal.

[1741] Step 5:

[1742] The device converts the received response text into speech using gTTS. The input is the response text, and the output is audio data. This audio data is played on the smartphone speaker.

[1743] Step 6:

[1744] The server summarizes the conversation history and contents with the elderly person and reports them to the family. The input is the text data of the conversation, and the output is the summarized report data. This report data is notified to the family.

[1745] Step 7:

[1746] If the server detects an abnormality, it notifies the family and emergency contacts. The input is data for detecting the abnormality, and the output is notification data. This notification data is sent to the family and emergency contacts.

[1747] Example 2

[1748] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1749] There is a need to detect the risk of dementia and loneliness in the elderly at an early stage and take appropriate measures. However, conventional methods often lack communication with the elderly, and risk detection is often delayed. In addition, there are limited means for families to understand the condition of the elderly. This makes it difficult to prevent the deterioration of the elderly's health.

[1750] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1751] In this invention, the server includes a means for providing a speech bot for communicating with elderly people living alone, a means for adjusting personal information and conversation habits of family members, a means for generating and speaking a conversation on behalf of the elderly person, a means for summarizing the conversation history and content with the elderly person and reporting it to the family, a means for reproducing the voice of the elderly person using voice reproduction technology, a means for recording the conversation content in real time and transmitting it to the server, a means for storing the received conversation data in a database, a means for evaluating the conversation content using a generative AI model for analyzing the conversation data in the database, a means for calculating the dementia risk based on the evaluation result of the generative AI model, and a means for notifying the family member of the result if the dementia risk is determined to be high. This makes it possible to detect the dementia risk and loneliness of elderly people early and notify the family member promptly.

[1752] A "talkbot" is a conversational artificial intelligence system designed to interact with the elderly.

[1753] "Family personal information" refers to basic information about an elderly person's family members, including names, contact details, and relationships.

[1754] "Conversational habits" refers to the words, phrases and speaking patterns that older people use on a daily basis.

[1755] "Means of adjustment" refers to ways in which the system can optimize its settings and operation to suit the characteristics of the elderly person and their family.

[1756] "Means of generating and speaking conversation" refers to the system's ability to automatically create conversation content and output it as audio.

[1757] "Conversation history" refers to records of past conversations between the elderly person and the speech bot.

[1758] "Summarization techniques" refer to methods for summarizing long conversations and extracting important information.

[1759] "Voice reproduction technology" refers to technology that imitates and generates the voice of a specific person.

[1760] "Real-time recording means" refers to a method of recording the content of a conversation as it occurs.

[1761] "Server" refers to a computer system for storing, processing, and managing data.

[1762] "Database" refers to a system for efficiently storing, retrieving, and managing structured data.

[1763] A "generative AI model" refers to a model that uses artificial intelligence technology to analyze data and extract specific patterns and information.

[1764] "Dementia risk" refers to an indicator used to assess the likelihood that an elderly person will develop dementia.

[1765] "Means of notification" refers to the method by which specific information is communicated to families.

[1766] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1767] This invention is a system for assessing the risk of dementia and loneliness in elderly people and notifying their families. This system is realized using hardware and software such as a speech bot, a server, a generative AI model, and a database.

[1768] Hardware and Software Used

[1769] Hardware: Servers, terminals (smartphones, tablets, PCs, etc.)

[1770] Software: Talkbots, generative AI models (e.g. GPT-4), database management systems (e.g. MySQL)

[1771] System Overview

[1772] The user uses a device to access the speech bot and begin a conversation. The speech bot records the content of the conversation in real time and sends it to the server. The server stores the received conversation data in a database and periodically analyzes the content of the conversation using a generative AI model. The generative AI model detects signs of dementia, and the server calculates the dementia risk based on the evaluation results. If the risk is determined to be high, the server notifies the family of the result.

[1773] Examples

[1774] For example, if a user had the following conversation with a speech bot:

[1775] User: "I can't remember what I had for dinner yesterday."

[1776] Speech Bot: "That's concerning. Have you been forgetful at any other times lately?"

[1777] User: "Yeah, I keep telling the same stories over and over."

[1778] This conversation data is sent to a server and analyzed by the generative AI model. The model detects signs of dementia from statements such as "I don't remember what I had for dinner yesterday" or "I keep repeating the same thing." The server uses this data to assess the risk of dementia and notify family members if necessary.

[1779] Examples of prompt statements

[1780] Here is an example of a prompt to be input to the generative AI model:

[1781] Analyze the user's conversation and evaluate whether they have signs of dementia. See the following conversation for reference:

[1782] User: "I can't remember what I had for dinner yesterday."

[1783] Speech Bot: "That's concerning. Have you been forgetful at any other times lately?"

[1784] User: "Yeah, I keep telling the same stories over and over."

[1785] By inputting this prompt into a generative AI model, the model analyzes the conversation content and assesses signs of dementia.

[1786] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1787] Step 1:

[1788] A user accesses the speech bot using a terminal.

[1789] Input: A user accesses the speech bot's application or website using a smartphone, tablet, or PC.

[1790] Output: The speechbot home screen is displayed.

[1791] Specific operation: A user opens the app and the login screen is displayed. The user enters their ID and password to log in.

[1792] Step 2:

[1793] A user starts a conversation with a speech bot.

[1794] Input: The user speaks to the speech bot.

[1795] Output: The speech bot recognizes what the user said and generates a response.

[1796] Specific behavior: After logging in, the user taps the "Start conversation" button, then taps the microphone icon to speak. For example, the user speaks to the speech bot, "Hello, how are you today?"

[1797] Step 3:

[1798] The speech bot records the conversation in real time and sends it to the server.

[1799] Input: What the user says.

[1800] Output: Sent to the server as text data.

[1801] Specific operation: The speech bot converts the user's speech into text using speech recognition technology and immediately sends it to the server. For example, the user's speech "Hello, how are you today?" is sent to the server as text data.

[1802] Step 4:

[1803] The server stores the received conversation data in a database.

[1804] Input: The text data sent to the server.

[1805] Output: Conversation data stored in a database.

[1806] Specific operation: The server saves the received text data with a timestamp in the "conversation history" table of the database. For example, the server saves the conversation data "Hello, how are you today?" in the database.

[1807] Step 5:

[1808] The server periodically analyzes the conversation data in the database and evaluates the conversation content using a generative AI model.

[1809] Input: Conversation data in the database.

[1810] Output: Evaluation results from the generative AI model.

[1811] Specific operation: The server executes a scheduled task at 2 a.m. every day and inputs the past 24 hours of conversation data into the generative AI model. For example, the server obtains the past 24 hours of conversation data and inputs it into the generative AI model as a prompt sentence.

[1812] Step 6:

[1813] Generative AI model detects signs of dementia.

[1814] Input: Conversation data entered as prompt sentences.

[1815] Output: Assessment results indicating signs of dementia.

[1816] Specific operation: The generative AI model analyzes the conversation data based on the prompt sentence and detects keywords and patterns that indicate signs of dementia. For example, the generative AI model detects signs of dementia from statements such as "I don't remember what I had for dinner yesterday" or "I keep repeating the same thing."

[1817] Step 7:

[1818] The server calculates the risk of dementia based on the evaluation results of the generated AI model.

[1819] Input: Evaluation results of a generative AI model.

[1820] Output: Dementia risk score.

[1821] How it works: The server analyzes the output of the generative AI model and runs an algorithm to calculate a risk score. For example, the server calculates a "dementia risk score" and determines whether the score exceeds a certain threshold.

[1822] Step 8:

[1823] If the server determines that there is a high risk of dementia, it notifies the family of the result.

[1824] Input: Dementia risk score.

[1825] Output: A notification message to family members.

[1826] Specific operation: If the risk score exceeds the threshold, the server generates a message to notify the family and sends it via the registered email address or app notification. For example, the server sends a message to the family saying, "Based on your father's recent conversations, we have determined that he is at high risk for dementia. We recommend a detailed evaluation."

[1827] (Application example 2)

[1828] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1829] There is a need to detect dementia risk in the elderly at an early stage and take appropriate measures. However, with the number of elderly people living alone increasing, it is difficult for family members and caregivers to keep track of the elderly's condition on a daily basis. In addition, reducing the elderly's sense of loneliness and maintaining their mental health are also important issues. To solve these issues, a system is needed that assesses the dementia risk through daily conversations with the elderly and notifies the family and caregivers as necessary.

[1830] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1831] In this invention, the server includes a means for providing a speech ChatBot for communicating with elderly people who live alone, a means for fine-tuning personal information and conversation habits of family members, a means for generating and speaking a conversation on behalf of the elderly person, a means for summarizing the conversation history and content with the elderly person and reporting it to the family, a means for reproducing the voice of the elderly person using voice reproduction artificial intelligence, a means for converting the conversation content from voice to text, a means for analyzing the conversation pattern and frequency, and a means for evaluating the dementia risk and notifying the family member or caregiver. This allows the dementia risk of the elderly person to be detected early, and the family member or caregiver to take appropriate measures. It also reduces the elderly person's sense of loneliness and maintains their mental health.

[1832] "Talk ChatBot" is a dialogue system that uses artificial intelligence to conduct voice conversations with elderly people.

[1833] "Fine-tuning" means that the system adjusts the optimal content and method of conversation based on a family member's personal information and conversational habits.

[1834] "Summarization" means concisely summarizing the conversation history and content with the elderly person.

[1835] "Voice reproduction artificial intelligence" is an artificial intelligence technology that imitates and reproduces the voice of a specific person.

[1836] "Convert to text" refers to the process of converting voice data into character data.

[1837] "Analysis" refers to analyzing conversation patterns and frequencies to detect certain trends or anomalies.

[1838] "Dementia risk" is an indicator that evaluates the degree of likelihood that an elderly person will develop dementia.

[1839] "Notification" refers to informing family members or caregivers of information detected by the system.

[1840] A system for implementing this invention has the following configuration: The server includes means for providing a speech ChatBot for communicating with elderly people living alone, means for fine-tuning personal information and conversational habits of family members, means for generating and speaking conversations on behalf of the elderly person, means for summarizing the conversation history and content with the elderly person and reporting it to the family, means for reproducing the elderly person's voice using voice reproduction AI, means for converting the content of the conversation from voice to text, means for analyzing the conversation pattern and frequency, and means for assessing the risk of dementia and notifying the family member or caregiver.

[1841] Hardware and Software Used

[1842] Hardware: Smartphone (iOS or Android)

[1843] Software: Speech recognition API (Google Cloud Speech-to-Text), natural language processing library (spaCy), notification service (messaging service)

[1844] Data processing and calculation

[1845] 1. Audio recording and transcription:

[1846] Conversations with elderly people are recorded using the microphone on the device (smartphone).

[1847] The recorded audio data is converted to text using the Google Cloud Speech-to-Text API.

[1848] 2. Conversation Analysis:

[1849] The server analyzes the converted text data using a natural language processing library (spaCy).

[1850] Analyze conversation patterns, frequency and content to detect signs of dementia.

[1851] 3. Risk Assessment:

[1852] The server assesses dementia risk based on the symptoms detected, for example scoring how often a person repeats themselves or has difficulty finding words.

[1853] 4. Notification function:

[1854] The server uses a messaging service to send notifications to family members or caregivers if there is a high risk.

[1855] Examples

[1856] Scenario: An elderly person talks with their family members through an app every day. The device records the conversation, converts it into text, and sends it to a server. The server analyzes the content of the conversation and detects an increase in the frequency of repeating the same things. The server assesses that the risk of dementia is increasing and sends a notification to the family members to encourage early medical consultation.

[1857] Examples of prompt statements

[1858] Develop an application that records conversations with seniors, converts them into text, and analyzes them. Analyze conversation patterns and frequency to detect signs of dementia. Evaluate dementia risk based on detected signs, and send notifications to family members or caregivers if risk is high. Technologies used are Google Cloud Speech-to-Text API, spaCy, and messaging services.

[1859] In this way, we demonstrated a concrete example of implementing a talking ChatBot to assess the dementia risk of elderly people as a smartphone app.

[1860] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1861] Step 1:

[1862] The terminal records conversations with the user (elderly person). The input is the user's voice, and the output is the recorded voice data. Specifically, the terminal's microphone is used to capture the voice and save it as an audio file.

[1863] Step 2:

[1864] The device converts the recorded voice data into text using the Google Cloud Speech-to-Text API. The input is the recorded voice data, and the output is the converted text data. The specific operation is to send the audio file to the API and receive the returned text data.

[1865] Step 3:

[1866] The terminal transmits the converted text data to the server. The input is the text data, and the output is the text data transmitted to the server. In concrete terms, the text data is uploaded to the server via a network.

[1867] Step 4:

[1868] The server analyzes the received text data using a natural language processing library (spaCy). The input is text data, and the output is the analysis results. Specifically, the text data is input into spaCy, and the conversation patterns and frequency are analyzed.

[1869] Step 5:

[1870] The server evaluates the dementia risk based on the analysis results. The input is the analysis results, and the output is a dementia risk assessment score. In specific operations, the analysis results are input to an assessment algorithm to calculate a risk score.

[1871] Step 6:

[1872] The server uses a messaging service to send a notification to family members or caregivers if the dementia risk assessment score is high. The input is the dementia risk assessment score and the output is the sent notification. Specific operations include evaluating the risk score and generating and sending a notification message.

[1873] Step 7:

[1874] The server summarizes the conversation history and content with the elderly person and reports it to the family. The input is text data and analysis results, and the output is a summarized report. Specifically, the server generates a summary based on the text data and analysis results and sends it to the family.

[1875] Example 3

[1876] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1877] There is a need to detect loneliness and dementia risk in elderly people at an early stage and take appropriate measures. However, with conventional systems, it was difficult to effectively analyze the content and frequency of conversations with elderly people and their emotional expressions to evaluate loneliness and dementia risk. There was also a lack of means to notify family members at an appropriate time. This led to a high possibility that loneliness and dementia risk in elderly people would be overlooked, and appropriate measures would be delayed, which was a problem.

[1878] The specific processing by the specific processing unit 290 of the d...

Claims

1. means for providing a speech system for interacting with a user; means for adjusting the response of the speech system based on personal information and speech habits of the user's family; means for collecting conversation data between the user and the speech system; means for analyzing the collected conversation data and assessing an emotional state of the user; means for reporting a summary of the collected conversation data and an evaluation of the user's emotional state; A system including:

2. The speech system includes a means for assessing a dementia risk of the user; and a means for notifying the assessed dementia risk. The system of claim 1 .

3. The speech system includes: a means for assessing a loneliness of the user; and a means for notifying the assessed loneliness. The system of claim 1 .

Images